pig/pig-farm-controller
1
0
Fork 0
Code Issues 11 Pull Requests Actions Packages Projects Releases Wiki Activity

用户登录和接口鉴权

Browse Source
This commit is contained in:
huang
2025-09-07 21:13:15 +08:00
parent c4522b974b
commit 565cf3fa6a
380 changed files with 18330 additions and 16854 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
vendor/github.com/bytedance/sonic/.gitignore generated vendored
View File

@@ -49,7 +49,4 @@ ast/bench.sh
!testdata/*.json.gz
fuzz/testdata
*__debug_bin*
*pprof
*coverage.txt
tools/venv/*
*__debug_bin

3
vendor/github.com/bytedance/sonic/.gitmodules generated vendored
View File

@@ -4,6 +4,3 @@
[submodule "tools/simde"]
path = tools/simde
url = https://github.com/simd-everywhere/simde.git
[submodule "fuzz/go-fuzz-corpus"]
path = fuzz/go-fuzz-corpus
url = https://github.com/dvyukov/go-fuzz-corpus.git

56
vendor/github.com/bytedance/sonic/README.md generated vendored
View File

@@ -6,10 +6,9 @@ A blazingly fast JSON serializing & deserializing library, accelerated by JI
## Requirement
- Go: 1.18~1.25
- Notice: Go1.24.0 is not supported due to the [issue](https://github.com/golang/go/issues/71672), please use higher go version or add build tag `--ldflags="-checklinkname=0"`
- OS: Linux / MacOS / Windows
- CPU: AMD64 / (ARM64, need go1.20 above)
- Go 1.16~1.22
- Linux / MacOS / Windows(need go1.17 above)
- Amd64 ARCH
## Features
@@ -212,7 +211,7 @@ ret, err := Encode(v, EscapeHTML) // ret == `{"\u0026\u0026":{"X":"\u003c\u003e"
### Compact Format
Sonic encodes primitive objects (struct/map...) as compact-format JSON by default, except marshaling `json.RawMessage` or `json.Marshaler`: sonic ensures validating their output JSON but **DO NOT** compacting them for performance concerns. We provide the option `encoder.CompactMarshaler` to add compacting process.
Sonic encodes primitive objects (struct/map...) as compact-format JSON by default, except marshaling `json.RawMessage` or `json.Marshaler`: sonic ensures validating their output JSON but **DONOT** compacting them for performance concerns. We provide the option `encoder.CompactMarshaler` to add compacting process.
### Print Error
@@ -283,22 +282,6 @@ sub := root.Get("key3").Index(2).Int64() // == 3
**Tip**: since `Index()` uses offset to locate data, which is much faster than scanning like `Get()`, we suggest you use it as much as possible. And sonic also provides another API `IndexOrGet()` to underlying use offset as well as ensure the key is matched.
#### SearchOption
`Searcher` provides some options for user to meet different needs:
```go
opts := ast.SearchOption{ CopyReturn: true ... }
val, err := sonic.GetWithOptions(JSON, opts, "key")
```
- CopyReturn
Indicate the searcher to copy the result JSON string instead of refer from the input. This can help to reduce memory usage if you cache the results
- ConcurentRead
Since `ast.Node` use `Lazy-Load` design, it doesn't support Concurrently-Read by default. If you want to read it concurrently, please specify it.
- ValidateJSON
Indicate the searcher to validate the entire JSON. This option is enabled by default, which slow down the search speed a little.
#### Set/Unset
Modify the json content by Set()/Unset()
@@ -385,12 +368,16 @@ See [ast/visitor.go](https://github.com/bytedance/sonic/blob/main/ast/visitor.go
## Compatibility
For developers who want to use sonic to meet different scenarios, we provide some integrated configs as `sonic.API`
Sonic **DOES NOT** ensure to support all environments, due to the difficulty of developing high-performance codes. For developers who use sonic to build their applications in different environments, we have the following suggestions:
- `ConfigDefault`: the sonic's default config (`EscapeHTML=false`,`SortKeys=false`...) to run sonic fast meanwhile ensure security.
- `ConfigStd`: the std-compatible config (`EscapeHTML=true`,`SortKeys=true`...)
- `ConfigFastest`: the fastest config (`NoQuoteTextMarshaler=true`) to run on sonic as fast as possible.
Sonic **DOES NOT** ensure to support all environments, due to the difficulty of developing high-performance codes. On non-sonic-supporting environment, the implementation will fall back to `encoding/json`. Thus below configs will all equal to `ConfigStd`.
- Developing on **Mac M1**: Make sure you have Rosetta 2 installed on your machine, and set `GOARCH=amd64` when building your application. Rosetta 2 can automatically translate x86 binaries to arm64 binaries and run x86 applications on Mac M1.
- Developing on **Linux arm64**: You can install qemu and use the `qemu-x86_64 -cpu max` command to convert x86 binaries to amr64 binaries for applications built with sonic. The qemu can achieve a similar transfer effect to Rosetta 2 on Mac M1.
For developers who want to use sonic on Linux arm64 without qemu, or those who want to handle JSON strictly consistent with `encoding/json`, we provide some compatible APIs as `sonic.API`
- `ConfigDefault`: the sonic's default config (`EscapeHTML=false`,`SortKeys=false`...) to run on sonic-supporting environment. It will fall back to `encoding/json` with the corresponding config, and some options like `SortKeys=false` will be invalid.
- `ConfigStd`: the std-compatible config (`EscapeHTML=true`,`SortKeys=true`...) to run on sonic-supporting environment. It will fall back to `encoding/json`.
- `ConfigFastest`: the fastest config (`NoQuoteTextMarshaler=true`) to run on sonic-supporting environment. It will fall back to `encoding/json` with the corresponding config, and some options will be invalid.
## Tips
@@ -479,23 +466,6 @@ For better performance, in previous case the `ast.Visitor` will be the better ch
But `ast.Visitor` is not a very handy API. You might need to write a lot of code to implement your visitor and carefully maintain the tree hierarchy during decoding. Please read the comments in [ast/visitor.go](https://github.com/bytedance/sonic/blob/main/ast/visitor.go) carefully if you decide to use this API.
### Buffer Size
Sonic use memory pool in many places like `encoder.Encode`, `ast.Node.MarshalJSON` to improve performance, which may produce more memory usage (in-use) when server's load is high. See [issue 614](https://github.com/bytedance/sonic/issues/614). Therefore, we introduce some options to let user control the behavior of memory pool. See [option](https://pkg.go.dev/github.com/bytedance/sonic@v1.11.9/option#pkg-variables) package.
### Faster JSON Skip
For security, sonic use [FSM](native/skip_one.c) algorithm to validate JSON when decoding raw JSON or encoding `json.Marshaler`, which is much slower (1~10x) than [SIMD-searching-pair](native/skip_one_fast.c) algorithm. If user has many redundant JSON value and DO NOT NEED to strictly validate JSON correctness, you can enable below options:
- `Config.NoValidateSkipJSON`: for faster skipping JSON when decoding, such as unknown fields, json.Unmarshaler(json.RawMessage), mismatched values, and redundant array elements
- `Config.NoValidateJSONMarshaler`: avoid validating JSON when encoding `json.Marshaler`
- `SearchOption.ValidateJSON`: indicates if validate located JSON value when `Get`
## JSON-Path Support (GJSON)
[tidwall/gjson](https://github.com/tidwall/gjson) has provided a comprehensive and popular JSON-Path API, and
a lot of older codes heavily relies on it. Therefore, we provides a wrapper library, which combines gjson's API with sonic's SIMD algorithm to boost up the performance. See [cloudwego/gjson](https://github.com/cloudwego/gjson).
## Community
Sonic is a subproject of [CloudWeGo](https://www.cloudwego.io/). We are committed to building a cloud native ecosystem.

51
vendor/github.com/bytedance/sonic/README_ZH_CN.md generated vendored
View File

@@ -6,10 +6,9 @@
## 依赖
- Go: 1.18~1.25
- 注意：Go1.24.0 由于 [issue](https://github.com/golang/go/issues/71672) 不可用，请升级到更高 Go 版本，或添加编译选项 `--ldflags="-checklinkname=0"`
- OS: Linux / MacOS / Windows
- CPU: AMD64 / (ARM64, 需要 Go1.20 以上）
- Go 1.16~1.22
- Linux / MacOS / Windows（需要 Go1.17 以上）
- Amd64 架构
## 接口
@@ -261,7 +260,7 @@ fmt.Printf("%+v", data) // {A:0 B:1}
### `Ast.Node`
Sonic/ast.Node 是完全独立的 JSON 抽象语法树库。它实现了序列化和反序列化，并提供了获取和修改JSON数据的鲁棒的 API。
Sonic/ast.Node 是完全独立的 JSON 抽象语法树库。它实现了序列化和反序列化，并提供了获取和修改通用数据的鲁棒的 API。
#### 查找/索引
@@ -283,22 +282,6 @@ sub := root.Get("key3").Index(2).Int64() // == 3
**注意**：由于 `Index()` 使用偏移量来定位数据，比使用扫描的 `Get()` 要快的多，建议尽可能的使用 `Index` 。 Sonic 也提供了另一个 API， `IndexOrGet()` ，以偏移量为基础并且也确保键的匹配。
#### 查找选项
`ast.Searcher`提供了一些选项，以满足用户的不同需求:
```go
opts := ast.SearchOption{CopyReturn: true…}
val, err := sonic.GetWithOptions(JSON, opts, "key")
```
- CopyReturn
指示搜索器复制结果JSON字符串，而不是从输入引用。如果用户缓存结果，这有助于减少内存使用
- ConcurentRead
因为`ast.Node`使用`Lazy-Load`设计，默认不支持并发读取。如果您想同时读取，请指定它。
- ValidateJSON
指示搜索器来验证整个JSON。默认情况下启用该选项, 但是对于查找速度有一定影响。
#### 修改
使用 `Set()` / `Unset()` 修改 json 的内容
@@ -385,12 +368,16 @@ type Visitor interface {
## 兼容性
对于想要使用sonic来满足不同场景的开发人员，我们提供了一些集成配置:
由于开发高性能代码的困难性， Sonic **不**保证对所有环境的支持。对于在不同环境中使用 Sonic 构建应用程序的开发者，我们有以下建议：
- `ConfigDefault`: sonic的默认配置 (`EscapeHTML=false`， `SortKeys=false`…) 保证性能同时兼顾安全性。
- `ConfigStd`: 与 `encoding/json` 保证完全兼容的配置
- `ConfigFastest`: 最快的配置(`NoQuoteTextMarshaler=true...`) 保证性能最优但是会缺少一些安全性检查（validate UTF8 等）
Sonic **不**确保支持所有环境，由于开发高性能代码的困难。在不支持sonic的环境中，实现将回落到 `encoding/json`。因此上述配置将全部等于`ConfigStd`。
- 在 **Mac M1** 上开发：确保在您的计算机上安装了 Rosetta 2，并在构建时设置 `GOARCH=amd64` 。 Rosetta 2 可以自动将 x86 二进制文件转换为 arm64 二进制文件，并在 Mac M1 上运行 x86 应用程序。
- 在 **Linux arm64** 上开发：您可以安装 qemu 并使用 `qemu-x86_64 -cpu max` 命令来将 x86 二进制文件转换为 arm64 二进制文件。qemu可以实现与Mac M1上的Rosetta 2类似的转换效果。
对于希望在不使用 qemu 下使用 sonic 的开发者，或者希望处理 JSON 时与 `encoding/JSON` 严格保持一致的开发者，我们在 `sonic.API` 中提供了一些兼容性 API
- `ConfigDefault`: 在支持 sonic 的环境下 sonic 的默认配置（`EscapeHTML=false`，`SortKeys=false`等）。行为与具有相应配置的 `encoding/json` 一致，一些选项，如 `SortKeys=false` 将无效。
- `ConfigStd`: 在支持 sonic 的环境下与标准库兼容的配置（`EscapeHTML=true`，`SortKeys=true`等）。行为与 `encoding/json` 一致。
- `ConfigFastest`: 在支持 sonic 的环境下运行最快的配置（`NoQuoteTextMarshaler=true`）。行为与具有相应配置的 `encoding/json` 一致，某些选项将无效。
## 注意事项
@@ -477,18 +464,6 @@ go someFunc(user)
但是，`ast.Visitor` 并不是一个很易用的 API。你可能需要写大量的代码去实现自己的 `ast.Visitor`，并且需要在解析过程中仔细维护树的层级。如果你决定要使用这个 API，请先仔细阅读 [ast/visitor.go](https://github.com/bytedance/sonic/blob/main/ast/visitor.go) 中的注释。
### 缓冲区大小
Sonic在许多地方使用内存池，如`encoder.Encode`, `ast.Node.MarshalJSON`等来提高性能，这可能会在服务器负载高时产生更多的内存使用(in-use)。参见[issue 614](https://github.com/bytedance/sonic/issues/614)。因此，我们引入了一些选项来让用户配置内存池的行为。参见[option](https://pkg.go.dev/github.com/bytedance/sonic@v1.11.9/option#pkg-variables)包。
### 更快的 JSON Skip
为了安全起见，在跳过原始JSON 时，sonic decoder 默认使用[FSM](native/skip_one.c)算法扫描来跳过同时校验 JSON。它相比[SIMD-searching-pair](native/skip_one_fast.c)算法跳过要慢得多(1~10倍)。如果用户有很多冗余的JSON值，并且不需要严格验证JSON的正确性，你可以启用以下选项:
- `Config.NoValidateSkipJSON`: 用于在解码时更快地跳过JSON，例如未知字段，`json.RawMessage`，不匹配的值和冗余的数组元素等
- `Config.NoValidateJSONMarshaler`: 编码JSON时避免验证JSON。封送拆收器
- `SearchOption.ValidateJSON`: 指示当`Get`时是否验证定位的JSON值
## 社区
Sonic 是 [CloudWeGo](https://www.cloudwego.io/) 下的一个子项目。我们致力于构建云原生生态系统。

41
vendor/github.com/bytedance/sonic/api.go generated vendored
View File

@@ -23,16 +23,6 @@ import (
`github.com/bytedance/sonic/internal/rt`
)
const (
// UseStdJSON indicates you are using fallback implementation (encoding/json)
UseStdJSON = iota
// UseSonicJSON indicates you are using real sonic implementation
UseSonicJSON
)
// APIKind is the kind of API, 0 is std json, 1 is sonic.
const APIKind = apiKind
// Config is a combination of sonic/encoder.Options and sonic/decoder.Options
type Config struct {
// EscapeHTML indicates encoder to escape all HTML characters
@@ -77,26 +67,16 @@ type Config struct {
// CopyString indicates decoder to decode string values by copying instead of referring.
CopyString bool
// ValidateString indicates decoder and encoder to validate string values: decoder will return errors
// ValidateString indicates decoder and encoder to valid string values: decoder will return errors
// when unescaped control chars(\u0000-\u001f) in the string value of JSON.
ValidateString bool
// NoValidateJSONMarshaler indicates that the encoder should not validate the output string
// after encoding the JSONMarshaler to JSON.
NoValidateJSONMarshaler bool
// NoValidateJSONSkip indicates the decoder should not validate the JSON value when skipping it,
// such as unknown-fields, mismatched-type, redundant elements..
NoValidateJSONSkip bool
// NoEncoderNewline indicates that the encoder should not add a newline after every message
NoEncoderNewline bool
// Encode Infinity or Nan float into `null`, instead of returning an error.
EncodeNullForInfOrNan bool
// CaseSensitive indicates that the decoder should not ignore the case of object keys.
CaseSensitive bool
}
var (
@@ -114,15 +94,15 @@ var (
// ConfigFastest is the fastest config of APIs, aiming at speed.
ConfigFastest = Config{
NoQuoteTextMarshaler: true,
NoValidateJSONMarshaler: true,
NoValidateJSONSkip: true,
}.Froze()
)
// API is a binding of specific config.
// This interface is inspired by github.com/json-iterator/go,
// and has same behaviors under equivalent config.
// and has same behaviors under equavilent config.
type API interface {
// MarshalToString returns the JSON encoding string of v
MarshalToString(v interface{}) (string, error)
@@ -177,13 +157,6 @@ func Marshal(val interface{}) ([]byte, error) {
return ConfigDefault.Marshal(val)
}
// MarshalIndent is like Marshal but applies Indent to format the output.
// Each JSON element in the output will begin on a new line beginning with prefix
// followed by one or more copies of indent according to the indentation nesting.
func MarshalIndent(v interface{}, prefix, indent string) ([]byte, error) {
return ConfigDefault.MarshalIndent(v, prefix, indent)
}
// MarshalString returns the JSON encoding string of v.
func MarshalString(val interface{}) (string, error) {
return ConfigDefault.MarshalToString(val)
@@ -216,14 +189,6 @@ func Get(src []byte, path ...interface{}) (ast.Node, error) {
return GetCopyFromString(rt.Mem2Str(src), path...)
}
//GetWithOptions searches and locates the given path from src json,
// with specific options of ast.Searcher
func GetWithOptions(src []byte, opts ast.SearchOptions, path ...interface{}) (ast.Node, error) {
s := ast.NewSearcher(rt.Mem2Str(src))
s.SearchOptions = opts
return s.GetByPath(path...)
}
// GetFromString is same with Get except src is string.
//
// WARNING: The returned JSON is **Referenced** from the input.

17
vendor/github.com/bytedance/sonic/ast/api_compat.go generated vendored
View File

@@ -1,4 +1,4 @@
// +build !amd64,!arm64 go1.26 !go1.17 arm64,!go1.20
// +build !amd64,!arm64 go1.23 !go1.16 arm64,!go1.20
/*
* Copyright 2022 ByteDance Inc.
@@ -23,17 +23,28 @@ import (
`unicode/utf8`
`github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/internal/compat`
`github.com/bytedance/sonic/internal/rt`
)
func init() {
compat.Warn("sonic/ast")
println("WARNING:(ast) sonic only supports Go1.16~1.22, but your environment is not suitable")
}
func quote(buf *[]byte, val string) {
quoteString(buf, val)
}
// unquote unescapes a internal JSON string (it doesn't count quotas at the begining and end)
func unquote(src string) (string, types.ParsingError) {
sp := rt.IndexChar(src, -1)
out, ok := unquoteBytes(rt.BytesFrom(sp, len(src)+2, len(src)+2))
if !ok {
return "", types.ERR_INVALID_ESCAPE
}
return rt.Mem2Str(out), 0
}
func (self *Parser) decodeValue() (val types.JsonState) {
e, v := decodeValue(self.s, self.p, self.dbuf == nil)
if e < 0 {

97
vendor/github.com/bytedance/sonic/ast/decode.go generated vendored
View File

@@ -17,35 +17,36 @@
package ast
import (
"encoding/base64"
"runtime"
"strconv"
"unsafe"
`encoding/base64`
`runtime`
`strconv`
`unsafe`
"github.com/bytedance/sonic/internal/native/types"
"github.com/bytedance/sonic/internal/rt"
"github.com/bytedance/sonic/internal/utils"
"github.com/bytedance/sonic/unquote"
`github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/internal/rt`
)
var bytesNull = []byte("null")
const _blankCharsMask = (1 << ' ') | (1 << '\t') | (1 << '\r') | (1 << '\n')
const (
strNull = "null"
bytesNull = "null"
bytesTrue = "true"
bytesFalse = "false"
bytesObject = "{}"
bytesArray = "[]"
)
func isSpace(c byte) bool {
return (int(1<<c) & _blankCharsMask) != 0
}
//go:nocheckptr
func skipBlank(src string, pos int) int {
se := uintptr(rt.IndexChar(src, len(src)))
sp := uintptr(rt.IndexChar(src, pos))
for sp < se {
if !utils.IsSpace(*(*byte)(unsafe.Pointer(sp))) {
if !isSpace(*(*byte)(unsafe.Pointer(sp))) {
break
}
sp += 1
@@ -62,7 +63,7 @@ func decodeNull(src string, pos int) (ret int) {
if ret > len(src) {
return -int(types.ERR_EOF)
}
if src[pos:ret] == strNull {
if src[pos:ret] == bytesNull {
return ret
} else {
return -int(types.ERR_INVALID_CHAR)
@@ -102,13 +103,13 @@ func decodeString(src string, pos int) (ret int, v string) {
return ret, v
}
result, err := unquote.String(src[pos:ret])
if err != 0 {
vv, ok := unquoteBytes(rt.Str2Mem(src[pos:ret]))
if !ok {
return -int(types.ERR_INVALID_CHAR), ""
}
runtime.KeepAlive(src)
return ret, result
return ret, rt.Mem2Str(vv)
}
func decodeBinary(src string, pos int) (ret int, v []byte) {
@@ -286,7 +287,67 @@ func decodeValue(src string, pos int, skipnum bool) (ret int, v types.JsonState)
//go:nocheckptr
func skipNumber(src string, pos int) (ret int) {
return utils.SkipNumber(src, pos)
sp := uintptr(rt.IndexChar(src, pos))
se := uintptr(rt.IndexChar(src, len(src)))
if uintptr(sp) >= se {
return -int(types.ERR_EOF)
}
if c := *(*byte)(unsafe.Pointer(sp)); c == '-' {
sp += 1
}
ss := sp
var pointer bool
var exponent bool
var lastIsDigit bool
var nextNeedDigit = true
for ; sp < se; sp += uintptr(1) {
c := *(*byte)(unsafe.Pointer(sp))
if isDigit(c) {
lastIsDigit = true
nextNeedDigit = false
continue
} else if nextNeedDigit {
return -int(types.ERR_INVALID_CHAR)
} else if c == '.' {
if !lastIsDigit || pointer || exponent || sp == ss {
return -int(types.ERR_INVALID_CHAR)
}
pointer = true
lastIsDigit = false
nextNeedDigit = true
continue
} else if c == 'e' || c == 'E' {
if !lastIsDigit || exponent {
return -int(types.ERR_INVALID_CHAR)
}
if sp == se-1 {
return -int(types.ERR_EOF)
}
exponent = true
lastIsDigit = false
nextNeedDigit = false
continue
} else if c == '-' || c == '+' {
if prev := *(*byte)(unsafe.Pointer(sp - 1)); prev != 'e' && prev != 'E' {
return -int(types.ERR_INVALID_CHAR)
}
lastIsDigit = false
nextNeedDigit = true
continue
} else {
break
}
}
if nextNeedDigit {
return -int(types.ERR_EOF)
}
runtime.KeepAlive(src)
return int(uintptr(sp) - uintptr((*rt.GoString)(unsafe.Pointer(&src)).Ptr))
}
//go:nocheckptr
@@ -544,7 +605,7 @@ func _DecodeString(src string, pos int, needEsc bool, validStr bool) (v string,
return str, p.p, true
}
/* unquote the string */
out, err := unquote.String(str)
out, err := unquote(str)
/* check for errors */
if err != 0 {
return "", -int(err), true

61
vendor/github.com/bytedance/sonic/ast/encode.go generated vendored
View File

@@ -17,12 +17,12 @@
package ast
import (
"sync"
"unicode/utf8"
`sync`
`unicode/utf8`
)
"github.com/bytedance/gopkg/lang/dirtmake"
"github.com/bytedance/sonic/internal/rt"
"github.com/bytedance/sonic/option"
const (
_MaxBuffer = 1024 // 1KB buffer size
)
func quoteString(e *[]byte, s string) {
@@ -30,7 +30,7 @@ func quoteString(e *[]byte, s string) {
start := 0
for i := 0; i < len(s); {
if b := s[i]; b < utf8.RuneSelf {
if rt.SafeSet[b] {
if safeSet[b] {
i++
continue
}
@@ -54,8 +54,8 @@ func quoteString(e *[]byte, s string) {
// user-controlled strings are rendered into JSON
// and served to some browsers.
*e = append(*e, `u00`...)
*e = append(*e, rt.Hex[b>>4])
*e = append(*e, rt.Hex[b&0xF])
*e = append(*e, hex[b>>4])
*e = append(*e, hex[b&0xF])
}
i++
start = i
@@ -76,7 +76,7 @@ func quoteString(e *[]byte, s string) {
*e = append(*e, s[start:i]...)
}
*e = append(*e, `\u202`...)
*e = append(*e, rt.Hex[c&0xF])
*e = append(*e, hex[c&0xF])
i += size
start = i
continue
@@ -92,29 +92,16 @@ func quoteString(e *[]byte, s string) {
var bytesPool = sync.Pool{}
func (self *Node) MarshalJSON() ([]byte, error) {
if self == nil {
return bytesNull, nil
}
// fast path for raw node
if self.isRaw() {
return rt.Str2Mem(self.toString()), nil
}
buf := newBuffer()
err := self.encode(buf)
if err != nil {
freeBuffer(buf)
return nil, err
}
var ret []byte
if !rt.CanSizeResue(cap(*buf)) {
ret = *buf
} else {
ret = dirtmake.Bytes(len(*buf), len(*buf))
copy(ret, *buf)
freeBuffer(buf)
}
ret := make([]byte, len(*buf))
copy(ret, *buf)
freeBuffer(buf)
return ret, err
}
@@ -122,24 +109,21 @@ func newBuffer() *[]byte {
if ret := bytesPool.Get(); ret != nil {
return ret.(*[]byte)
} else {
buf := make([]byte, 0, option.DefaultAstBufferSize)
buf := make([]byte, 0, _MaxBuffer)
return &buf
}
}
func freeBuffer(buf *[]byte) {
if !rt.CanSizeResue(cap(*buf)) {
return
}
*buf = (*buf)[:0]
bytesPool.Put(buf)
}
func (self *Node) encode(buf *[]byte) error {
if self.isRaw() {
if self.IsRaw() {
return self.encodeRaw(buf)
}
switch int(self.itype()) {
switch self.Type() {
case V_NONE : return ErrNotExist
case V_ERROR : return self.Check()
case V_NULL : return self.encodeNull(buf)
@@ -155,21 +139,16 @@ func (self *Node) encode(buf *[]byte) error {
}
func (self *Node) encodeRaw(buf *[]byte) error {
lock := self.rlock()
if !self.isRaw() {
self.runlock()
return self.encode(buf)
}
raw := self.toString()
if lock {
self.runlock()
raw, err := self.Raw()
if err != nil {
return err
}
*buf = append(*buf, raw...)
return nil
}
func (self *Node) encodeNull(buf *[]byte) error {
*buf = append(*buf, strNull...)
*buf = append(*buf, bytesNull...)
return nil
}

6
vendor/github.com/bytedance/sonic/ast/error.go generated vendored
View File

@@ -17,10 +17,6 @@ func newError(err types.ParsingError, msg string) *Node {
}
}
func newErrorPair(err SyntaxError) *Pair {
return &Pair{0, "", *newSyntaxError(err)}
}
// Error returns error message if the node is invalid
func (self Node) Error() string {
if self.t != V_ERROR {
@@ -83,7 +79,7 @@ func (self SyntaxError) description() string {
/* check for empty source */
if self.Src == "" {
return fmt.Sprintf("no sources available, the input json is empty: %#v", self)
return fmt.Sprintf("no sources available: %#v", self)
}
/* prevent slicing before the beginning */

25
vendor/github.com/bytedance/sonic/ast/iterator.go generated vendored
View File

@@ -17,29 +17,19 @@
package ast
import (
"fmt"
`fmt`
"github.com/bytedance/sonic/internal/caching"
"github.com/bytedance/sonic/internal/native/types"
`github.com/bytedance/sonic/internal/native/types`
)
type Pair struct {
hash uint64
Key string
Value Node
}
func NewPair(key string, val Node) Pair {
return Pair{
hash: caching.StrHash(key),
Key: key,
Value: val,
}
}
// Values returns iterator for array's children traversal
func (self *Node) Values() (ListIterator, error) {
if err := self.should(types.V_ARRAY); err != nil {
if err := self.should(types.V_ARRAY, "an array"); err != nil {
return ListIterator{}, err
}
return self.values(), nil
@@ -51,7 +41,7 @@ func (self *Node) values() ListIterator {
// Properties returns iterator for object's children traversal
func (self *Node) Properties() (ObjectIterator, error) {
if err := self.should(types.V_OBJECT); err != nil {
if err := self.should(types.V_OBJECT, "an object"); err != nil {
return ObjectIterator{}, err
}
return self.properties(), nil
@@ -173,14 +163,11 @@ type Scanner func(path Sequence, node *Node) bool
// ForEach scans one V_OBJECT node's children from JSON head to tail,
// and pass the Sequence and Node of corresponding JSON value.
//
// Especially, if the node is not V_ARRAY or V_OBJECT,
// Especailly, if the node is not V_ARRAY or V_OBJECT,
// the node itself will be returned and Sequence.Index == -1.
//
// NOTICE: An unset node WON'T trigger sc, but its index still counts into Path.Index
// NOTICE: A unsetted node WON'T trigger sc, but its index still counts into Path.Index
func (self *Node) ForEach(sc Scanner) error {
if err := self.checkRaw(); err != nil {
return err
}
switch self.itype() {
case types.V_ARRAY:
iter, err := self.Values()

354
vendor/github.com/bytedance/sonic/ast/node.go generated vendored
View File

@@ -17,15 +17,13 @@
package ast
import (
"encoding/json"
"fmt"
"strconv"
"sync"
"sync/atomic"
"unsafe"
"github.com/bytedance/sonic/internal/native/types"
"github.com/bytedance/sonic/internal/rt"
`encoding/json`
`fmt`
`strconv`
`unsafe`
`github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/internal/rt`
)
const (
@@ -38,7 +36,7 @@ const (
_V_ARRAY_LAZY = _V_LAZY | types.V_ARRAY
_V_OBJECT_LAZY = _V_LAZY | types.V_OBJECT
_MASK_LAZY = _V_LAZY - 1
_MASK_RAW = _V_RAW - 1
_MASK_RAW = _V_RAW - 1
)
const (
@@ -58,20 +56,19 @@ type Node struct {
t types.ValueType
l uint
p unsafe.Pointer
m *sync.RWMutex
}
// UnmarshalJSON is just an adapter to json.Unmarshaler.
// If you want better performance, use Searcher.GetByPath() directly
func (self *Node) UnmarshalJSON(data []byte) (err error) {
*self = newRawNode(rt.Mem2Str(data), switchRawType(data[0]), false)
return nil
*self = NewRaw(string(data))
return self.Check()
}
/** Node Type Accessor **/
// Type returns json type represented by the node
// It will be one of bellows:
// It will be one of belows:
// V_NONE = 0 (empty node, key not exists)
// V_ERROR = 1 (error node)
// V_NULL = 2 (json value `null`, key exists)
@@ -82,39 +79,17 @@ func (self *Node) UnmarshalJSON(data []byte) (err error) {
// V_STRING = 7 (json value string)
// V_NUMBER = 33 (json value number )
// V_ANY = 34 (golang interface{})
//
// Deprecated: not concurrent safe. Use TypeSafe instead
func (self Node) Type() int {
return int(self.t & _MASK_LAZY & _MASK_RAW)
}
// Type concurrently-safe returns json type represented by the node
// It will be one of bellows:
// V_NONE = 0 (empty node, key not exists)
// V_ERROR = 1 (error node)
// V_NULL = 2 (json value `null`, key exists)
// V_TRUE = 3 (json value `true`)
// V_FALSE = 4 (json value `false`)
// V_ARRAY = 5 (json value array)
// V_OBJECT = 6 (json value object)
// V_STRING = 7 (json value string)
// V_NUMBER = 33 (json value number )
// V_ANY = 34 (golang interface{})
func (self *Node) TypeSafe() int {
return int(self.loadt() & _MASK_LAZY & _MASK_RAW)
}
func (self *Node) itype() types.ValueType {
func (self Node) itype() types.ValueType {
return self.t & _MASK_LAZY & _MASK_RAW
}
// Exists returns false only if the self is nil or empty node V_NONE
func (self *Node) Exists() bool {
if self == nil {
return false
}
t := self.loadt()
return t != V_ERROR && t != _V_NONE
return self.Valid() && self.t != _V_NONE
}
// Valid reports if self is NOT V_ERROR or nil
@@ -122,7 +97,7 @@ func (self *Node) Valid() bool {
if self == nil {
return false
}
return self.loadt() != V_ERROR
return self.t != V_ERROR
}
// Check checks if the node itself is valid, and return:
@@ -131,31 +106,24 @@ func (self *Node) Valid() bool {
func (self *Node) Check() error {
if self == nil {
return ErrNotExist
} else if self.loadt() != V_ERROR {
} else if self.t != V_ERROR {
return nil
} else {
return self
}
}
// isRaw returns true if node's underlying value is raw json
//
// Deprecated: not concurrent safe
func (self Node) IsRaw() bool {
return self.t & _V_RAW != 0
}
// IsRaw returns true if node's underlying value is raw json
func (self *Node) isRaw() bool {
return self.loadt() & _V_RAW != 0
func (self Node) IsRaw() bool {
return self.t&_V_RAW != 0
}
func (self *Node) isLazy() bool {
return self != nil && self.t & _V_LAZY != 0
return self != nil && self.t&_V_LAZY != 0
}
func (self *Node) isAny() bool {
return self != nil && self.loadt() == _V_ANY
return self != nil && self.t == _V_ANY
}
/** Simple Value Methods **/
@@ -165,26 +133,18 @@ func (self *Node) Raw() (string, error) {
if self == nil {
return "", ErrNotExist
}
lock := self.rlock()
if !self.isRaw() {
if lock {
self.runlock()
}
if !self.IsRaw() {
buf, err := self.MarshalJSON()
return rt.Mem2Str(buf), err
}
ret := self.toString()
if lock {
self.runlock()
}
return ret, nil
return self.toString(), nil
}
func (self *Node) checkRaw() error {
if err := self.Check(); err != nil {
return err
}
if self.isRaw() {
if self.IsRaw() {
self.parseRaw(false)
}
return self.Check()
@@ -440,7 +400,7 @@ func (self *Node) String() (string, error) {
}
}
// StrictString returns string value (unescaped), including V_STRING, V_ANY of string.
// StrictString returns string value (unescaped), includeing V_STRING, V_ANY of string.
// In other cases, it will return empty string.
func (self *Node) StrictString() (string, error) {
if err := self.checkRaw(); err != nil {
@@ -509,24 +469,7 @@ func (self *Node) Float64() (float64, error) {
}
}
func (self *Node) StrictBool() (bool, error) {
if err := self.checkRaw(); err!= nil {
return false, err
}
switch self.t {
case types.V_TRUE : return true, nil
case types.V_FALSE : return false, nil
case _V_ANY :
any := self.packAny()
switch v := any.(type) {
case bool : return v, nil
default : return false, ErrUnsupportType
}
default : return false, ErrUnsupportType
}
}
// Float64 exports underlying float64 value, including V_NUMBER, V_ANY
// Float64 exports underlying float64 value, includeing V_NUMBER, V_ANY
func (self *Node) StrictFloat64() (float64, error) {
if err := self.checkRaw(); err != nil {
return 0.0, err
@@ -544,7 +487,7 @@ func (self *Node) StrictFloat64() (float64, error) {
}
}
/** Sequential Value Methods **/
/** Sequencial Value Methods **/
// Len returns children count of a array|object|string node
// WARN: For partially loaded node, it also works but only counts the parsed children
@@ -561,7 +504,7 @@ func (self *Node) Len() (int, error) {
}
}
func (self *Node) len() int {
func (self Node) len() int {
return int(self.l)
}
@@ -584,7 +527,7 @@ func (self *Node) Cap() (int, error) {
//
// If self is V_NONE or V_NULL, it becomes V_OBJECT and sets the node at the key.
func (self *Node) Set(key string, node Node) (bool, error) {
if err := self.checkRaw(); err != nil {
if err := self.Check(); err != nil {
return false, err
}
if err := node.Check(); err != nil {
@@ -592,7 +535,7 @@ func (self *Node) Set(key string, node Node) (bool, error) {
}
if self.t == _V_NONE || self.t == types.V_NULL {
*self = NewObject([]Pair{NewPair(key, node)})
*self = NewObject([]Pair{{key, node}})
return false, nil
} else if self.itype() != types.V_OBJECT {
return false, ErrUnsupportType
@@ -606,7 +549,7 @@ func (self *Node) Set(key string, node Node) (bool, error) {
*self = newObject(new(linkedPairs))
}
s := (*linkedPairs)(self.p)
s.Push(NewPair(key, node))
s.Push(Pair{key, node})
self.l++
return false, nil
@@ -625,10 +568,10 @@ func (self *Node) SetAny(key string, val interface{}) (bool, error) {
// Unset REMOVE (soft) the node of given key under object parent, and reports if the key has existed.
func (self *Node) Unset(key string) (bool, error) {
if err := self.should(types.V_OBJECT); err != nil {
if err := self.should(types.V_OBJECT, "an object"); err != nil {
return false, err
}
// NOTICE: must get accurate length before deduct
// NOTICE: must get acurate length before deduct
if err := self.skipAllKey(); err != nil {
return false, err
}
@@ -646,7 +589,7 @@ func (self *Node) Unset(key string) (bool, error) {
//
// The index must be within self's children.
func (self *Node) SetByIndex(index int, node Node) (bool, error) {
if err := self.checkRaw(); err != nil {
if err := self.Check(); err != nil {
return false, err
}
if err := node.Check(); err != nil {
@@ -674,7 +617,7 @@ func (self *Node) SetAnyByIndex(index int, val interface{}) (bool, error) {
return self.SetByIndex(index, NewAny(val))
}
// UnsetByIndex REMOVE (softly) the node of given index.
// UnsetByIndex REOMVE (softly) the node of given index.
//
// WARN: this will change address of elements, which is a dangerous action.
// Use Unset() for object or Pop() for array instead.
@@ -726,7 +669,7 @@ func (self *Node) UnsetByIndex(index int) (bool, error) {
//
// If self is V_NONE or V_NULL, it becomes V_ARRAY and sets the node at index 0.
func (self *Node) Add(node Node) error {
if err := self.checkRaw(); err != nil {
if err := self.Check(); err != nil {
return err
}
@@ -734,7 +677,7 @@ func (self *Node) Add(node Node) error {
*self = NewArray([]Node{node})
return nil
}
if err := self.should(types.V_ARRAY); err != nil {
if err := self.should(types.V_ARRAY, "an array"); err != nil {
return err
}
@@ -793,11 +736,11 @@ func (self *Node) Pop() error {
}
// Move moves the child at src index to dst index,
// meanwhile slides siblings from src+1 to dst.
// meanwhile slides sliblings from src+1 to dst.
//
// WARN: this will change address of elements, which is a dangerous action.
func (self *Node) Move(dst, src int) error {
if err := self.should(types.V_ARRAY); err != nil {
if err := self.should(types.V_ARRAY, "an array"); err != nil {
return err
}
@@ -833,7 +776,7 @@ func (self *Node) Move(dst, src int) error {
return nil
}
// AddAny wraps val with V_ANY node, and Add() the node.
// SetAny wraps val with V_ANY node, and Add() the node.
func (self *Node) AddAny(val interface{}) error {
return self.Add(NewAny(val))
}
@@ -869,7 +812,7 @@ func (self *Node) GetByPath(path ...interface{}) *Node {
// Get loads given key of an object node on demands
func (self *Node) Get(key string) *Node {
if err := self.should(types.V_OBJECT); err != nil {
if err := self.should(types.V_OBJECT, "an object"); err != nil {
return unwrapError(err)
}
n, _ := self.skipKey(key)
@@ -902,14 +845,14 @@ func (self *Node) Index(idx int) *Node {
// IndexPair indexies pair at given idx,
// node type MUST be either V_OBJECT
func (self *Node) IndexPair(idx int) *Pair {
if err := self.should(types.V_OBJECT); err != nil {
if err := self.should(types.V_OBJECT, "an object"); err != nil {
return nil
}
return self.skipIndexPair(idx)
}
func (self *Node) indexOrGet(idx int, key string) (*Node, int) {
if err := self.should(types.V_OBJECT); err != nil {
if err := self.should(types.V_OBJECT, "an object"); err != nil {
return unwrapError(err), idx
}
@@ -946,16 +889,16 @@ func (self *Node) Map() (map[string]interface{}, error) {
return nil, ErrUnsupportType
}
}
if err := self.should(types.V_OBJECT); err != nil {
if err := self.should(types.V_OBJECT, "an object"); err != nil {
return nil, err
}
if err := self.loadAllKey(false); err != nil {
if err := self.loadAllKey(); err != nil {
return nil, err
}
return self.toGenericObject()
}
// MapUseNumber loads all keys of an object node, with numeric nodes cast to json.Number
// MapUseNumber loads all keys of an object node, with numeric nodes casted to json.Number
func (self *Node) MapUseNumber() (map[string]interface{}, error) {
if self.isAny() {
any := self.packAny()
@@ -965,16 +908,16 @@ func (self *Node) MapUseNumber() (map[string]interface{}, error) {
return nil, ErrUnsupportType
}
}
if err := self.should(types.V_OBJECT); err != nil {
if err := self.should(types.V_OBJECT, "an object"); err != nil {
return nil, err
}
if err := self.loadAllKey(false); err != nil {
if err := self.loadAllKey(); err != nil {
return nil, err
}
return self.toGenericObjectUseNumber()
}
// MapUseNode scans both parsed and non-parsed children nodes,
// MapUseNode scans both parsed and non-parsed chidren nodes,
// and map them by their keys
func (self *Node) MapUseNode() (map[string]Node, error) {
if self.isAny() {
@@ -985,7 +928,7 @@ func (self *Node) MapUseNode() (map[string]Node, error) {
return nil, ErrUnsupportType
}
}
if err := self.should(types.V_OBJECT); err != nil {
if err := self.should(types.V_OBJECT, "an object"); err != nil {
return nil, err
}
if err := self.skipAllKey(); err != nil {
@@ -1091,16 +1034,16 @@ func (self *Node) Array() ([]interface{}, error) {
return nil, ErrUnsupportType
}
}
if err := self.should(types.V_ARRAY); err != nil {
if err := self.should(types.V_ARRAY, "an array"); err != nil {
return nil, err
}
if err := self.loadAllIndex(false); err != nil {
if err := self.loadAllIndex(); err != nil {
return nil, err
}
return self.toGenericArray()
}
// ArrayUseNumber loads all indexes of an array node, with numeric nodes cast to json.Number
// ArrayUseNumber loads all indexes of an array node, with numeric nodes casted to json.Number
func (self *Node) ArrayUseNumber() ([]interface{}, error) {
if self.isAny() {
any := self.packAny()
@@ -1110,16 +1053,16 @@ func (self *Node) ArrayUseNumber() ([]interface{}, error) {
return nil, ErrUnsupportType
}
}
if err := self.should(types.V_ARRAY); err != nil {
if err := self.should(types.V_ARRAY, "an array"); err != nil {
return nil, err
}
if err := self.loadAllIndex(false); err != nil {
if err := self.loadAllIndex(); err != nil {
return nil, err
}
return self.toGenericArrayUseNumber()
}
// ArrayUseNode copies both parsed and non-parsed children nodes,
// ArrayUseNode copys both parsed and non-parsed chidren nodes,
// and indexes them by original order
func (self *Node) ArrayUseNode() ([]Node, error) {
if self.isAny() {
@@ -1130,7 +1073,7 @@ func (self *Node) ArrayUseNode() ([]Node, error) {
return nil, ErrUnsupportType
}
}
if err := self.should(types.V_ARRAY); err != nil {
if err := self.should(types.V_ARRAY, "an array"); err != nil {
return nil, err
}
if err := self.skipAllIndex(); err != nil {
@@ -1164,9 +1107,9 @@ func (self *Node) unsafeArray() (*linkedNodes, error) {
return (*linkedNodes)(self.p), nil
}
// Interface loads all children under all paths from this node,
// Interface loads all children under all pathes from this node,
// and converts itself as generic type.
// WARN: all numeric nodes are cast to float64
// WARN: all numberic nodes are casted to float64
func (self *Node) Interface() (interface{}, error) {
if err := self.checkRaw(); err != nil {
return nil, err
@@ -1186,12 +1129,12 @@ func (self *Node) Interface() (interface{}, error) {
}
return v, nil
case _V_ARRAY_LAZY :
if err := self.loadAllIndex(false); err != nil {
if err := self.loadAllIndex(); err != nil {
return nil, err
}
return self.toGenericArray()
case _V_OBJECT_LAZY :
if err := self.loadAllKey(false); err != nil {
if err := self.loadAllKey(); err != nil {
return nil, err
}
return self.toGenericObject()
@@ -1210,7 +1153,7 @@ func (self *Node) packAny() interface{} {
}
// InterfaceUseNumber works same with Interface()
// except numeric nodes are cast to json.Number
// except numberic nodes are casted to json.Number
func (self *Node) InterfaceUseNumber() (interface{}, error) {
if err := self.checkRaw(); err != nil {
return nil, err
@@ -1225,12 +1168,12 @@ func (self *Node) InterfaceUseNumber() (interface{}, error) {
case types.V_STRING : return self.toString(), nil
case _V_NUMBER : return self.toNumber(), nil
case _V_ARRAY_LAZY :
if err := self.loadAllIndex(false); err != nil {
if err := self.loadAllIndex(); err != nil {
return nil, err
}
return self.toGenericArrayUseNumber()
case _V_OBJECT_LAZY :
if err := self.loadAllKey(false); err != nil {
if err := self.loadAllKey(); err != nil {
return nil, err
}
return self.toGenericObjectUseNumber()
@@ -1262,30 +1205,70 @@ func (self *Node) InterfaceUseNode() (interface{}, error) {
}
}
// LoadAll loads the node's children
// and ensure all its children can be READ concurrently (include its children's children)
// LoadAll loads all the node's children and children's children as parsed.
// After calling it, the node can be safely used on concurrency
func (self *Node) LoadAll() error {
return self.Load()
if self.IsRaw() {
self.parseRaw(true)
return self.Check()
}
switch self.itype() {
case types.V_ARRAY:
e := self.len()
if err := self.loadAllIndex(); err != nil {
return err
}
for i := 0; i < e; i++ {
n := self.nodeAt(i)
if n.IsRaw() {
n.parseRaw(true)
}
if err := n.Check(); err != nil {
return err
}
}
return nil
case types.V_OBJECT:
e := self.len()
if err := self.loadAllKey(); err != nil {
return err
}
for i := 0; i < e; i++ {
n := self.pairAt(i)
if n.Value.IsRaw() {
n.Value.parseRaw(true)
}
if err := n.Value.Check(); err != nil {
return err
}
}
return nil
default:
return self.Check()
}
}
// Load loads the node's children as parsed.
// and ensure all its children can be READ concurrently (include its children's children)
// After calling it, only the node itself can be used on concurrency (not include its children)
func (self *Node) Load() error {
if err := self.checkRaw(); err != nil {
return err
}
switch self.t {
case _V_ARRAY_LAZY: self.loadAllIndex(true)
case _V_OBJECT_LAZY: self.loadAllKey(true)
case V_ERROR: return self
case V_NONE: return nil
case _V_ARRAY_LAZY:
return self.skipAllIndex()
case _V_OBJECT_LAZY:
return self.skipAllKey()
default:
return self.Check()
}
if self.m == nil {
self.m = new(sync.RWMutex)
}
return self.checkRaw()
}
/**---------------------------------- Internal Helper Methods ----------------------------------**/
func (self *Node) should(t types.ValueType) error {
func (self *Node) should(t types.ValueType, s string) error {
if err := self.checkRaw(); err != nil {
return err
}
@@ -1456,17 +1439,13 @@ func (self *Node) skipIndexPair(index int) *Pair {
return nil
}
func (self *Node) loadAllIndex(loadOnce bool) error {
func (self *Node) loadAllIndex() error {
if !self.isLazy() {
return nil
}
var err types.ParsingError
parser, stack := self.getParserAndArrayStack()
if !loadOnce {
parser.noLazy = true
} else {
parser.loadOnce = true
}
parser.noLazy = true
*self, err = parser.decodeArray(&stack.v)
if err != 0 {
return parser.ExportError(err)
@@ -1474,19 +1453,14 @@ func (self *Node) loadAllIndex(loadOnce bool) error {
return nil
}
func (self *Node) loadAllKey(loadOnce bool) error {
func (self *Node) loadAllKey() error {
if !self.isLazy() {
return nil
}
var err types.ParsingError
parser, stack := self.getParserAndObjectStack()
if !loadOnce {
parser.noLazy = true
*self, err = parser.decodeObject(&stack.v)
} else {
parser.loadOnce = true
*self, err = parser.decodeObject(&stack.v)
}
parser.noLazy = true
*self, err = parser.decodeObject(&stack.v)
if err != 0 {
return parser.ExportError(err)
}
@@ -1655,23 +1629,7 @@ func NewRaw(json string) Node {
if it == _V_NONE {
return Node{}
}
return newRawNode(parser.s[start:parser.p], it, false)
}
// NewRawConcurrentRead creates a node of raw json, which can be READ
// (GetByPath/Get/Index/GetOrIndex/Int64/Bool/Float64/String/Number/Interface/Array/Map/Raw/MarshalJSON) concurrently.
// If the input json is invalid, NewRaw returns a error Node.
func NewRawConcurrentRead(json string) Node {
parser := NewParserObj(json)
start, err := parser.skip()
if err != 0 {
return *newError(err, err.Message())
}
it := switchRawType(parser.s[start])
if it == _V_NONE {
return Node{}
}
return newRawNode(parser.s[start:parser.p], it, true)
return newRawNode(parser.s[start:parser.p], it)
}
// NewAny creates a node of type V_ANY if any's type isn't Node or *Node,
@@ -1695,7 +1653,7 @@ func NewBytes(src []byte) Node {
if len(src) == 0 {
panic("empty src bytes")
}
out := rt.EncodeBase64ToString(src)
out := encodeBase64(src)
return NewString(out)
}
@@ -1731,15 +1689,15 @@ func NewNumber(v string) Node {
}
}
func (node *Node) toNumber() json.Number {
func (node Node) toNumber() json.Number {
return json.Number(rt.StrFrom(node.p, int64(node.l)))
}
func (self *Node) toString() string {
func (self Node) toString() string {
return rt.StrFrom(self.p, int64(self.l))
}
func (node *Node) toFloat64() (float64, error) {
func (node Node) toFloat64() (float64, error) {
ret, err := node.toNumber().Float64()
if err != nil {
return 0, err
@@ -1747,7 +1705,7 @@ func (node *Node) toFloat64() (float64, error) {
return ret, nil
}
func (node *Node) toInt64() (int64, error) {
func (node Node) toInt64() (int64, error) {
ret,err := node.toNumber().Int64()
if err != nil {
return 0, err
@@ -1783,8 +1741,6 @@ func NewArray(v []Node) Node {
return newArray(s)
}
const _Threshold_Index = 16
func newArray(v *linkedNodes) Node {
return Node{
t: types.V_ARRAY,
@@ -1808,9 +1764,6 @@ func NewObject(v []Pair) Node {
}
func newObject(v *linkedPairs) Node {
if v.size > _Threshold_Index {
v.BuildIndex()
}
return Node{
t: types.V_OBJECT,
l: uint(v.Len()),
@@ -1819,42 +1772,53 @@ func newObject(v *linkedPairs) Node {
}
func (self *Node) setObject(v *linkedPairs) {
if v.size > _Threshold_Index {
v.BuildIndex()
}
self.t = types.V_OBJECT
self.l = uint(v.Len())
self.p = unsafe.Pointer(v)
}
func (self *Node) parseRaw(full bool) {
lock := self.lock()
defer self.unlock()
if !self.isRaw() {
return
func newRawNode(str string, typ types.ValueType) Node {
return Node{
t: _V_RAW | typ,
p: rt.StrPtr(str),
l: uint(len(str)),
}
}
func (self *Node) parseRaw(full bool) {
raw := self.toString()
parser := NewParserObj(raw)
var e types.ParsingError
if full {
parser.noLazy = true
*self, e = parser.Parse()
} else if lock {
var n Node
parser.noLazy = true
parser.loadOnce = true
n, e = parser.Parse()
self.assign(n)
} else {
*self, e = parser.Parse()
parser.skipValue = false
}
var e types.ParsingError
*self, e = parser.Parse()
if e != 0 {
*self = *newSyntaxError(parser.syntaxError(e))
}
}
func (self *Node) assign(n Node) {
self.l = n.l
self.p = n.p
atomic.StoreInt64(&self.t, n.t)
var typeJumpTable = [256]types.ValueType{
'"' : types.V_STRING,
'-' : _V_NUMBER,
'0' : _V_NUMBER,
'1' : _V_NUMBER,
'2' : _V_NUMBER,
'3' : _V_NUMBER,
'4' : _V_NUMBER,
'5' : _V_NUMBER,
'6' : _V_NUMBER,
'7' : _V_NUMBER,
'8' : _V_NUMBER,
'9' : _V_NUMBER,
'[' : types.V_ARRAY,
'f' : types.V_FALSE,
'n' : types.V_NULL,
't' : types.V_TRUE,
'{' : types.V_OBJECT,
}
func switchRawType(c byte) types.ValueType {
return typeJumpTable[c]
}

164
vendor/github.com/bytedance/sonic/ast/parser.go generated vendored
View File

@@ -17,18 +17,14 @@
package ast
import (
"fmt"
"sync"
"sync/atomic"
`fmt`
"github.com/bytedance/sonic/internal/native/types"
"github.com/bytedance/sonic/internal/rt"
"github.com/bytedance/sonic/internal/utils"
"github.com/bytedance/sonic/unquote"
`github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/internal/rt`
)
const (
_DEFAULT_NODE_CAP int = 16
_DEFAULT_NODE_CAP int = 8
_APPEND_GROW_SHIFT = 1
)
@@ -49,7 +45,6 @@ type Parser struct {
p int
s string
noLazy bool
loadOnce bool
skipValue bool
dbuf *byte
}
@@ -65,7 +60,7 @@ func (self *Parser) delim() types.ParsingError {
return types.ERR_EOF
}
/* check for the delimiter */
/* check for the delimtier */
if self.s[p] != ':' {
return types.ERR_INVALID_CHAR
}
@@ -84,7 +79,7 @@ func (self *Parser) object() types.ParsingError {
return types.ERR_EOF
}
/* check for the delimiter */
/* check for the delimtier */
if self.s[p] != '{' {
return types.ERR_INVALID_CHAR
}
@@ -103,7 +98,7 @@ func (self *Parser) array() types.ParsingError {
return types.ERR_EOF
}
/* check for the delimiter */
/* check for the delimtier */
if self.s[p] != '[' {
return types.ERR_INVALID_CHAR
}
@@ -115,15 +110,11 @@ func (self *Parser) array() types.ParsingError {
func (self *Parser) lspace(sp int) int {
ns := len(self.s)
for ; sp<ns && utils.IsSpace(self.s[sp]); sp+=1 {}
for ; sp<ns && isSpace(self.s[sp]); sp+=1 {}
return sp
}
func (self *Parser) backward() {
for ; self.p >= 0 && utils.IsSpace(self.s[self.p]); self.p-=1 {}
}
func (self *Parser) decodeArray(ret *linkedNodes) (Node, types.ParsingError) {
sp := self.p
ns := len(self.s)
@@ -157,7 +148,7 @@ func (self *Parser) decodeArray(ret *linkedNodes) (Node, types.ParsingError) {
if t == _V_NONE {
return Node{}, types.ERR_INVALID_CHAR
}
val = newRawNode(self.s[start:self.p], t, false)
val = newRawNode(self.s[start:self.p], t)
}else{
/* decode the value */
if val, err = self.Parse(); err != 0 {
@@ -219,7 +210,7 @@ func (self *Parser) decodeObject(ret *linkedPairs) (Node, types.ParsingError) {
/* check for escape sequence */
if njs.Ep != -1 {
if key, err = unquote.String(key); err != 0 {
if key, err = unquote(key); err != 0 {
return Node{}, err
}
}
@@ -243,7 +234,7 @@ func (self *Parser) decodeObject(ret *linkedPairs) (Node, types.ParsingError) {
if t == _V_NONE {
return Node{}, types.ERR_INVALID_CHAR
}
val = newRawNode(self.s[start:self.p], t, false)
val = newRawNode(self.s[start:self.p], t)
} else {
/* decode the value */
if val, err = self.Parse(); err != 0 {
@@ -253,7 +244,7 @@ func (self *Parser) decodeObject(ret *linkedPairs) (Node, types.ParsingError) {
/* add the value to result */
// FIXME: ret's address may change here, thus previous referred node in ret may be invalid !!
ret.Push(NewPair(key, val))
ret.Push(Pair{Key: key, Value: val})
self.p = self.lspace(self.p)
/* check for EOF */
@@ -284,7 +275,7 @@ func (self *Parser) decodeString(iv int64, ep int) (Node, types.ParsingError) {
}
/* unquote the string */
out, err := unquote.String(s)
out, err := unquote(s)
/* check for errors */
if err != 0 {
@@ -300,10 +291,6 @@ func (self *Parser) Pos() int {
return self.p
}
// Parse returns a ast.Node representing the parser's JSON.
// NOTICE: the specific parsing lazy dependens parser's option
// It only parse first layer and first child for Object or Array be default
func (self *Parser) Parse() (Node, types.ParsingError) {
switch val := self.decodeValue(); val.Vt {
case types.V_EOF : return Node{}, types.ERR_EOF
@@ -312,48 +299,22 @@ func (self *Parser) Parse() (Node, types.ParsingError) {
case types.V_FALSE : return falseNode, 0
case types.V_STRING : return self.decodeString(val.Iv, val.Ep)
case types.V_ARRAY:
s := self.p - 1;
if p := skipBlank(self.s, self.p); p >= self.p && self.s[p] == ']' {
self.p = p + 1
return Node{t: types.V_ARRAY}, 0
}
if self.noLazy {
if self.loadOnce {
self.noLazy = false
}
return self.decodeArray(new(linkedNodes))
}
// NOTICE: loadOnce always keep raw json for object or array
if self.loadOnce {
self.p = s
s, e := self.skipFast()
if e != 0 {
return Node{}, e
}
return newRawNode(self.s[s:self.p], types.V_ARRAY, true), 0
}
return newLazyArray(self), 0
case types.V_OBJECT:
s := self.p - 1;
if p := skipBlank(self.s, self.p); p >= self.p && self.s[p] == '}' {
self.p = p + 1
return Node{t: types.V_OBJECT}, 0
}
// NOTICE: loadOnce always keep raw json for object or array
if self.noLazy {
if self.loadOnce {
self.noLazy = false
}
return self.decodeObject(new(linkedPairs))
}
if self.loadOnce {
self.p = s
s, e := self.skipFast()
if e != 0 {
return Node{}, e
}
return newRawNode(self.s[s:self.p], types.V_OBJECT, true), 0
}
return newLazyObject(self), 0
case types.V_DOUBLE : return NewNumber(self.s[val.Ep:self.p]), 0
case types.V_INTEGER : return NewNumber(self.s[val.Ep:self.p]), 0
@@ -394,7 +355,7 @@ func (self *Parser) searchKey(match string) types.ParsingError {
/* check for escape sequence */
if njs.Ep != -1 {
if key, err = unquote.String(key); err != 0 {
if key, err = unquote(key); err != 0 {
return err
}
}
@@ -510,7 +471,7 @@ func (self *Node) skipNextNode() *Node {
if t == _V_NONE {
return newSyntaxError(parser.syntaxError(types.ERR_INVALID_CHAR))
}
val = newRawNode(parser.s[start:parser.p], t, false)
val = newRawNode(parser.s[start:parser.p], t)
}
/* add the value to result */
@@ -549,7 +510,7 @@ func (self *Node) skipNextPair() (*Pair) {
/* check for EOF */
if parser.p = parser.lspace(sp); parser.p >= ns {
return newErrorPair(parser.syntaxError(types.ERR_EOF))
return &Pair{"", *newSyntaxError(parser.syntaxError(types.ERR_EOF))}
}
/* check for empty object */
@@ -566,7 +527,7 @@ func (self *Node) skipNextPair() (*Pair) {
/* decode the key */
if njs = parser.decodeValue(); njs.Vt != types.V_STRING {
return newErrorPair(parser.syntaxError(types.ERR_INVALID_CHAR))
return &Pair{"", *newSyntaxError(parser.syntaxError(types.ERR_INVALID_CHAR))}
}
/* extract the key */
@@ -575,35 +536,35 @@ func (self *Node) skipNextPair() (*Pair) {
/* check for escape sequence */
if njs.Ep != -1 {
if key, err = unquote.String(key); err != 0 {
return newErrorPair(parser.syntaxError(err))
if key, err = unquote(key); err != 0 {
return &Pair{key, *newSyntaxError(parser.syntaxError(err))}
}
}
/* expect a ':' delimiter */
if err = parser.delim(); err != 0 {
return newErrorPair(parser.syntaxError(err))
return &Pair{key, *newSyntaxError(parser.syntaxError(err))}
}
/* skip the value */
if start, err := parser.skipFast(); err != 0 {
return newErrorPair(parser.syntaxError(err))
return &Pair{key, *newSyntaxError(parser.syntaxError(err))}
} else {
t := switchRawType(parser.s[start])
if t == _V_NONE {
return newErrorPair(parser.syntaxError(types.ERR_INVALID_CHAR))
return &Pair{key, *newSyntaxError(parser.syntaxError(types.ERR_INVALID_CHAR))}
}
val = newRawNode(parser.s[start:parser.p], t, false)
val = newRawNode(parser.s[start:parser.p], t)
}
/* add the value to result */
ret.Push(NewPair(key, val))
ret.Push(Pair{Key: key, Value: val})
self.l++
parser.p = parser.lspace(parser.p)
/* check for EOF */
if parser.p >= ns {
return newErrorPair(parser.syntaxError(types.ERR_EOF))
return &Pair{key, *newSyntaxError(parser.syntaxError(types.ERR_EOF))}
}
/* check for the next character */
@@ -616,7 +577,7 @@ func (self *Node) skipNextPair() (*Pair) {
self.setObject(ret)
return ret.At(ret.Len()-1)
default:
return newErrorPair(parser.syntaxError(types.ERR_INVALID_CHAR))
return &Pair{key, *newSyntaxError(parser.syntaxError(types.ERR_INVALID_CHAR))}
}
}
@@ -640,7 +601,7 @@ func Loads(src string) (int, interface{}, error) {
}
}
// LoadsUseNumber parse all json into interface{}, with numeric nodes cast to json.Number
// LoadsUseNumber parse all json into interface{}, with numeric nodes casted to json.Number
func LoadsUseNumber(src string) (int, interface{}, error) {
ps := &Parser{s: src}
np, err := ps.Parse()
@@ -694,75 +655,6 @@ func (self *Parser) ExportError(err types.ParsingError) error {
}
func backward(src string, i int) int {
for ; i>=0 && utils.IsSpace(src[i]); i-- {}
for ; i>=0 && isSpace(src[i]); i-- {}
return i
}
func newRawNode(str string, typ types.ValueType, lock bool) Node {
ret := Node{
t: typ | _V_RAW,
p: rt.StrPtr(str),
l: uint(len(str)),
}
if lock {
ret.m = new(sync.RWMutex)
}
return ret
}
var typeJumpTable = [256]types.ValueType{
'"' : types.V_STRING,
'-' : _V_NUMBER,
'0' : _V_NUMBER,
'1' : _V_NUMBER,
'2' : _V_NUMBER,
'3' : _V_NUMBER,
'4' : _V_NUMBER,
'5' : _V_NUMBER,
'6' : _V_NUMBER,
'7' : _V_NUMBER,
'8' : _V_NUMBER,
'9' : _V_NUMBER,
'[' : types.V_ARRAY,
'f' : types.V_FALSE,
'n' : types.V_NULL,
't' : types.V_TRUE,
'{' : types.V_OBJECT,
}
func switchRawType(c byte) types.ValueType {
return typeJumpTable[c]
}
func (self *Node) loadt() types.ValueType {
return (types.ValueType)(atomic.LoadInt64(&self.t))
}
func (self *Node) lock() bool {
if m := self.m; m != nil {
m.Lock()
return true
}
return false
}
func (self *Node) unlock() {
if m := self.m; m != nil {
m.Unlock()
}
}
func (self *Node) rlock() bool {
if m := self.m; m != nil {
m.RLock()
return true
}
return false
}
func (self *Node) runlock() {
if m := self.m; m != nil {
m.RUnlock()
}
}

31
vendor/github.com/bytedance/sonic/ast/search.go generated vendored
View File

@@ -21,23 +21,8 @@ import (
`github.com/bytedance/sonic/internal/native/types`
)
// SearchOptions controls Searcher's behavior
type SearchOptions struct {
// ValidateJSON indicates the searcher to validate the entire JSON
ValidateJSON bool
// CopyReturn indicates the searcher to copy the result JSON instead of refer from the input
// This can help to reduce memory usage if you cache the results
CopyReturn bool
// ConcurrentRead indicates the searcher to return a concurrently-READ-safe node,
// including: GetByPath/Get/Index/GetOrIndex/Int64/Bool/Float64/String/Number/Interface/Array/Map/Raw/MarshalJSON
ConcurrentRead bool
}
type Searcher struct {
parser Parser
SearchOptions
}
func NewSearcher(str string) *Searcher {
@@ -46,16 +31,12 @@ func NewSearcher(str string) *Searcher {
s: str,
noLazy: false,
},
SearchOptions: SearchOptions{
ValidateJSON: true,
},
}
}
// GetByPathCopy search in depth from top json and returns a **Copied** json node at the path location
func (self *Searcher) GetByPathCopy(path ...interface{}) (Node, error) {
self.CopyReturn = true
return self.getByPath(path...)
return self.getByPath(true, true, path...)
}
// GetByPathNoCopy search in depth from top json and returns a **Referenced** json node at the path location
@@ -63,15 +44,15 @@ func (self *Searcher) GetByPathCopy(path ...interface{}) (Node, error) {
// WARN: this search directly refer partial json from top json, which has faster speed,
// may consumes more memory.
func (self *Searcher) GetByPath(path ...interface{}) (Node, error) {
return self.getByPath(path...)
return self.getByPath(false, true, path...)
}
func (self *Searcher) getByPath(path ...interface{}) (Node, error) {
func (self *Searcher) getByPath(copystring bool, validate bool, path ...interface{}) (Node, error) {
var err types.ParsingError
var start int
self.parser.p = 0
start, err = self.parser.getByPath(self.ValidateJSON, path...)
start, err = self.parser.getByPath(validate, path...)
if err != 0 {
// for compatibility with old version
if err == types.ERR_NOT_FOUND {
@@ -90,12 +71,12 @@ func (self *Searcher) getByPath(path ...interface{}) (Node, error) {
// copy string to reducing memory usage
var raw string
if self.CopyReturn {
if copystring {
raw = rt.Mem2Str([]byte(self.parser.s[start:self.parser.p]))
} else {
raw = self.parser.s[start:self.parser.p]
}
return newRawNode(raw, t, self.ConcurrentRead), nil
return newRawNode(raw, t), nil
}
// GetByPath searches a path and returns relaction and types of target

16
vendor/github.com/bytedance/sonic/compat.go generated vendored
View File

@@ -1,4 +1,4 @@
// +build !amd64,!arm64 go1.26 !go1.17 arm64,!go1.20
// +build !amd64 !go1.16 go1.23
/*
* Copyright 2021 ByteDance Inc.
@@ -27,8 +27,6 @@ import (
`github.com/bytedance/sonic/option`
)
const apiKind = UseStdJSON
type frozenConfig struct {
Config
}
@@ -87,17 +85,7 @@ func (cfg frozenConfig) UnmarshalFromString(buf string, val interface{}) error {
if cfg.DisallowUnknownFields {
dec.DisallowUnknownFields()
}
err := dec.Decode(val)
if err != nil {
return err
}
// check the trailing chars
offset := dec.InputOffset()
if t, err := dec.Token(); !(t == nil && err == io.EOF) {
return &json.SyntaxError{ Offset: offset}
}
return nil
return dec.Decode(val)
}
// Unmarshal is implemented by sonic

43
vendor/github.com/bytedance/sonic/decoder/decoder_compat.go generated vendored
View File

@@ -1,5 +1,4 @@
//go:build (!amd64 && !arm64) || go1.26 || !go1.17 || (arm64 && !go1.20)
// +build !amd64,!arm64 go1.26 !go1.17 arm64,!go1.20
// +build !amd64 !go1.16 go1.23
/*
* Copyright 2023 ByteDance Inc.
@@ -20,33 +19,29 @@
package decoder
import (
"bytes"
"encoding/json"
"io"
"reflect"
"unsafe"
`bytes`
`encoding/json`
`io`
`reflect`
`unsafe`
"github.com/bytedance/sonic/internal/decoder/consts"
"github.com/bytedance/sonic/internal/native/types"
"github.com/bytedance/sonic/option"
"github.com/bytedance/sonic/internal/compat"
`github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/option`
)
func init() {
compat.Warn("sonic/decoder")
println("WARNING: sonic only supports Go1.16~1.22 && CPU amd64, but your environment is not suitable")
}
const (
_F_use_int64 = consts.F_use_int64
_F_disable_urc = consts.F_disable_unknown
_F_disable_unknown = consts.F_disable_unknown
_F_copy_string = consts.F_copy_string
_F_use_int64 = 0
_F_disable_urc = 2
_F_disable_unknown = 3
_F_copy_string = 4
_F_use_number = consts.F_use_number
_F_validate_string = consts.F_validate_string
_F_allow_control = consts.F_allow_control
_F_no_validate_json = consts.F_no_validate_json
_F_case_sensitive = consts.F_case_sensitive
_F_use_number = types.B_USE_NUMBER
_F_validate_string = types.B_VALIDATE_STRING
_F_allow_control = types.B_ALLOW_CONTROL
)
type Options uint64
@@ -58,8 +53,6 @@ const (
OptionDisableUnknown Options = 1 << _F_disable_unknown
OptionCopyString Options = 1 << _F_copy_string
OptionValidateString Options = 1 << _F_validate_string
OptionNoValidateJSON Options = 1 << _F_no_validate_json
OptionCaseSensitive Options = 1 << _F_case_sensitive
)
func (self *Decoder) SetOptions(opts Options) {
@@ -197,5 +190,5 @@ func (s SyntaxError) Error() string {
return (*json.SyntaxError)(unsafe.Pointer(&s)).Error()
}
// MismatchTypeError represents mismatching between json and object
type MismatchTypeError json.UnmarshalTypeError
// MismatchTypeError represents dismatching between json and object
type MismatchTypeError json.UnmarshalTypeError

7
vendor/github.com/bytedance/sonic/encoder/encoder_compat.go generated vendored
View File

@@ -1,4 +1,4 @@
// +build !amd64,!arm64 go1.26 !go1.17 arm64,!go1.20
// +build !amd64 !go1.16 go1.23
/*
* Copyright 2023 ByteDance Inc.
@@ -19,17 +19,16 @@
package encoder
import (
`io`
`io`
`bytes`
`encoding/json`
`reflect`
`github.com/bytedance/sonic/option`
`github.com/bytedance/sonic/internal/compat`
)
func init() {
compat.Warn("sonic/encoder")
println("WARNING:(encoder) sonic only supports Go1.16~1.22 && CPU amd64, but your environment is not suitable")
}
// EnableFallback indicates if encoder use fallback

8
vendor/github.com/bytedance/sonic/internal/caching/hashing.go generated vendored
View File

@@ -23,12 +23,16 @@ import (
)
var (
V_strhash = rt.UnpackEface(rt.Strhash)
V_strhash = rt.UnpackEface(strhash)
S_strhash = *(*uintptr)(V_strhash.Value)
)
//go:noescape
//go:linkname strhash runtime.strhash
func strhash(_ unsafe.Pointer, _ uintptr) uintptr
func StrHash(s string) uint64 {
if v := rt.Strhash(unsafe.Pointer(&s), 0); v == 0 {
if v := strhash(unsafe.Pointer(&s), 0); v == 0 {
return 1
} else {
return uint64(v)

4
vendor/github.com/bytedance/sonic/internal/cpu/features.go generated vendored
View File

@@ -24,6 +24,7 @@ import (
)
var (
HasAVX = cpuid.CPU.Has(cpuid.AVX)
HasAVX2 = cpuid.CPU.Has(cpuid.AVX2)
HasSSE = cpuid.CPU.Has(cpuid.SSE)
)
@@ -32,8 +33,7 @@ func init() {
switch v := os.Getenv("SONIC_MODE"); v {
case "" : break
case "auto" : break
case "noavx" : HasAVX2 = false
// will also disable avx, act as `noavx`, we remain it to make sure forward compatibility
case "noavx" : HasAVX = false; fallthrough
case "noavx2" : HasAVX2 = false
default : panic(fmt.Sprintf("invalid mode: '%s', should be one of 'auto', 'noavx', 'noavx2'", v))
}

1384
vendor/github.com/bytedance/sonic/internal/encoder/compiler.go generated vendored
View File

File diff suppressed because it is too large Load Diff

170
vendor/github.com/bytedance/sonic/internal/encoder/encoder.go generated vendored
View File

@@ -17,63 +17,72 @@
package encoder
import (
"bytes"
"encoding/json"
"reflect"
"runtime"
"unsafe"
`bytes`
`encoding/json`
`reflect`
`runtime`
`unsafe`
"github.com/bytedance/sonic/utf8"
"github.com/bytedance/sonic/internal/encoder/alg"
"github.com/bytedance/sonic/internal/encoder/vars"
"github.com/bytedance/sonic/internal/rt"
"github.com/bytedance/sonic/option"
"github.com/bytedance/gopkg/lang/dirtmake"
`github.com/bytedance/sonic/internal/native`
`github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/internal/rt`
`github.com/bytedance/sonic/utf8`
`github.com/bytedance/sonic/option`
)
// Options is a set of encoding options.
type Options uint64
const (
bitSortMapKeys = iota
bitEscapeHTML
bitCompactMarshaler
bitNoQuoteTextMarshaler
bitNoNullSliceOrMap
bitValidateString
bitNoValidateJSONMarshaler
bitNoEncoderNewline
// used for recursive compile
bitPointerValue = 63
)
const (
// SortMapKeys indicates that the keys of a map needs to be sorted
// before serializing into JSON.
// WARNING: This hurts performance A LOT, USE WITH CARE.
SortMapKeys Options = 1 << alg.BitSortMapKeys
SortMapKeys Options = 1 << bitSortMapKeys
// EscapeHTML indicates encoder to escape all HTML characters
// after serializing into JSON (see https://pkg.go.dev/encoding/json#HTMLEscape).
// WARNING: This hurts performance A LOT, USE WITH CARE.
EscapeHTML Options = 1 << alg.BitEscapeHTML
EscapeHTML Options = 1 << bitEscapeHTML
// CompactMarshaler indicates that the output JSON from json.Marshaler
// is always compact and needs no validation
CompactMarshaler Options = 1 << alg.BitCompactMarshaler
CompactMarshaler Options = 1 << bitCompactMarshaler
// NoQuoteTextMarshaler indicates that the output text from encoding.TextMarshaler
// is always escaped string and needs no quoting
NoQuoteTextMarshaler Options = 1 << alg.BitNoQuoteTextMarshaler
NoQuoteTextMarshaler Options = 1 << bitNoQuoteTextMarshaler
// NoNullSliceOrMap indicates all empty Array or Object are encoded as '[]' or '{}',
// instead of 'null'.
// NOTE: The priority of this option is lower than json tag `omitempty`.
NoNullSliceOrMap Options = 1 << alg.BitNoNullSliceOrMap
// instead of 'null'
NoNullSliceOrMap Options = 1 << bitNoNullSliceOrMap
// ValidateString indicates that encoder should validate the input string
// before encoding it into JSON.
ValidateString Options = 1 << alg.BitValidateString
ValidateString Options = 1 << bitValidateString
// NoValidateJSONMarshaler indicates that the encoder should not validate the output string
// after encoding the JSONMarshaler to JSON.
NoValidateJSONMarshaler Options = 1 << alg.BitNoValidateJSONMarshaler
NoValidateJSONMarshaler Options = 1 << bitNoValidateJSONMarshaler
// NoEncoderNewline indicates that the encoder should not add a newline after every message
NoEncoderNewline Options = 1 << alg.BitNoEncoderNewline
NoEncoderNewline Options = 1 << bitNoEncoderNewline
// CompatibleWithStd is used to be compatible with std encoder.
CompatibleWithStd Options = SortMapKeys | EscapeHTML | CompactMarshaler
// Encode Infinity or Nan float into `null`, instead of returning an error.
EncodeNullForInfOrNan Options = 1 << alg.BitEncodeNullForInfOrNan
)
// Encoder represents a specific set of encoder configurations.
@@ -162,45 +171,53 @@ func (enc *Encoder) SetIndent(prefix, indent string) {
// Quote returns the JSON-quoted version of s.
func Quote(s string) string {
buf := make([]byte, 0, len(s)+2)
buf = alg.Quote(buf, s, false)
return rt.Mem2Str(buf)
var n int
var p []byte
/* check for empty string */
if s == "" {
return `""`
}
/* allocate space for result */
n = len(s) + 2
p = make([]byte, 0, n)
/* call the encoder */
_ = encodeString(&p, s)
return rt.Mem2Str(p)
}
// Encode returns the JSON encoding of val, encoded with opts.
func Encode(val interface{}, opts Options) ([]byte, error) {
var ret []byte
buf := vars.NewBytes()
err := encodeIntoCheckRace(buf, val, opts)
buf := newBytes()
err := encodeInto(&buf, val, opts)
/* check for errors */
if err != nil {
vars.FreeBytes(buf)
freeBytes(buf)
return nil, err
}
/* htmlescape or correct UTF-8 if opts enable */
old := buf
*buf = encodeFinish(*old, opts)
pbuf := ((*rt.GoSlice)(unsafe.Pointer(buf))).Ptr
pold := ((*rt.GoSlice)(unsafe.Pointer(old))).Ptr
buf = encodeFinish(old, opts)
pbuf := ((*rt.GoSlice)(unsafe.Pointer(&buf))).Ptr
pold := ((*rt.GoSlice)(unsafe.Pointer(&old))).Ptr
/* return when allocated a new buffer */
if pbuf != pold {
vars.FreeBytes(old)
return *buf, nil
freeBytes(old)
return buf, nil
}
/* make a copy of the result */
if rt.CanSizeResue(cap(*buf)) {
ret = dirtmake.Bytes(len(*buf), len(*buf))
copy(ret, *buf)
vars.FreeBytes(buf)
} else {
ret = *buf
}
ret = make([]byte, len(buf))
copy(ret, buf)
freeBytes(buf)
/* return the buffer into pool */
return ret, nil
}
@@ -208,7 +225,7 @@ func Encode(val interface{}, opts Options) ([]byte, error) {
// EncodeInto is like Encode but uses a user-supplied buffer instead of allocating
// a new one.
func EncodeInto(buf *[]byte, val interface{}, opts Options) error {
err := encodeIntoCheckRace(buf, val, opts)
err := encodeInto(buf, val, opts)
if err != nil {
return err
}
@@ -217,15 +234,15 @@ func EncodeInto(buf *[]byte, val interface{}, opts Options) error {
}
func encodeInto(buf *[]byte, val interface{}, opts Options) error {
stk := vars.NewStack()
stk := newStack()
efv := rt.UnpackEface(val)
err := encodeTypedPointer(buf, efv.Type, &efv.Value, stk, uint64(opts))
/* return the stack into pool */
if err != nil {
vars.ResetStack(stk)
resetStack(stk)
}
vars.FreeStack(stk)
freeStack(stk)
/* avoid GC ahead */
runtime.KeepAlive(buf)
@@ -237,12 +254,13 @@ func encodeFinish(buf []byte, opts Options) []byte {
if opts & EscapeHTML != 0 {
buf = HTMLEscape(nil, buf)
}
if (opts & ValidateString != 0) && !utf8.Validate(buf) {
if opts & ValidateString != 0 && !utf8.Validate(buf) {
buf = utf8.CorrectWith(nil, buf, `\ufffd`)
}
return buf
}
var typeByte = rt.UnpackType(reflect.TypeOf(byte(0)))
// HTMLEscape appends to dst the JSON-encoded src with <, >, &, U+2028 and U+2029
// characters inside string literals changed to \u003c, \u003e, \u0026, \u2028, \u2029
@@ -251,7 +269,7 @@ func encodeFinish(buf []byte, opts Options) []byte {
// escaping within <script> tags, so an alternative JSON encoding must
// be used.
func HTMLEscape(dst []byte, src []byte) []byte {
return alg.HtmlEscape(dst, src)
return htmlEscape(dst, src)
}
// EncodeIndented is like Encode but applies Indent to format the output.
@@ -259,40 +277,37 @@ func HTMLEscape(dst []byte, src []byte) []byte {
// followed by one or more copies of indent according to the indentation nesting.
func EncodeIndented(val interface{}, prefix string, indent string, opts Options) ([]byte, error) {
var err error
var out []byte
var buf *bytes.Buffer
/* encode into the buffer */
out := vars.NewBytes()
err = EncodeInto(out, val, opts)
out = newBytes()
err = EncodeInto(&out, val, opts)
/* check for errors */
if err != nil {
vars.FreeBytes(out)
freeBytes(out)
return nil, err
}
/* indent the JSON */
buf = vars.NewBuffer()
err = json.Indent(buf, *out, prefix, indent)
vars.FreeBytes(out)
buf = newBuffer()
err = json.Indent(buf, out, prefix, indent)
/* check for errors */
if err != nil {
vars.FreeBuffer(buf)
freeBytes(out)
freeBuffer(buf)
return nil, err
}
/* copy to the result buffer */
var ret []byte
if rt.CanSizeResue(cap(buf.Bytes())) {
ret = make([]byte, buf.Len())
copy(ret, buf.Bytes())
/* return the buffers into pool */
vars.FreeBuffer(buf)
} else {
ret = buf.Bytes()
}
ret := make([]byte, buf.Len())
copy(ret, buf.Bytes())
/* return the buffers into pool */
freeBytes(out)
freeBuffer(buf)
return ret, nil
}
@@ -315,5 +330,26 @@ func Pretouch(vt reflect.Type, opts ...option.CompileOption) error {
//
// Note: it does not check for the invalid UTF-8 characters.
func Valid(data []byte) (ok bool, start int) {
return alg.Valid(data)
n := len(data)
if n == 0 {
return false, -1
}
s := rt.Mem2Str(data)
p := 0
m := types.NewStateMachine()
ret := native.ValidateOne(&s, &p, m, types.F_VALIDATE_STRING)
types.FreeStateMachine(m)
if ret < 0 {
return false, p-1
}
/* check for trailing spaces */
for ;p < n; p++ {
if (types.SPACE_MASK & (1 << data[p])) == 0 {
return false, p
}
}
return true, ret
}

28
vendor/github.com/bytedance/sonic/internal/encoder/stream.go generated vendored
View File

@@ -17,10 +17,8 @@
package encoder
import (
"encoding/json"
"io"
"github.com/bytedance/sonic/internal/encoder/vars"
`encoding/json`
`io`
)
// StreamEncoder uses io.Writer as input.
@@ -38,20 +36,21 @@ func NewStreamEncoder(w io.Writer) *StreamEncoder {
// Encode encodes interface{} as JSON to io.Writer
func (enc *StreamEncoder) Encode(val interface{}) (err error) {
out := vars.NewBytes()
buf := newBytes()
out := buf
/* encode into the buffer */
err = EncodeInto(out, val, enc.Opts)
err = EncodeInto(&out, val, enc.Opts)
if err != nil {
goto free_bytes
}
if enc.indent != "" || enc.prefix != "" {
/* indent the JSON */
buf := vars.NewBuffer()
err = json.Indent(buf, *out, enc.prefix, enc.indent)
buf := newBuffer()
err = json.Indent(buf, out, enc.prefix, enc.indent)
if err != nil {
vars.FreeBuffer(buf)
freeBuffer(buf)
goto free_bytes
}
@@ -63,17 +62,16 @@ func (enc *StreamEncoder) Encode(val interface{}) (err error) {
/* copy into io.Writer */
_, err = io.Copy(enc.w, buf)
if err != nil {
vars.FreeBuffer(buf)
freeBuffer(buf)
goto free_bytes
}
} else {
/* copy into io.Writer */
var n int
buf := *out
for len(buf) > 0 {
n, err = enc.w.Write(buf)
buf = buf[n:]
for len(out) > 0 {
n, err = enc.w.Write(out)
out = out[n:]
if err != nil {
goto free_bytes
}
@@ -86,6 +84,6 @@ func (enc *StreamEncoder) Encode(val interface{}) (err error) {
}
free_bytes:
vars.FreeBytes(out)
freeBytes(buf)
return err
}

13
vendor/github.com/bytedance/sonic/internal/jit/arch_amd64.go generated vendored
View File

@@ -17,10 +17,8 @@
package jit
import (
"unsafe"
"github.com/twitchyliquid64/golang-asm/asm/arch"
"github.com/twitchyliquid64/golang-asm/obj"
`github.com/twitchyliquid64/golang-asm/asm/arch`
`github.com/twitchyliquid64/golang-asm/obj`
)
var (
@@ -35,13 +33,6 @@ func As(op string) obj.As {
}
}
func ImmPtr(imm unsafe.Pointer) obj.Addr {
return obj.Addr {
Type : obj.TYPE_CONST,
Offset : int64(uintptr(imm)),
}
}
func Imm(imm int64) obj.Addr {
return obj.Addr {
Type : obj.TYPE_CONST,

7
vendor/github.com/bytedance/sonic/internal/jit/backend.go generated vendored
View File

@@ -21,7 +21,6 @@ import (
`sync`
_ `unsafe`
`github.com/bytedance/sonic/internal/rt`
`github.com/twitchyliquid64/golang-asm/asm/arch`
`github.com/twitchyliquid64/golang-asm/obj`
`github.com/twitchyliquid64/golang-asm/objabi`
@@ -39,6 +38,10 @@ var (
_progPool sync.Pool
)
//go:nosplit
//go:linkname throw runtime.throw
func throw(_ string)
func newProg() *obj.Prog {
if val := _progPool.Get(); val == nil {
return new(obj.Prog)
@@ -68,7 +71,7 @@ func newLinkContext(arch *obj.LinkArch) (ret *obj.Link) {
}
func diagLinkContext(str string, args ...interface{}) {
rt.Throw(fmt.Sprintf(str, args...))
throw(fmt.Sprintf(str, args...))
}
func (self *Backend) New() (ret *obj.Prog) {

2
vendor/github.com/bytedance/sonic/internal/jit/runtime.go generated vendored
View File

@@ -37,7 +37,7 @@ func Type(t reflect.Type) obj.Addr {
}
func Itab(i *rt.GoType, t reflect.Type) obj.Addr {
return Imm(int64(uintptr(unsafe.Pointer(rt.GetItab(rt.IfaceType(i), rt.UnpackType(t), false)))))
return Imm(int64(uintptr(unsafe.Pointer(rt.Getitab(rt.IfaceType(i), rt.UnpackType(t), false)))))
}
func Gitab(i *rt.GoItab) obj.Addr {

75
vendor/github.com/bytedance/sonic/internal/native/dispatch_amd64.go generated vendored
View File

@@ -20,6 +20,7 @@ import (
`unsafe`
`github.com/bytedance/sonic/internal/cpu`
`github.com/bytedance/sonic/internal/native/avx`
`github.com/bytedance/sonic/internal/native/avx2`
`github.com/bytedance/sonic/internal/native/sse`
`github.com/bytedance/sonic/internal/native/types`
@@ -86,10 +87,6 @@ var (
__ValidateUTF8 func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer) (ret int)
__ValidateUTF8Fast func(s unsafe.Pointer) (ret int)
__ParseWithPadding func(parser unsafe.Pointer) (ret int)
__LookupSmallKey func(key unsafe.Pointer, table unsafe.Pointer, lowerOff int) (index int)
)
//go:nosplit
@@ -162,22 +159,12 @@ func ValidateUTF8Fast(s *string) (ret int) {
return __ValidateUTF8Fast(rt.NoEscape(unsafe.Pointer(s)))
}
//go:nosplit
func ParseWithPadding(parser unsafe.Pointer) (ret int) {
return __ParseWithPadding(rt.NoEscape(unsafe.Pointer(parser)))
}
//go:nosplit
func LookupSmallKey(key *string, table *[]byte, lowerOff int) (index int) {
return __LookupSmallKey(rt.NoEscape(unsafe.Pointer(key)), rt.NoEscape(unsafe.Pointer(table)), lowerOff)
}
func useSSE() {
sse.Use()
S_f64toa = sse.S_f64toa
__F64toa = sse.F_f64toa
S_f32toa = sse.S_f32toa
__F32toa = sse.F_f32toa
__F64toa = sse.F_f64toa
S_i64toa = sse.S_i64toa
__I64toa = sse.F_i64toa
S_u64toa = sse.S_u64toa
@@ -205,8 +192,42 @@ func useSSE() {
__ValidateOne = sse.F_validate_one
__ValidateUTF8= sse.F_validate_utf8
__ValidateUTF8Fast = sse.F_validate_utf8_fast
__ParseWithPadding = sse.F_parse_with_padding
__LookupSmallKey = sse.F_lookup_small_key
}
func useAVX() {
avx.Use()
S_f64toa = avx.S_f64toa
__F64toa = avx.F_f64toa
S_f32toa = avx.S_f32toa
__F64toa = avx.F_f64toa
S_i64toa = avx.S_i64toa
__I64toa = avx.F_i64toa
S_u64toa = avx.S_u64toa
__U64toa = avx.F_u64toa
S_lspace = avx.S_lspace
S_quote = avx.S_quote
__Quote = avx.F_quote
S_unquote = avx.S_unquote
__Unquote = avx.F_unquote
S_value = avx.S_value
__Value = avx.F_value
S_vstring = avx.S_vstring
S_vnumber = avx.S_vnumber
S_vsigned = avx.S_vsigned
S_vunsigned = avx.S_vunsigned
S_skip_one = avx.S_skip_one
__SkipOne = avx.F_skip_one
__SkipOneFast = avx.F_skip_one_fast
S_skip_array = avx.S_skip_array
S_skip_object = avx.S_skip_object
S_skip_number = avx.S_skip_number
S_get_by_path = avx.S_get_by_path
__GetByPath = avx.F_get_by_path
__HTMLEscape = avx.F_html_escape
__ValidateOne = avx.F_validate_one
__ValidateUTF8= avx.F_validate_utf8
__ValidateUTF8Fast = avx.F_validate_utf8_fast
}
func useAVX2() {
@@ -214,7 +235,7 @@ func useAVX2() {
S_f64toa = avx2.S_f64toa
__F64toa = avx2.F_f64toa
S_f32toa = avx2.S_f32toa
__F32toa = avx2.F_f32toa
__F64toa = avx2.F_f64toa
S_i64toa = avx2.S_i64toa
__I64toa = avx2.F_i64toa
S_u64toa = avx2.S_u64toa
@@ -242,17 +263,17 @@ func useAVX2() {
__ValidateOne = avx2.F_validate_one
__ValidateUTF8= avx2.F_validate_utf8
__ValidateUTF8Fast = avx2.F_validate_utf8_fast
__ParseWithPadding = avx2.F_parse_with_padding
__LookupSmallKey = avx2.F_lookup_small_key
}
func init() {
if cpu.HasAVX2 {
useAVX2()
} else if cpu.HasSSE {
useSSE()
} else {
panic("Unsupported CPU, lacks of AVX2 or SSE CPUID Flag. maybe it's too old to run Sonic.")
}
if cpu.HasAVX2 {
useAVX2()
} else if cpu.HasAVX {
useAVX()
} else if cpu.HasSSE {
useSSE()
} else {
panic("Unsupported CPU, maybe it's too old to run Sonic.")
}
}

5
vendor/github.com/bytedance/sonic/internal/native/types/types.go generated vendored
View File

@@ -22,7 +22,7 @@ import (
`unsafe`
)
type ValueType = int64
type ValueType int
type ParsingError uint
type SearchingError uint
@@ -57,9 +57,6 @@ const (
B_USE_NUMBER = 1
B_VALIDATE_STRING = 5
B_ALLOW_CONTROL = 31
// for native.SkipOne() flags
B_NO_VALIDATE_JSON= 6
)
const (

40
vendor/github.com/bytedance/sonic/internal/resolver/resolver.go generated vendored
View File

@@ -17,11 +17,10 @@
package resolver
import (
"fmt"
"reflect"
"strings"
"sync"
_ "unsafe"
`fmt`
`reflect`
`strings`
`sync`
)
type FieldOpts int
@@ -30,7 +29,6 @@ type OffsetType int
const (
F_omitempty FieldOpts = 1 << iota
F_stringize
F_omitzero
)
const (
@@ -49,7 +47,6 @@ type FieldMeta struct {
Path []Offset
Opts FieldOpts
Type reflect.Type
IsZero func(reflect.Value) bool
}
func (self *FieldMeta) String() string {
@@ -120,26 +117,20 @@ func resolveFields(vt reflect.Type) []FieldMeta {
/* convert each field */
for _, fv := range tfv.list {
/* add to result */
ret = append(ret, FieldMeta{})
fm := &ret[len(ret)-1]
item := vt
path := []Offset(nil)
opts := FieldOpts(0)
/* check for "string" */
if fv.quoted {
fm.Opts |= F_stringize
opts |= F_stringize
}
/* check for "omitempty" */
if fv.omitEmpty {
fm.Opts |= F_omitempty
opts |= F_omitempty
}
/* handle the "omitzero" */
handleOmitZero(fv, fm)
/* dump the field path */
for _, i := range fv.index {
kind := F_offset
@@ -170,9 +161,13 @@ func resolveFields(vt reflect.Type) []FieldMeta {
path[idx].Kind = F_offset
}
fm.Type = fvt
fm.Path = path
fm.Name = fv.name
/* add to result */
ret = append(ret, FieldMeta {
Type: fvt,
Opts: opts,
Path: path,
Name: fv.name,
})
}
/* optimize the offsets */
@@ -217,10 +212,3 @@ func ResolveStruct(vt reflect.Type) []FieldMeta {
fieldCache[vt] = fm
return fm
}
func handleOmitZero(fv StdField, fm *FieldMeta) {
if fv.omitZero {
fm.Opts |= F_omitzero
fm.IsZero = fv.isZero
}
}

41
vendor/github.com/bytedance/sonic/internal/rt/asm_amd64.s generated vendored
View File

@@ -1,5 +1,4 @@
// +build !noasm,amd64 !appengine,amd64
// Code generated by asm2asm, DO NOT EDIT·
#include "go_asm.h"
#include "funcdata.h"
@@ -18,3 +17,43 @@ _entry:
_stack_grow:
CALL runtime·morestack_noctxt<>(SB)
JMP _entry
TEXT ·StopProf(SB), NOSPLIT, $0-0
NO_LOCAL_POINTERS
CMPB github·com∕bytedance∕sonic∕internal∕rt·StopProfiling(SB), $0
JEQ _ret_1
MOVL $1, AX
LEAQ github·com∕bytedance∕sonic∕internal∕rt·yieldCount(SB), CX
LOCK
XADDL AX, (CX)
MOVL runtime·prof+4(SB), AX
TESTL AX, AX
JEQ _ret_1
MOVL AX, github·com∕bytedance∕sonic∕internal∕rt·oldHz(SB)
MOVL $0, runtime·prof+4(SB)
_ret_1:
RET
TEXT ·StartProf(SB), NOSPLIT, $0-0
NO_LOCAL_POINTERS
CMPB github·com∕bytedance∕sonic∕internal∕rt·StopProfiling(SB), $0
JEQ _ret_2
MOVL $-1, AX
LEAQ github·com∕bytedance∕sonic∕internal∕rt·yieldCount(SB), CX
LOCK
XADDL AX, (CX)
CMPL github·com∕bytedance∕sonic∕internal∕rt·yieldCount(SB), $0
JNE _ret_2
CMPL runtime·prof+4(SB), $0
JNE _ret_2
CMPL github·com∕bytedance∕sonic∕internal∕rt·oldHz(SB), $0
JNE _branch_1
MOVL $100, github·com∕bytedance∕sonic∕internal∕rt·oldHz(SB)
_branch_1:
MOVL github·com∕bytedance∕sonic∕internal∕rt·oldHz(SB), AX
MOVL AX, runtime·prof+4(SB)
_ret_2:
RET

36
vendor/github.com/bytedance/sonic/internal/rt/fastmem.go generated vendored
View File

@@ -17,10 +17,8 @@
package rt
import (
"reflect"
"unsafe"
"github.com/bytedance/sonic/option"
`unsafe`
`reflect`
)
//go:nosplit
@@ -92,21 +90,6 @@ func GuardSlice(buf *[]byte, n int) {
}
}
func GuardSlice2(buf []byte, n int) []byte {
c := cap(buf)
l := len(buf)
if c-l < n {
c = c>>1 + n + l
if c < 32 {
c = 32
}
tmp := make([]byte, l, c)
copy(tmp, buf)
buf = tmp
}
return buf
}
//go:nosplit
func Ptr2SlicePtr(s unsafe.Pointer, l int, c int) unsafe.Pointer {
slice := &GoSlice{
@@ -139,17 +122,4 @@ func StrFrom(p unsafe.Pointer, n int64) (s string) {
func NoEscape(p unsafe.Pointer) unsafe.Pointer {
x := uintptr(p)
return unsafe.Pointer(x ^ 0)
}
//go:nosplit
func MoreStack(size uintptr)
//go:nosplit
func Add(ptr unsafe.Pointer, off uintptr) unsafe.Pointer {
return unsafe.Pointer(uintptr(ptr) + off)
}
// CanSizeResue
func CanSizeResue(cap int) bool {
return cap <= int(option.LimitBufferSize)
}
}

218
vendor/github.com/bytedance/sonic/internal/rt/fastvalue.go generated vendored
View File

@@ -17,179 +17,209 @@
package rt
import (
"reflect"
"unsafe"
`reflect`
`unsafe`
)
var (
reflectRtypeItab = findReflectRtypeItab()
reflectRtypeItab = findReflectRtypeItab()
)
// GoType.KindFlags const
const (
F_direct = 1 << 5
F_kind_mask = (1 << 5) - 1
F_direct = 1 << 5
F_kind_mask = (1 << 5) - 1
)
// GoType.Flags const
const (
tflagUncommon uint8 = 1 << 0
tflagExtraStar uint8 = 1 << 1
tflagNamed uint8 = 1 << 2
tflagRegularMemory uint8 = 1 << 3
tflagUncommon uint8 = 1 << 0
tflagExtraStar uint8 = 1 << 1
tflagNamed uint8 = 1 << 2
tflagRegularMemory uint8 = 1 << 3
)
type GoType struct {
Size uintptr
PtrData uintptr
Hash uint32
Flags uint8
Align uint8
FieldAlign uint8
KindFlags uint8
Traits unsafe.Pointer
GCData *byte
Str int32
PtrToSelf int32
Size uintptr
PtrData uintptr
Hash uint32
Flags uint8
Align uint8
FieldAlign uint8
KindFlags uint8
Traits unsafe.Pointer
GCData *byte
Str int32
PtrToSelf int32
}
func (self *GoType) IsNamed() bool {
return (self.Flags & tflagNamed) != 0
return (self.Flags & tflagNamed) != 0
}
func (self *GoType) Kind() reflect.Kind {
return reflect.Kind(self.KindFlags & F_kind_mask)
return reflect.Kind(self.KindFlags & F_kind_mask)
}
func (self *GoType) Pack() (t reflect.Type) {
(*GoIface)(unsafe.Pointer(&t)).Itab = reflectRtypeItab
(*GoIface)(unsafe.Pointer(&t)).Value = unsafe.Pointer(self)
return
(*GoIface)(unsafe.Pointer(&t)).Itab = reflectRtypeItab
(*GoIface)(unsafe.Pointer(&t)).Value = unsafe.Pointer(self)
return
}
func (self *GoType) String() string {
return self.Pack().String()
return self.Pack().String()
}
func (self *GoType) Indirect() bool {
return self.KindFlags&F_direct == 0
return self.KindFlags & F_direct == 0
}
type GoMap struct {
Count int
Flags uint8
B uint8
Overflow uint16
Hash0 uint32
Buckets unsafe.Pointer
OldBuckets unsafe.Pointer
Evacuate uintptr
Extra unsafe.Pointer
}
type GoMapIterator struct {
K unsafe.Pointer
V unsafe.Pointer
T *GoMapType
H *GoMap
Buckets unsafe.Pointer
Bptr *unsafe.Pointer
Overflow *[]unsafe.Pointer
OldOverflow *[]unsafe.Pointer
StartBucket uintptr
Offset uint8
Wrapped bool
B uint8
I uint8
Bucket uintptr
CheckBucket uintptr
}
type GoItab struct {
it unsafe.Pointer
Vt *GoType
hv uint32
_ [4]byte
fn [1]uintptr
it unsafe.Pointer
Vt *GoType
hv uint32
_ [4]byte
fn [1]uintptr
}
type GoIface struct {
Itab *GoItab
Value unsafe.Pointer
Itab *GoItab
Value unsafe.Pointer
}
type GoEface struct {
Type *GoType
Value unsafe.Pointer
Type *GoType
Value unsafe.Pointer
}
func (self GoEface) Pack() (v interface{}) {
*(*GoEface)(unsafe.Pointer(&v)) = self
return
*(*GoEface)(unsafe.Pointer(&v)) = self
return
}
type GoPtrType struct {
GoType
Elem *GoType
GoType
Elem *GoType
}
type GoMapType struct {
GoType
Key *GoType
Elem *GoType
Bucket *GoType
Hasher func(unsafe.Pointer, uintptr) uintptr
KeySize uint8
ElemSize uint8
BucketSize uint16
Flags uint32
GoType
Key *GoType
Elem *GoType
Bucket *GoType
Hasher func(unsafe.Pointer, uintptr) uintptr
KeySize uint8
ElemSize uint8
BucketSize uint16
Flags uint32
}
func (self *GoMapType) IndirectElem() bool {
return self.Flags&2 != 0
return self.Flags & 2 != 0
}
type GoStructType struct {
GoType
Pkg *byte
Fields []GoStructField
GoType
Pkg *byte
Fields []GoStructField
}
type GoStructField struct {
Name *byte
Type *GoType
OffEmbed uintptr
Name *byte
Type *GoType
OffEmbed uintptr
}
type GoInterfaceType struct {
GoType
PkgPath *byte
Methods []GoInterfaceMethod
GoType
PkgPath *byte
Methods []GoInterfaceMethod
}
type GoInterfaceMethod struct {
Name int32
Type int32
Name int32
Type int32
}
type GoSlice struct {
Ptr unsafe.Pointer
Len int
Cap int
Ptr unsafe.Pointer
Len int
Cap int
}
type GoString struct {
Ptr unsafe.Pointer
Len int
Ptr unsafe.Pointer
Len int
}
func PtrElem(t *GoType) *GoType {
return (*GoPtrType)(unsafe.Pointer(t)).Elem
return (*GoPtrType)(unsafe.Pointer(t)).Elem
}
func MapType(t *GoType) *GoMapType {
return (*GoMapType)(unsafe.Pointer(t))
return (*GoMapType)(unsafe.Pointer(t))
}
func IfaceType(t *GoType) *GoInterfaceType {
return (*GoInterfaceType)(unsafe.Pointer(t))
return (*GoInterfaceType)(unsafe.Pointer(t))
}
func UnpackType(t reflect.Type) *GoType {
return (*GoType)((*GoIface)(unsafe.Pointer(&t)).Value)
return (*GoType)((*GoIface)(unsafe.Pointer(&t)).Value)
}
func UnpackEface(v interface{}) GoEface {
return *(*GoEface)(unsafe.Pointer(&v))
return *(*GoEface)(unsafe.Pointer(&v))
}
func UnpackIface(v interface{}) GoIface {
return *(*GoIface)(unsafe.Pointer(&v))
return *(*GoIface)(unsafe.Pointer(&v))
}
func findReflectRtypeItab() *GoItab {
v := reflect.TypeOf(struct{}{})
return (*GoIface)(unsafe.Pointer(&v)).Itab
v := reflect.TypeOf(struct{}{})
return (*GoIface)(unsafe.Pointer(&v)).Itab
}
func AssertI2I2(t *GoType, i GoIface) (r GoIface) {
inter := IfaceType(t)
inter := IfaceType(t)
tab := i.Itab
if tab == nil {
return
}
if (*GoInterfaceType)(tab.it) != inter {
tab = GetItab(inter, tab.Vt, true)
tab = Getitab(inter, tab.Vt, true)
if tab == nil {
return
}
@@ -199,33 +229,15 @@ func AssertI2I2(t *GoType, i GoIface) (r GoIface) {
return
}
func (t *GoType) IsInt64() bool {
return t.Kind() == reflect.Int64 || (t.Kind() == reflect.Int && t.Size == 8)
}
func (t *GoType) IsInt32() bool {
return t.Kind() == reflect.Int32 || (t.Kind() == reflect.Int && t.Size == 4)
}
//go:nosplit
func (t *GoType) IsUint64() bool {
isUint := t.Kind() == reflect.Uint || t.Kind() == reflect.Uintptr
return t.Kind() == reflect.Uint64 || (isUint && t.Size == 8)
}
//go:nosplit
func (t *GoType) IsUint32() bool {
isUint := t.Kind() == reflect.Uint || t.Kind() == reflect.Uintptr
return t.Kind() == reflect.Uint32 || (isUint && t.Size == 4)
}
//go:nosplit
func PtrAdd(ptr unsafe.Pointer, offset uintptr) unsafe.Pointer {
return unsafe.Pointer(uintptr(ptr) + offset)
}
//go:noescape
//go:linkname GetItab runtime.getitab
func GetItab(inter *GoInterfaceType, typ *GoType, canfail bool) *GoItab
//go:linkname Getitab runtime.getitab
func Getitab(inter *GoInterfaceType, typ *GoType, canfail bool) *GoItab
func GetFuncPC(fn interface{}) uintptr {
ft := UnpackEface(fn)
if ft.Type.Kind() != reflect.Func {
panic("not a function")
}
return *(*uintptr)(ft.Value)
}

55
vendor/github.com/bytedance/sonic/internal/rt/gcwb.go generated vendored
View File

@@ -1,5 +1,3 @@
// +build go1.21,!go1.26
/*
* Copyright 2021 ByteDance Inc.
*
@@ -19,18 +17,13 @@
package rt
import (
`os`
`sync/atomic`
`unsafe`
`golang.org/x/arch/x86/x86asm`
)
//go:linkname GcWriteBarrier2 runtime.gcWriteBarrier2
func GcWriteBarrier2()
//go:linkname RuntimeWriteBarrier runtime.writeBarrier
var RuntimeWriteBarrier uintptr
const (
_MaxInstr = 15
)
@@ -83,3 +76,49 @@ func GcwbAddr() uintptr {
}
}
// StopProfiling is used to stop traceback introduced by SIGPROF while native code is running.
// WARN: this option is only a workaround for traceback issue (https://github.com/bytedance/sonic/issues/310),
// and will be dropped when the issue is fixed.
var StopProfiling = os.Getenv("SONIC_STOP_PROFILING") != ""
// WARN: must be aligned with runtime.Prof
// type Prof struct {
// signalLock uint32
// hz int32
// }
var (
// // go:linkname runtimeProf runtime.prof
// runtimeProf Prof
// count of native-C calls
yieldCount uint32
// previous value of runtimeProf.hz
oldHz int32
)
//go:nosplit
func MoreStack(size uintptr)
func StopProf()
// func StopProf() {
// atomic.AddUint32(&yieldCount, 1)
// if runtimeProf.hz != 0 {
// oldHz = runtimeProf.hz
// runtimeProf.hz = 0
// }
// }
func StartProf()
// func StartProf() {
// atomic.AddUint32(&yieldCount, ^uint32(0))
// if yieldCount == 0 && runtimeProf.hz == 0 {
// if oldHz == 0 {
// oldHz = 100
// }
// runtimeProf.hz = oldHz
// }
// }

32
vendor/github.com/bytedance/sonic/loader/stubs.go generated vendored
View File

@@ -17,6 +17,8 @@
package loader
import (
"sync/atomic"
"unsafe"
_ `unsafe`
)
@@ -24,5 +26,35 @@ import (
//goland:noinspection GoUnusedGlobalVariable
var lastmoduledatap *moduledata
func registerModule(mod *moduledata) {
registerModuleLockFree(&lastmoduledatap, mod)
}
//go:linkname moduledataverify1 runtime.moduledataverify1
func moduledataverify1(_ *moduledata)
func registerModuleLockFree(tail **moduledata, mod *moduledata) {
for {
oldTail := loadModule(tail)
if casModule(tail, oldTail, mod) {
storeModule(&oldTail.next, mod)
break
}
}
}
func loadModule(p **moduledata) *moduledata {
return (*moduledata)(atomic.LoadPointer((*unsafe.Pointer)(unsafe.Pointer(p))))
}
func storeModule(p **moduledata, value *moduledata) {
atomic.StorePointer((*unsafe.Pointer)(unsafe.Pointer(p)), unsafe.Pointer(value))
}
func casModule(p **moduledata, oldValue *moduledata, newValue *moduledata) bool {
return atomic.CompareAndSwapPointer(
(*unsafe.Pointer)(unsafe.Pointer(p)),
unsafe.Pointer(oldValue),
unsafe.Pointer(newValue),
)
}

14
vendor/github.com/bytedance/sonic/option/option.go generated vendored
View File

@@ -18,17 +18,10 @@ package option
var (
// DefaultDecoderBufferSize is the initial buffer size of StreamDecoder
DefaultDecoderBufferSize uint = 4 * 1024
DefaultDecoderBufferSize uint = 128 * 1024
// DefaultEncoderBufferSize is the initial buffer size of Encoder
DefaultEncoderBufferSize uint = 4 * 1024
// DefaultAstBufferSize is the initial buffer size of ast.Node.MarshalJSON()
DefaultAstBufferSize uint = 4 * 1024
// LimitBufferSize indicates the max pool buffer size, in case of OOM.
// See issue https://github.com/bytedance/sonic/issues/614
LimitBufferSize uint = 1024 * 1024
DefaultEncoderBufferSize uint = 128 * 1024
)
// CompileOptions includes all options for encoder or decoder compiler.
@@ -68,7 +61,7 @@ type CompileOption func(o *CompileOptions)
//
// For deep nested struct (depth exceeds MaxInlineDepth),
// try to set more loops to completely compile,
// thus reduce JIT instability in the first hit.
// thus reduce JIT unstability in the first hit.
func WithCompileRecursiveDepth(loop int) CompileOption {
return func(o *CompileOptions) {
if loop < 0 {
@@ -90,3 +83,4 @@ func WithCompileMaxInlineDepth(depth int) CompileOption {
o.MaxInlineDepth = depth
}
}

15
vendor/github.com/bytedance/sonic/sonic.go generated vendored
View File

@@ -1,5 +1,4 @@
//go:build (amd64 && go1.17 && !go1.26) || (arm64 && go1.20 && !go1.26)
// +build amd64,go1.17,!go1.26 arm64,go1.20,!go1.26
// +build amd64,go1.16,!go1.23
/*
* Copyright 2021 ByteDance Inc.
@@ -17,6 +16,7 @@
* limitations under the License.
*/
//go:generate make
package sonic
import (
@@ -29,8 +29,6 @@ import (
`github.com/bytedance/sonic/internal/rt`
)
const apiKind = UseSonicJSON
type frozenConfig struct {
Config
encoderOpts encoder.Options
@@ -66,14 +64,8 @@ func (cfg Config) Froze() API {
if cfg.NoEncoderNewline {
api.encoderOpts |= encoder.NoEncoderNewline
}
if cfg.EncodeNullForInfOrNan {
api.encoderOpts |= encoder.EncodeNullForInfOrNan
}
// configure decoder options:
if cfg.NoValidateJSONSkip {
api.decoderOpts |= decoder.OptionNoValidateJSON
}
if cfg.UseInt64 {
api.decoderOpts |= decoder.OptionUseInt64
}
@@ -89,9 +81,6 @@ func (cfg Config) Froze() API {
if cfg.ValidateString {
api.decoderOpts |= decoder.OptionValidateString
}
if cfg.CaseSensitive {
api.decoderOpts |= decoder.OptionCaseSensitive
}
return api
}

8
vendor/github.com/bytedance/sonic/unquote/unquote.go generated vendored
View File

@@ -1,7 +1,3 @@
//go:build (amd64 && go1.17 && !go1.26) || (arm64 && go1.20 && !go1.26)
// +build amd64,go1.17,!go1.26 arm64,go1.20,!go1.26
/*
* Copyright 2021 ByteDance Inc.
*
@@ -29,7 +25,7 @@ import (
`github.com/bytedance/sonic/internal/rt`
)
// String unescapes an escaped string (not including `"` at beginning and end)
// String unescapes a escaped string (not including `"` at begining and end)
// It validates invalid UTF8 and replace with `\ufffd`
func String(s string) (ret string, err types.ParsingError) {
mm := make([]byte, 0, len(s))
@@ -47,7 +43,7 @@ func IntoBytes(s string, m *[]byte) types.ParsingError {
}
}
// String unescapes an escaped string (not including `"` at beginning and end)
// String unescapes a escaped string (not including `"` at begining and end)
// - replace enables replacing invalid utf8 escaped char with `\uffd`
func _String(s string, replace bool) (ret string, err error) {
mm := make([]byte, 0, len(s))

19
vendor/github.com/bytedance/sonic/utf8/utf8.go generated vendored
View File

@@ -1,6 +1,3 @@
//go:build (amd64 && go1.17 && !go1.26) || (arm64 && go1.20 && !go1.26)
// +build amd64,go1.17,!go1.26 arm64,go1.20,!go1.26
/*
* Copyright 2022 ByteDance Inc.
*
@@ -20,8 +17,6 @@
package utf8
import (
`runtime`
`github.com/bytedance/sonic/internal/rt`
`github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/internal/native`
@@ -32,7 +27,7 @@ func CorrectWith(dst []byte, src []byte, repl string) []byte {
sstr := rt.Mem2Str(src)
sidx := 0
/* state machine records the invalid positions */
/* state machine records the invalid postions */
m := types.NewStateMachine()
m.Sp = 0 // invalid utf8 numbers
@@ -65,20 +60,12 @@ func CorrectWith(dst []byte, src []byte, repl string) []byte {
return dst
}
// Validate is a simd-accelerated drop-in replacement for the standard library's utf8.Valid.
// Validate is a simd-accelereated drop-in replacement for the standard library's utf8.Valid.
func Validate(src []byte) bool {
if src == nil {
return true
}
return ValidateString(rt.Mem2Str(src))
}
// ValidateString as Validate, but for string.
func ValidateString(src string) bool {
if src == "" {
return true
}
ret := native.ValidateUTF8Fast(&src) == 0
runtime.KeepAlive(src)
return ret
return native.ValidateUTF8Fast(&src) == 0
}

2
vendor/github.com/gabriel-vasile/mimetype/LICENSE generated vendored
View File

@@ -1,6 +1,6 @@
MIT License
Copyright (c) 2018 Gabriel Vasile
Copyright (c) 2018-2020 Gabriel Vasile
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal

40
vendor/github.com/gabriel-vasile/mimetype/README.md generated vendored
View File

@@ -16,6 +16,9 @@
<a href="https://goreportcard.com/report/github.com/gabriel-vasile/mimetype">
<img alt="Go report card" src="https://goreportcard.com/badge/github.com/gabriel-vasile/mimetype">
</a>
<a href="https://codecov.io/gh/gabriel-vasile/mimetype">
<img alt="Code coverage" src="https://codecov.io/gh/gabriel-vasile/mimetype/branch/master/graph/badge.svg?token=qcfJF1kkl2"/>
</a>
<a href="LICENSE">
<img alt="License" src="https://img.shields.io/badge/License-MIT-green.svg">
</a>
@@ -27,7 +30,6 @@
- possibility to [extend](https://pkg.go.dev/github.com/gabriel-vasile/mimetype#example-package-Extend) with other file formats
- common file formats are prioritized
- [text vs. binary files differentiation](https://pkg.go.dev/github.com/gabriel-vasile/mimetype#example-package-TextVsBinary)
- no external dependencies
- safe for concurrent usage
## Install
@@ -46,7 +48,8 @@ fmt.Println(mtype.String(), mtype.Extension())
```
See the [runnable Go Playground examples](https://pkg.go.dev/github.com/gabriel-vasile/mimetype#pkg-overview).
Caution: only use libraries like **mimetype** as a last resort. Content type detection
## Usage'
Only use libraries like **mimetype** as a last resort. Content type detection
using magic numbers is slow, inaccurate, and non-standard. Most of the times
protocols have methods for specifying such metadata; e.g., `Content-Type` header
in HTTP and SMTP.
@@ -67,18 +70,6 @@ mimetype.DetectFile("file.doc")
If increasing the limit does not help, please
[open an issue](https://github.com/gabriel-vasile/mimetype/issues/new?assignees=&labels=&template=mismatched-mime-type-detected.md&title=).
## Tests
In addition to unit tests,
[mimetype_tests](https://github.com/gabriel-vasile/mimetype_tests) compares the
library with the [Unix file utility](https://en.wikipedia.org/wiki/File_(command))
for around 50 000 sample files. Check the latest comparison results
[here](https://github.com/gabriel-vasile/mimetype_tests/actions).
## Benchmarks
Benchmarks for each file format are performed when a PR is open. The results can
be seen on the [workflows page](https://github.com/gabriel-vasile/mimetype/actions/workflows/benchmark.yml).
Performance improvements are welcome but correctness is prioritized.
## Structure
**mimetype** uses a hierarchical structure to keep the MIME type detection logic.
This reduces the number of calls needed for detecting the file type. The reason
@@ -93,11 +84,22 @@ To prevent loading entire files into memory, when detecting from a
or from a [file](https://pkg.go.dev/github.com/gabriel-vasile/mimetype#DetectFile)
**mimetype** limits itself to reading only the header of the input.
<div align="center">
<img alt="how project is structured" src="https://raw.githubusercontent.com/gabriel-vasile/mimetype/master/testdata/gif.gif" width="88%">
<img alt="structure" src="https://github.com/gabriel-vasile/mimetype/blob/420a05228c6a6efbb6e6f080168a25663414ff36/mimetype.gif?raw=true" width="88%">
</div>
## Performance
Thanks to the hierarchical structure, searching for common formats first,
and limiting itself to file headers, **mimetype** matches the performance of
stdlib `http.DetectContentType` while outperforming the alternative package.
```bash
mimetype http.DetectContentType filetype
BenchmarkMatchTar-24 250 ns/op 400 ns/op 3778 ns/op
BenchmarkMatchZip-24 524 ns/op 351 ns/op 4884 ns/op
BenchmarkMatchJpeg-24 103 ns/op 228 ns/op 839 ns/op
BenchmarkMatchGif-24 139 ns/op 202 ns/op 751 ns/op
BenchmarkMatchPng-24 165 ns/op 221 ns/op 1176 ns/op
```
## Contributing
Contributions are unexpected but welcome. When submitting a PR for detection of
a new file format, please make sure to add a record to the list of testcases
from [mimetype_test.go](mimetype_test.go). For complex files a record can be added
in the [testdata](testdata) directory.
See [CONTRIBUTING.md](CONTRIBUTING.md).

232
vendor/github.com/gabriel-vasile/mimetype/internal/charset/charset.go generated vendored
View File

@@ -2,10 +2,11 @@ package charset
import (
"bytes"
"encoding/xml"
"strings"
"unicode/utf8"
"github.com/gabriel-vasile/mimetype/internal/markup"
"github.com/gabriel-vasile/mimetype/internal/scan"
"golang.org/x/net/html"
)
const (
@@ -140,31 +141,20 @@ func FromXML(content []byte) string {
}
return FromPlain(content)
}
func fromXML(s scan.Bytes) string {
xml := []byte("<?XML")
lxml := len(xml)
for {
if len(s) == 0 {
return ""
}
for scan.ByteIsWS(s.Peek()) {
s.Advance(1)
}
if len(s) <= lxml {
return ""
}
if !s.Match(xml, scan.IgnoreCase) {
s = s[1:] // safe to slice instead of s.Advance(1) because bounds are checked
continue
}
aName, aVal, hasMore := "", "", true
for hasMore {
aName, aVal, hasMore = markup.GetAnAttribute(&s)
if aName == "encoding" && aVal != "" {
return aVal
}
}
func fromXML(content []byte) string {
content = trimLWS(content)
dec := xml.NewDecoder(bytes.NewReader(content))
rawT, err := dec.RawToken()
if err != nil {
return ""
}
t, ok := rawT.(xml.ProcInst)
if !ok {
return ""
}
return strings.ToLower(xmlEncoding(string(t.Inst)))
}
// FromHTML returns the charset of an HTML document. It first looks if a BOM is
@@ -181,103 +171,139 @@ func FromHTML(content []byte) string {
return FromPlain(content)
}
func fromHTML(s scan.Bytes) string {
const (
dontKnow = iota
doNeedPragma
doNotNeedPragma
)
meta := []byte("<META")
body := []byte("<BODY")
lmeta := len(meta)
func fromHTML(content []byte) string {
z := html.NewTokenizer(bytes.NewReader(content))
for {
if markup.SkipAComment(&s) {
continue
}
if len(s) <= lmeta {
switch z.Next() {
case html.ErrorToken:
return ""
}
// Abort when <body is reached.
if s.Match(body, scan.IgnoreCase) {
return ""
}
if !s.Match(meta, scan.IgnoreCase) {
s = s[1:] // safe to slice instead of s.Advance(1) because bounds are checked
continue
}
s = s[lmeta:]
c := s.Pop()
if c == 0 || (!scan.ByteIsWS(c) && c != '/') {
return ""
}
attrList := make(map[string]bool)
gotPragma := false
needPragma := dontKnow
charset := ""
aName, aVal, hasMore := "", "", true
for hasMore {
aName, aVal, hasMore = markup.GetAnAttribute(&s)
if attrList[aName] {
case html.StartTagToken, html.SelfClosingTagToken:
tagName, hasAttr := z.TagName()
if !bytes.Equal(tagName, []byte("meta")) {
continue
}
// processing step
if len(aName) == 0 && len(aVal) == 0 {
if needPragma == dontKnow {
attrList := make(map[string]bool)
gotPragma := false
const (
dontKnow = iota
doNeedPragma
doNotNeedPragma
)
needPragma := dontKnow
name := ""
for hasAttr {
var key, val []byte
key, val, hasAttr = z.TagAttr()
ks := string(key)
if attrList[ks] {
continue
}
if needPragma == doNeedPragma && !gotPragma {
continue
attrList[ks] = true
for i, c := range val {
if 'A' <= c && c <= 'Z' {
val[i] = c + 0x20
}
}
switch ks {
case "http-equiv":
if bytes.Equal(val, []byte("content-type")) {
gotPragma = true
}
case "content":
name = fromMetaElement(string(val))
if name != "" {
needPragma = doNeedPragma
}
case "charset":
name = string(val)
needPragma = doNotNeedPragma
}
}
attrList[aName] = true
if aName == "http-equiv" && scan.Bytes(aVal).Match([]byte("CONTENT-TYPE"), scan.IgnoreCase) {
gotPragma = true
} else if aName == "content" {
charset = string(extractCharsetFromMeta(scan.Bytes(aVal)))
if len(charset) != 0 {
needPragma = doNeedPragma
}
} else if aName == "charset" {
charset = aVal
needPragma = doNotNeedPragma
if needPragma == dontKnow || needPragma == doNeedPragma && !gotPragma {
continue
}
}
if needPragma == dontKnow || needPragma == doNeedPragma && !gotPragma {
continue
}
if strings.HasPrefix(name, "utf-16") {
name = "utf-8"
}
return charset
return name
}
}
}
// https://html.spec.whatwg.org/multipage/urls-and-fetching.html#algorithm-for-extracting-a-character-encoding-from-a-meta-element
func extractCharsetFromMeta(s scan.Bytes) []byte {
for {
i := bytes.Index(s, []byte("charset"))
if i == -1 {
return nil
func fromMetaElement(s string) string {
for s != "" {
csLoc := strings.Index(s, "charset")
if csLoc == -1 {
return ""
}
s.Advance(i + len("charset"))
for scan.ByteIsWS(s.Peek()) {
s.Advance(1)
}
if s.Pop() != '=' {
s = s[csLoc+len("charset"):]
s = strings.TrimLeft(s, " \t\n\f\r")
if !strings.HasPrefix(s, "=") {
continue
}
for scan.ByteIsWS(s.Peek()) {
s.Advance(1)
s = s[1:]
s = strings.TrimLeft(s, " \t\n\f\r")
if s == "" {
return ""
}
quote := s.Peek()
if quote == 0 {
return nil
}
if quote == '"' || quote == '\'' {
s.Advance(1)
return bytes.TrimSpace(s.PopUntil(quote))
if q := s[0]; q == '"' || q == '\'' {
s = s[1:]
closeQuote := strings.IndexRune(s, rune(q))
if closeQuote == -1 {
return ""
}
return s[:closeQuote]
}
return bytes.TrimSpace(s.PopUntil(';', '\t', '\n', '\x0c', '\r', ' '))
end := strings.IndexAny(s, "; \t\n\f\r")
if end == -1 {
end = len(s)
}
return s[:end]
}
return ""
}
func xmlEncoding(s string) string {
param := "encoding="
idx := strings.Index(s, param)
if idx == -1 {
return ""
}
v := s[idx+len(param):]
if v == "" {
return ""
}
if v[0] != '\'' && v[0] != '"' {
return ""
}
idx = strings.IndexRune(v[1:], rune(v[0]))
if idx == -1 {
return ""
}
return v[1 : idx+1]
}
// trimLWS trims whitespace from beginning of the input.
// TODO: find a way to call trimLWS once per detection instead of once in each
// detector which needs the trimmed input.
func trimLWS(in []byte) []byte {
firstNonWS := 0
for ; firstNonWS < len(in) && isWS(in[firstNonWS]); firstNonWS++ {
}
return in[firstNonWS:]
}
func isWS(b byte) bool {
return b == '\t' || b == '\n' || b == '\x0c' || b == '\r' || b == ' '
}

115
vendor/github.com/gabriel-vasile/mimetype/internal/magic/archive.go generated vendored
View File

@@ -52,15 +52,10 @@ func InstallShieldCab(raw []byte, _ uint32) bool {
}
// Zstd matches a Zstandard archive file.
// https://github.com/facebook/zstd/blob/dev/doc/zstd_compression_format.md
func Zstd(raw []byte, limit uint32) bool {
if len(raw) < 4 {
return false
}
sig := binary.LittleEndian.Uint32(raw)
// Check for Zstandard frames and skippable frames.
return (sig >= 0xFD2FB522 && sig <= 0xFD2FB528) ||
(sig >= 0x184D2A50 && sig <= 0x184D2A5F)
return len(raw) >= 4 &&
(0x22 <= raw[0] && raw[0] <= 0x28 || raw[0] == 0x1E) && // Different Zstandard versions.
bytes.HasPrefix(raw[1:], []byte{0xB5, 0x2F, 0xFD})
}
// CRX matches a Chrome extension file: a zip archive prepended by a package header.
@@ -79,85 +74,51 @@ func CRX(raw []byte, limit uint32) bool {
}
// Tar matches a (t)ape (ar)chive file.
// Tar files are divided into 512 bytes records. First record contains a 257
// bytes header padded with NUL.
func Tar(raw []byte, _ uint32) bool {
const sizeRecord = 512
// The structure of a tar header:
// type TarHeader struct {
// Name [100]byte
// Mode [8]byte
// Uid [8]byte
// Gid [8]byte
// Size [12]byte
// Mtime [12]byte
// Chksum [8]byte
// Linkflag byte
// Linkname [100]byte
// Magic [8]byte
// Uname [32]byte
// Gname [32]byte
// Devmajor [8]byte
// Devminor [8]byte
// }
if len(raw) < sizeRecord {
return false
// The "magic" header field for files in in UStar (POSIX IEEE P1003.1) archives
// has the prefix "ustar". The values of the remaining bytes in this field vary
// by archiver implementation.
if len(raw) >= 512 && bytes.HasPrefix(raw[257:], []byte{0x75, 0x73, 0x74, 0x61, 0x72}) {
return true
}
raw = raw[:sizeRecord]
// First 100 bytes of the header represent the file name.
// Check if file looks like Gentoo GLEP binary package.
if bytes.Contains(raw[:100], []byte("/gpkg-1\x00")) {
if len(raw) < 256 {
return false
}
// Get the checksum recorded into the file.
recsum := tarParseOctal(raw[148:156])
if recsum == -1 {
return false
// The older v7 format has no "magic" field, and therefore must be identified
// with heuristics based on legal ranges of values for other header fields:
// https://www.nationalarchives.gov.uk/PRONOM/Format/proFormatSearch.aspx?status=detailReport&id=385&strPageToDisplay=signatures
rules := []struct {
min, max uint8
i int
}{
{0x21, 0xEF, 0},
{0x30, 0x37, 105},
{0x20, 0x37, 106},
{0x00, 0x00, 107},
{0x30, 0x37, 113},
{0x20, 0x37, 114},
{0x00, 0x00, 115},
{0x30, 0x37, 121},
{0x20, 0x37, 122},
{0x00, 0x00, 123},
{0x30, 0x37, 134},
{0x30, 0x37, 146},
{0x30, 0x37, 153},
{0x00, 0x37, 154},
}
sum1, sum2 := tarChksum(raw)
return recsum == sum1 || recsum == sum2
}
// tarParseOctal converts octal string to decimal int.
func tarParseOctal(b []byte) int64 {
// Because unused fields are filled with NULs, we need to skip leading NULs.
// Fields may also be padded with spaces or NULs.
// So we remove leading and trailing NULs and spaces to be sure.
b = bytes.Trim(b, " \x00")
if len(b) == 0 {
return -1
}
ret := int64(0)
for _, b := range b {
if b == 0 {
break
for _, r := range rules {
if raw[r.i] < r.min || raw[r.i] > r.max {
return false
}
if b < '0' || b > '7' {
return -1
}
ret = (ret << 3) | int64(b-'0')
}
return ret
}
// tarChksum computes the checksum for the header block b.
// The actual checksum is written to same b block after it has been calculated.
// Before calculation the bytes from b reserved for checksum have placeholder
// value of ASCII space 0x20.
// POSIX specifies a sum of the unsigned byte values, but the Sun tar used
// signed byte values. We compute and return both.
func tarChksum(b []byte) (unsigned, signed int64) {
for i, c := range b {
if 148 <= i && i < 156 {
c = ' ' // Treat the checksum field itself as all spaces.
for _, i := range []uint8{135, 147, 155} {
if raw[i] != 0x00 && raw[i] != 0x20 {
return false
}
unsigned += int64(c)
signed += int64(int8(c))
}
return unsigned, signed
return true
}

41
vendor/github.com/gabriel-vasile/mimetype/internal/magic/binary.go generated vendored
View File

@@ -21,10 +21,6 @@ var (
SWF = prefix([]byte("CWS"), []byte("FWS"), []byte("ZWS"))
// Torrent has bencoded text in the beginning.
Torrent = prefix([]byte("d8:announce"))
// PAR1 matches a parquet file.
Par1 = prefix([]byte{0x50, 0x41, 0x52, 0x31})
// CBOR matches a Concise Binary Object Representation https://cbor.io/
CBOR = prefix([]byte{0xD9, 0xD9, 0xF7})
)
// Java bytecode and Mach-O binaries share the same magic number.
@@ -36,7 +32,7 @@ func classOrMachOFat(in []byte) bool {
return false
}
return binary.BigEndian.Uint32(in) == macho.MagicFat
return bytes.HasPrefix(in, []byte{0xCA, 0xFE, 0xBA, 0xBE})
}
// Class matches a java class file.
@@ -46,7 +42,7 @@ func Class(raw []byte, limit uint32) bool {
// MachO matches Mach-O binaries format.
func MachO(raw []byte, limit uint32) bool {
if classOrMachOFat(raw) && raw[7] < 0x14 {
if classOrMachOFat(raw) && raw[7] < 20 {
return true
}
@@ -71,7 +67,7 @@ func Dbf(raw []byte, limit uint32) bool {
}
// 3rd and 4th bytes contain the last update month and day of month.
if raw[2] == 0 || raw[2] > 12 || raw[3] == 0 || raw[3] > 31 {
if !(0 < raw[2] && raw[2] < 13 && 0 < raw[3] && raw[3] < 32) {
return false
}
@@ -153,36 +149,35 @@ func Marc(raw []byte, limit uint32) bool {
return bytes.Contains(raw[:min(2048, len(raw))], []byte{0x1E})
}
// GLB matches a glTF model format file.
// Glb matches a glTF model format file.
// GLB is the binary file format representation of 3D models saved in
// the GL transmission Format (glTF).
// GLB uses little endian and its header structure is as follows:
//
// <-- 12-byte header -->
// | magic | version | length |
// | (uint32) | (uint32) | (uint32) |
// | \x67\x6C\x54\x46 | \x01\x00\x00\x00 | ... |
// | g l T F | 1 | ... |
// <-- 12-byte header -->
// | magic | version | length |
// | (uint32) | (uint32) | (uint32) |
// | \x67\x6C\x54\x46 | \x01\x00\x00\x00 | ... |
// | g l T F | 1 | ... |
//
// Visit [glTF specification] and [IANA glTF entry] for more details.
//
// [glTF specification]: https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html
// [IANA glTF entry]: https://www.iana.org/assignments/media-types/model/gltf-binary
var GLB = prefix([]byte("\x67\x6C\x54\x46\x02\x00\x00\x00"),
var Glb = prefix([]byte("\x67\x6C\x54\x46\x02\x00\x00\x00"),
[]byte("\x67\x6C\x54\x46\x01\x00\x00\x00"))
// TzIf matches a Time Zone Information Format (TZif) file.
// See more: https://tools.ietf.org/id/draft-murchison-tzdist-tzif-00.html#rfc.section.3
// Its header structure is shown below:
//
// +---------------+---+
// | magic (4) | <-+-- version (1)
// +---------------+---+---------------------------------------+
// | [unused - reserved for future use] (15) |
// +---------------+---------------+---------------+-----------+
// | isutccnt (4) | isstdcnt (4) | leapcnt (4) |
// +---------------+---------------+---------------+
// | timecnt (4) | typecnt (4) | charcnt (4) |
// +---------------+---+
// | magic (4) | <-+-- version (1)
// +---------------+---+---------------------------------------+
// | [unused - reserved for future use] (15) |
// +---------------+---------------+---------------+-----------+
// | isutccnt (4) | isstdcnt (4) | leapcnt (4) |
// +---------------+---------------+---------------+
// | timecnt (4) | typecnt (4) | charcnt (4) |
func TzIf(raw []byte, limit uint32) bool {
// File is at least 44 bytes (header size).
if len(raw) < 44 {

47
vendor/github.com/gabriel-vasile/mimetype/internal/magic/document.go generated vendored
View File

@@ -1,11 +1,18 @@
package magic
import (
"bytes"
"encoding/binary"
)
import "bytes"
var (
// Pdf matches a Portable Document Format file.
// https://github.com/file/file/blob/11010cc805546a3e35597e67e1129a481aed40e8/magic/Magdir/pdf
Pdf = prefix(
// usual pdf signature
[]byte("%PDF-"),
// new-line prefixed signature
[]byte("\012%PDF-"),
// UTF-8 BOM prefixed signature
[]byte("\xef\xbb\xbf%PDF-"),
)
// Fdf matches a Forms Data Format file.
Fdf = prefix([]byte("%FDF"))
// Mobi matches a Mobi file.
@@ -14,18 +21,8 @@ var (
Lit = prefix([]byte("ITOLITLS"))
)
// PDF matches a Portable Document Format file.
// The %PDF- header should be the first thing inside the file but many
// implementations don't follow the rule. The PDF spec at Appendix H says the
// signature can be prepended by anything.
// https://bugs.astron.com/view.php?id=446
func PDF(raw []byte, _ uint32) bool {
raw = raw[:min(len(raw), 1024)]
return bytes.Contains(raw, []byte("%PDF-"))
}
// DjVu matches a DjVu file.
func DjVu(raw []byte, _ uint32) bool {
func DjVu(raw []byte, limit uint32) bool {
if len(raw) < 12 {
return false
}
@@ -39,7 +36,7 @@ func DjVu(raw []byte, _ uint32) bool {
}
// P7s matches an .p7s signature File (PEM, Base64).
func P7s(raw []byte, _ uint32) bool {
func P7s(raw []byte, limit uint32) bool {
// Check for PEM Encoding.
if bytes.HasPrefix(raw, []byte("-----BEGIN PKCS7")) {
return true
@@ -63,21 +60,3 @@ func P7s(raw []byte, _ uint32) bool {
return false
}
// Lotus123 matches a Lotus 1-2-3 spreadsheet document.
func Lotus123(raw []byte, _ uint32) bool {
if len(raw) <= 20 {
return false
}
version := binary.BigEndian.Uint32(raw)
if version == 0x00000200 {
return raw[6] != 0 && raw[7] == 0
}
return version == 0x00001a00 && raw[20] > 0 && raw[20] < 32
}
// CHM matches a Microsoft Compiled HTML Help file.
func CHM(raw []byte, _ uint32) bool {
return bytes.HasPrefix(raw, []byte("ITSF\003\000\000\000\x60\000\000\000"))
}

43
vendor/github.com/gabriel-vasile/mimetype/internal/magic/ftyp.go generated vendored
View File

@@ -1,14 +1,22 @@
package magic
import (
"bytes"
)
import "bytes"
var (
// AVIF matches an AV1 Image File Format still or animated.
// Wikipedia page seems outdated listing image/avif-sequence for animations.
// https://github.com/AOMediaCodec/av1-avif/issues/59
AVIF = ftyp([]byte("avif"), []byte("avis"))
// Mp4 matches an MP4 file.
Mp4 = ftyp(
[]byte("avc1"), []byte("dash"), []byte("iso2"), []byte("iso3"),
[]byte("iso4"), []byte("iso5"), []byte("iso6"), []byte("isom"),
[]byte("mmp4"), []byte("mp41"), []byte("mp42"), []byte("mp4v"),
[]byte("mp71"), []byte("MSNV"), []byte("NDAS"), []byte("NDSC"),
[]byte("NSDC"), []byte("NSDH"), []byte("NDSM"), []byte("NDSP"),
[]byte("NDSS"), []byte("NDXC"), []byte("NDXH"), []byte("NDXM"),
[]byte("NDXP"), []byte("NDXS"), []byte("F4V "), []byte("F4P "),
)
// ThreeGP matches a 3GPP file.
ThreeGP = ftyp(
[]byte("3gp1"), []byte("3gp2"), []byte("3gp3"), []byte("3gp4"),
@@ -45,17 +53,6 @@ var (
Heif = ftyp([]byte("mif1"), []byte("heim"), []byte("heis"), []byte("avic"))
// HeifSequence matches a High Efficiency Image File Format (HEIF) file sequence.
HeifSequence = ftyp([]byte("msf1"), []byte("hevm"), []byte("hevs"), []byte("avcs"))
// Mj2 matches a Motion JPEG 2000 file: https://en.wikipedia.org/wiki/Motion_JPEG_2000.
Mj2 = ftyp([]byte("mj2s"), []byte("mjp2"), []byte("MFSM"), []byte("MGSV"))
// Dvb matches a Digital Video Broadcasting file: https://dvb.org.
// https://cconcolato.github.io/mp4ra/filetype.html
// https://github.com/file/file/blob/512840337ead1076519332d24fefcaa8fac36e06/magic/Magdir/animation#L135-L154
Dvb = ftyp(
[]byte("dby1"), []byte("dsms"), []byte("dts1"), []byte("dts2"),
[]byte("dts3"), []byte("dxo "), []byte("dmb1"), []byte("dmpf"),
[]byte("drc1"), []byte("dv1a"), []byte("dv1b"), []byte("dv2a"),
[]byte("dv2b"), []byte("dv3a"), []byte("dv3b"), []byte("dvr1"),
[]byte("dvt1"), []byte("emsg"))
// TODO: add support for remaining video formats at ftyps.com.
)
@@ -89,21 +86,3 @@ func QuickTime(raw []byte, _ uint32) bool {
}
return bytes.Equal(raw[:8], []byte("\x00\x00\x00\x08wide"))
}
// Mp4 detects an .mp4 file. Mp4 detections only does a basic ftyp check.
// Mp4 has many registered and unregistered code points so it's hard to keep track
// of all. Detection will default on video/mp4 for all ftyp files.
// ISO_IEC_14496-12 is the specification for the iso container.
func Mp4(raw []byte, _ uint32) bool {
if len(raw) < 12 {
return false
}
// ftyps are made out of boxes. The first 4 bytes of the box represent
// its size in big-endian uint32. First box is the ftyp box and it is small
// in size. Check most significant byte is 0 to filter out false positive
// text files that happen to contain the string "ftyp" at index 4.
if raw[0] != 0 {
return false
}
return bytes.Equal(raw[4:8], []byte("ftyp"))
}

14
vendor/github.com/gabriel-vasile/mimetype/internal/magic/geo.go generated vendored
View File

@@ -12,13 +12,13 @@ func Shp(raw []byte, limit uint32) bool {
return false
}
if binary.BigEndian.Uint32(raw[0:4]) != 9994 ||
binary.BigEndian.Uint32(raw[4:8]) != 0 ||
binary.BigEndian.Uint32(raw[8:12]) != 0 ||
binary.BigEndian.Uint32(raw[12:16]) != 0 ||
binary.BigEndian.Uint32(raw[16:20]) != 0 ||
binary.BigEndian.Uint32(raw[20:24]) != 0 ||
binary.LittleEndian.Uint32(raw[28:32]) != 1000 {
if !(binary.BigEndian.Uint32(raw[0:4]) == 9994 &&
binary.BigEndian.Uint32(raw[4:8]) == 0 &&
binary.BigEndian.Uint32(raw[8:12]) == 0 &&
binary.BigEndian.Uint32(raw[12:16]) == 0 &&
binary.BigEndian.Uint32(raw[16:20]) == 0 &&
binary.BigEndian.Uint32(raw[20:24]) == 0 &&
binary.LittleEndian.Uint32(raw[28:32]) == 1000) {
return false
}

73
vendor/github.com/gabriel-vasile/mimetype/internal/magic/magic.go generated vendored
View File

@@ -4,8 +4,6 @@ package magic
import (
"bytes"
"fmt"
"github.com/gabriel-vasile/mimetype/internal/scan"
)
type (
@@ -76,13 +74,12 @@ func ciCheck(sig, raw []byte) bool {
// matches the raw input.
func xml(sigs ...xmlSig) Detector {
return func(raw []byte, limit uint32) bool {
b := scan.Bytes(raw)
b.TrimLWS()
if len(b) == 0 {
raw = trimLWS(raw)
if len(raw) == 0 {
return false
}
for _, s := range sigs {
if xmlCheck(s, b) {
if xmlCheck(s, raw) {
return true
}
}
@@ -107,19 +104,19 @@ func xmlCheck(sig xmlSig, raw []byte) bool {
// matches the raw input.
func markup(sigs ...[]byte) Detector {
return func(raw []byte, limit uint32) bool {
b := scan.Bytes(raw)
if bytes.HasPrefix(b, []byte{0xEF, 0xBB, 0xBF}) {
if bytes.HasPrefix(raw, []byte{0xEF, 0xBB, 0xBF}) {
// We skip the UTF-8 BOM if present to ensure we correctly
// process any leading whitespace. The presence of the BOM
// is taken into account during charset detection in charset.go.
b.Advance(3)
raw = trimLWS(raw[3:])
} else {
raw = trimLWS(raw)
}
b.TrimLWS()
if len(b) == 0 {
if len(raw) == 0 {
return false
}
for _, s := range sigs {
if markupCheck(s, b) {
if markupCheck(s, raw) {
return true
}
}
@@ -142,7 +139,7 @@ func markupCheck(sig, raw []byte) bool {
}
}
// Next byte must be space or right angle bracket.
if db := raw[len(sig)]; !scan.ByteIsWS(db) && db != '>' {
if db := raw[len(sig)]; db != ' ' && db != '>' {
return false
}
@@ -157,7 +154,7 @@ func ftyp(sigs ...[]byte) Detector {
return false
}
for _, s := range sigs {
if bytes.Equal(raw[8:12], s) {
if bytes.Equal(raw[4:12], append([]byte("ftyp"), s...)) {
return true
}
}
@@ -186,10 +183,8 @@ func newXMLSig(localName, xmlns string) xmlSig {
// /usr/bin/env is the interpreter, php is the first and only argument.
func shebang(sigs ...[]byte) Detector {
return func(raw []byte, limit uint32) bool {
b := scan.Bytes(raw)
line := b.Line()
for _, s := range sigs {
if shebangCheck(s, line) {
if shebangCheck(s, firstLine(raw)) {
return true
}
}
@@ -197,7 +192,7 @@ func shebang(sigs ...[]byte) Detector {
}
}
func shebangCheck(sig []byte, raw scan.Bytes) bool {
func shebangCheck(sig, raw []byte) bool {
if len(raw) < len(sig)+2 {
return false
}
@@ -205,8 +200,42 @@ func shebangCheck(sig []byte, raw scan.Bytes) bool {
return false
}
raw.Advance(2) // skip #! we checked above
raw.TrimLWS()
raw.TrimRWS()
return bytes.Equal(raw, sig)
return bytes.Equal(trimLWS(trimRWS(raw[2:])), sig)
}
// trimLWS trims whitespace from beginning of the input.
func trimLWS(in []byte) []byte {
firstNonWS := 0
for ; firstNonWS < len(in) && isWS(in[firstNonWS]); firstNonWS++ {
}
return in[firstNonWS:]
}
// trimRWS trims whitespace from the end of the input.
func trimRWS(in []byte) []byte {
lastNonWS := len(in) - 1
for ; lastNonWS > 0 && isWS(in[lastNonWS]); lastNonWS-- {
}
return in[:lastNonWS+1]
}
func firstLine(in []byte) []byte {
lineEnd := 0
for ; lineEnd < len(in) && in[lineEnd] != '\n'; lineEnd++ {
}
return in[:lineEnd]
}
func isWS(b byte) bool {
return b == '\t' || b == '\n' || b == '\x0c' || b == '\r' || b == ' '
}
func min(a, b int) int {
if a < b {
return a
}
return b
}

70
vendor/github.com/gabriel-vasile/mimetype/internal/magic/ms_office.go generated vendored
View File

@@ -5,36 +5,58 @@ import (
"encoding/binary"
)
var (
xlsxSigFiles = []string{
"xl/worksheets/",
"xl/drawings/",
"xl/theme/",
"xl/_rels/",
"xl/styles.xml",
"xl/workbook.xml",
"xl/sharedStrings.xml",
}
docxSigFiles = []string{
"word/media/",
"word/_rels/document.xml.rels",
"word/document.xml",
"word/styles.xml",
"word/fontTable.xml",
"word/settings.xml",
"word/numbering.xml",
"word/header",
"word/footer",
}
pptxSigFiles = []string{
"ppt/slides/",
"ppt/media/",
"ppt/slideLayouts/",
"ppt/theme/",
"ppt/slideMasters/",
"ppt/tags/",
"ppt/notesMasters/",
"ppt/_rels/",
"ppt/handoutMasters/",
"ppt/notesSlides/",
"ppt/presentation.xml",
"ppt/tableStyles.xml",
"ppt/presProps.xml",
"ppt/viewProps.xml",
}
)
// Xlsx matches a Microsoft Excel 2007 file.
func Xlsx(raw []byte, limit uint32) bool {
return msoxml(raw, zipEntries{{
name: []byte("xl/"),
dir: true,
}}, 100)
return zipContains(raw, xlsxSigFiles...)
}
// Docx matches a Microsoft Word 2007 file.
func Docx(raw []byte, limit uint32) bool {
return msoxml(raw, zipEntries{{
name: []byte("word/"),
dir: true,
}}, 100)
return zipContains(raw, docxSigFiles...)
}
// Pptx matches a Microsoft PowerPoint 2007 file.
func Pptx(raw []byte, limit uint32) bool {
return msoxml(raw, zipEntries{{
name: []byte("ppt/"),
dir: true,
}}, 100)
}
// Visio matches a Microsoft Visio 2013+ file.
func Visio(raw []byte, limit uint32) bool {
return msoxml(raw, zipEntries{{
name: []byte("visio/"),
dir: true,
}}, 100)
return zipContains(raw, pptxSigFiles...)
}
// Ole matches an Open Linking and Embedding file.
@@ -174,14 +196,6 @@ func Msi(raw []byte, limit uint32) bool {
})
}
// One matches a Microsoft OneNote file.
func One(raw []byte, limit uint32) bool {
return bytes.HasPrefix(raw, []byte{
0xe4, 0x52, 0x5c, 0x7b, 0x8c, 0xd8, 0xa7, 0x4d,
0xae, 0xb1, 0x53, 0x78, 0xd0, 0x29, 0x96, 0xd3,
})
}
// Helper to match by a specific CLSID of a compound file.
//
// http://fileformats.archiveteam.org/wiki/Microsoft_Compound_File

316
vendor/github.com/gabriel-vasile/mimetype/internal/magic/text.go generated vendored
View File

@@ -1,13 +1,13 @@
package magic
import (
"bufio"
"bytes"
"strings"
"time"
"github.com/gabriel-vasile/mimetype/internal/charset"
"github.com/gabriel-vasile/mimetype/internal/json"
mkup "github.com/gabriel-vasile/mimetype/internal/markup"
"github.com/gabriel-vasile/mimetype/internal/scan"
)
var (
@@ -29,7 +29,6 @@ var (
[]byte("<BODY"),
[]byte("<BR"),
[]byte("<P"),
[]byte("<!--"),
)
// XML matches an Extensible Markup Language file.
XML = markup([]byte("<?XML"))
@@ -108,18 +107,6 @@ var (
[]byte("/usr/bin/python"),
[]byte("/usr/local/bin/python"),
[]byte("/usr/bin/env python"),
[]byte("/usr/bin/python2"),
[]byte("/usr/local/bin/python2"),
[]byte("/usr/bin/env python2"),
[]byte("/usr/bin/python3"),
[]byte("/usr/local/bin/python3"),
[]byte("/usr/bin/env python3"),
)
// Ruby matches a Ruby programming language file.
Ruby = shebang(
[]byte("/usr/bin/ruby"),
[]byte("/usr/local/bin/ruby"),
[]byte("/usr/bin/env ruby"),
)
// Tcl matches a Tcl programming language file.
Tcl = shebang(
@@ -134,43 +121,20 @@ var (
[]byte("/usr/bin/env wish"),
)
// Rtf matches a Rich Text Format file.
Rtf = prefix([]byte("{\\rtf"))
// Shell matches a shell script file.
Shell = shebang(
[]byte("/bin/sh"),
[]byte("/bin/bash"),
[]byte("/usr/local/bin/bash"),
[]byte("/usr/bin/env bash"),
[]byte("/bin/csh"),
[]byte("/usr/local/bin/csh"),
[]byte("/usr/bin/env csh"),
[]byte("/bin/dash"),
[]byte("/usr/local/bin/dash"),
[]byte("/usr/bin/env dash"),
[]byte("/bin/ksh"),
[]byte("/usr/local/bin/ksh"),
[]byte("/usr/bin/env ksh"),
[]byte("/bin/tcsh"),
[]byte("/usr/local/bin/tcsh"),
[]byte("/usr/bin/env tcsh"),
[]byte("/bin/zsh"),
[]byte("/usr/local/bin/zsh"),
[]byte("/usr/bin/env zsh"),
)
Rtf = prefix([]byte("{\\rtf1"))
)
// Text matches a plain text file.
//
// TODO: This function does not parse BOM-less UTF16 and UTF32 files. Not really
// sure it should. Linux file utility also requires a BOM for UTF16 and UTF32.
func Text(raw []byte, _ uint32) bool {
func Text(raw []byte, limit uint32) bool {
// First look for BOM.
if cset := charset.FromBOM(raw); cset != "" {
return true
}
// Binary data bytes as defined here: https://mimesniff.spec.whatwg.org/#binary-data-byte
for i := 0; i < min(len(raw), 4096); i++ {
b := raw[i]
for _, b := range raw {
if b <= 0x08 ||
b == 0x0B ||
0x0E <= b && b <= 0x1A ||
@@ -181,14 +145,6 @@ func Text(raw []byte, _ uint32) bool {
return true
}
// XHTML matches an XHTML file. This check depends on the XML check to have passed.
func XHTML(raw []byte, limit uint32) bool {
raw = raw[:min(len(raw), 4096)]
b := scan.Bytes(raw)
return b.Search([]byte("<!DOCTYPE HTML"), scan.CompactWS|scan.IgnoreCase) != -1 ||
b.Search([]byte("<HTML XMLNS="), scan.CompactWS|scan.IgnoreCase) != -1
}
// Php matches a PHP: Hypertext Preprocessor file.
func Php(raw []byte, limit uint32) bool {
if res := phpPageF(raw, limit); res {
@@ -199,180 +155,189 @@ func Php(raw []byte, limit uint32) bool {
// JSON matches a JavaScript Object Notation file.
func JSON(raw []byte, limit uint32) bool {
raw = trimLWS(raw)
// #175 A single JSON string, number or bool is not considered JSON.
// JSON objects and arrays are reported as JSON.
return jsonHelper(raw, limit, json.QueryNone, json.TokObject|json.TokArray)
if len(raw) < 2 || (raw[0] != '[' && raw[0] != '{') {
return false
}
parsed, err := json.Scan(raw)
// If the full file content was provided, check there is no error.
if limit == 0 || len(raw) < int(limit) {
return err == nil
}
// If a section of the file was provided, check if all of it was parsed.
return parsed == len(raw) && len(raw) > 0
}
// GeoJSON matches a RFC 7946 GeoJSON file.
//
// GeoJSON detection implies searching for key:value pairs like: `"type": "Feature"`
// in the input.
// BUG(gabriel-vasile): The "type" key should be searched for in the root object.
func GeoJSON(raw []byte, limit uint32) bool {
return jsonHelper(raw, limit, json.QueryGeo, json.TokObject)
}
// HAR matches a HAR Spec file.
// Spec: http://www.softwareishard.com/blog/har-12-spec/
func HAR(raw []byte, limit uint32) bool {
return jsonHelper(raw, limit, json.QueryHAR, json.TokObject)
}
// GLTF matches a GL Transmission Format (JSON) file.
// Visit [glTF specification] and [IANA glTF entry] for more details.
//
// [glTF specification]: https://registry.khronos.org/glTF/specs/2.0/glTF-2.0.html
// [IANA glTF entry]: https://www.iana.org/assignments/media-types/model/gltf+json
func GLTF(raw []byte, limit uint32) bool {
return jsonHelper(raw, limit, json.QueryGLTF, json.TokObject)
}
func jsonHelper(raw []byte, limit uint32, q string, wantTok int) bool {
if !json.LooksLikeObjectOrArray(raw) {
raw = trimLWS(raw)
if len(raw) == 0 {
return false
}
lraw := len(raw)
parsed, inspected, firstToken, querySatisfied := json.Parse(q, raw)
if !querySatisfied || firstToken&wantTok == 0 {
// GeoJSON is always a JSON object, not a JSON array or any other JSON value.
if raw[0] != '{' {
return false
}
// If the full file content was provided, check that the whole input was parsed.
if limit == 0 || lraw < int(limit) {
return parsed == lraw
s := []byte(`"type"`)
si, sl := bytes.Index(raw, s), len(s)
if si == -1 {
return false
}
// If a section of the file was provided, check if all of it was inspected.
// In other words, check that if there was a problem parsing, that problem
// occured at the last byte in the input.
return inspected == lraw && lraw > 0
// If the "type" string is the suffix of the input,
// there is no need to search for the value of the key.
if si+sl == len(raw) {
return false
}
// Skip the "type" part.
raw = raw[si+sl:]
// Skip any whitespace before the colon.
raw = trimLWS(raw)
// Check for colon.
if len(raw) == 0 || raw[0] != ':' {
return false
}
// Skip any whitespace after the colon.
raw = trimLWS(raw[1:])
geoJSONTypes := [][]byte{
[]byte(`"Feature"`),
[]byte(`"FeatureCollection"`),
[]byte(`"Point"`),
[]byte(`"LineString"`),
[]byte(`"Polygon"`),
[]byte(`"MultiPoint"`),
[]byte(`"MultiLineString"`),
[]byte(`"MultiPolygon"`),
[]byte(`"GeometryCollection"`),
}
for _, t := range geoJSONTypes {
if bytes.HasPrefix(raw, t) {
return true
}
}
return false
}
// NdJSON matches a Newline delimited JSON file. All complete lines from raw
// must be valid JSON documents meaning they contain one of the valid JSON data
// types.
func NdJSON(raw []byte, limit uint32) bool {
lCount, objOrArr := 0, 0
s := scan.Bytes(raw)
s.DropLastLine(limit)
var l scan.Bytes
for len(s) != 0 {
l = s.Line()
_, inspected, firstToken, _ := json.Parse(json.QueryNone, l)
if len(l) != inspected {
lCount, hasObjOrArr := 0, false
sc := bufio.NewScanner(dropLastLine(raw, limit))
for sc.Scan() {
l := sc.Bytes()
// Empty lines are allowed in NDJSON.
if l = trimRWS(trimLWS(l)); len(l) == 0 {
continue
}
_, err := json.Scan(l)
if err != nil {
return false
}
if firstToken == json.TokArray || firstToken == json.TokObject {
objOrArr++
if l[0] == '[' || l[0] == '{' {
hasObjOrArr = true
}
lCount++
}
return lCount > 1 && objOrArr > 0
return lCount > 1 && hasObjOrArr
}
// HAR matches a HAR Spec file.
// Spec: http://www.softwareishard.com/blog/har-12-spec/
func HAR(raw []byte, limit uint32) bool {
s := []byte(`"log"`)
si, sl := bytes.Index(raw, s), len(s)
if si == -1 {
return false
}
// If the "log" string is the suffix of the input,
// there is no need to search for the value of the key.
if si+sl == len(raw) {
return false
}
// Skip the "log" part.
raw = raw[si+sl:]
// Skip any whitespace before the colon.
raw = trimLWS(raw)
// Check for colon.
if len(raw) == 0 || raw[0] != ':' {
return false
}
// Skip any whitespace after the colon.
raw = trimLWS(raw[1:])
harJSONTypes := [][]byte{
[]byte(`"version"`),
[]byte(`"creator"`),
[]byte(`"entries"`),
}
for _, t := range harJSONTypes {
si := bytes.Index(raw, t)
if si > -1 {
return true
}
}
return false
}
// Svg matches a SVG file.
func Svg(raw []byte, limit uint32) bool {
return svgWithoutXMLDeclaration(raw) || svgWithXMLDeclaration(raw)
}
// svgWithoutXMLDeclaration matches a SVG image that does not have an XML header.
// Example:
//
// <!-- xml comment ignored -->
// <svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
// <rect fill="#fff" stroke="#000" x="-70" y="-70" width="390" height="390"/>
// </svg>
func svgWithoutXMLDeclaration(s scan.Bytes) bool {
for scan.ByteIsWS(s.Peek()) {
s.Advance(1)
}
for mkup.SkipAComment(&s) {
}
if !bytes.HasPrefix(s, []byte("<svg")) {
return false
}
targetName, targetVal := "xmlns", "http://www.w3.org/2000/svg"
aName, aVal, hasMore := "", "", true
for hasMore {
aName, aVal, hasMore = mkup.GetAnAttribute(&s)
if aName == targetName && aVal == targetVal {
return true
}
if !hasMore {
return false
}
}
return false
}
// svgWithXMLDeclaration matches a SVG image that has an XML header.
// Example:
//
// <?xml version="1.0" encoding="UTF-8" standalone="no"?>
// <svg width="391" height="391" viewBox="-70.5 -70.5 391 391" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
// <rect fill="#fff" stroke="#000" x="-70" y="-70" width="390" height="390"/>
// </svg>
func svgWithXMLDeclaration(s scan.Bytes) bool {
for scan.ByteIsWS(s.Peek()) {
s.Advance(1)
}
if !bytes.HasPrefix(s, []byte("<?xml")) {
return false
}
// version is a required attribute for XML.
hasVersion := false
aName, hasMore := "", true
for hasMore {
aName, _, hasMore = mkup.GetAnAttribute(&s)
if aName == "version" {
hasVersion = true
break
}
if !hasMore {
break
}
}
if len(s) > 4096 {
s = s[:4096]
}
return hasVersion && bytes.Contains(s, []byte("<svg"))
return bytes.Contains(raw, []byte("<svg"))
}
// Srt matches a SubRip file.
func Srt(raw []byte, _ uint32) bool {
s := scan.Bytes(raw)
line := s.Line()
// First line must be 1.
if len(line) != 1 || line[0] != '1' {
func Srt(in []byte, _ uint32) bool {
s := bufio.NewScanner(bytes.NewReader(in))
if !s.Scan() {
return false
}
line = s.Line()
// Timestamp format (e.g: 00:02:16,612 --> 00:02:19,376) limits second line
// First line must be 1.
if s.Text() != "1" {
return false
}
if !s.Scan() {
return false
}
secondLine := s.Text()
// Timestamp format (e.g: 00:02:16,612 --> 00:02:19,376) limits secondLine
// length to exactly 29 characters.
if len(line) != 29 {
if len(secondLine) != 29 {
return false
}
// Decimal separator of fractional seconds in the timestamps must be a
// comma, not a period.
if bytes.IndexByte(line, '.') != -1 {
if strings.Contains(secondLine, ".") {
return false
}
sep := []byte(" --> ")
i := bytes.Index(line, sep)
if i == -1 {
// For Go <1.17, comma is not recognised as a decimal separator by `time.Parse`.
secondLine = strings.ReplaceAll(secondLine, ",", ".")
// Second line must be a time range.
ts := strings.Split(secondLine, " --> ")
if len(ts) != 2 {
return false
}
const layout = "15:04:05,000"
t0, err := time.Parse(layout, string(line[:i]))
const layout = "15:04:05.000"
t0, err := time.Parse(layout, ts[0])
if err != nil {
return false
}
t1, err := time.Parse(layout, string(line[i+len(sep):]))
t1, err := time.Parse(layout, ts[1])
if err != nil {
return false
}
@@ -380,9 +345,8 @@ func Srt(raw []byte, _ uint32) bool {
return false
}
line = s.Line()
// A third line must exist and not be empty. This is the actual subtitle text.
return len(line) != 0
return s.Scan() && len(s.Bytes()) != 0
}
// Vtt matches a Web Video Text Tracks (WebVTT) file. See

66
vendor/github.com/gabriel-vasile/mimetype/internal/magic/text_csv.go generated vendored
View File

@@ -1,43 +1,63 @@
package magic
import (
"github.com/gabriel-vasile/mimetype/internal/csv"
"github.com/gabriel-vasile/mimetype/internal/scan"
"bytes"
"encoding/csv"
"errors"
"io"
)
// CSV matches a comma-separated values file.
func CSV(raw []byte, limit uint32) bool {
// Csv matches a comma-separated values file.
func Csv(raw []byte, limit uint32) bool {
return sv(raw, ',', limit)
}
// TSV matches a tab-separated values file.
func TSV(raw []byte, limit uint32) bool {
// Tsv matches a tab-separated values file.
func Tsv(raw []byte, limit uint32) bool {
return sv(raw, '\t', limit)
}
func sv(in []byte, comma byte, limit uint32) bool {
s := scan.Bytes(in)
s.DropLastLine(limit)
r := csv.NewParser(comma, '#', s)
func sv(in []byte, comma rune, limit uint32) bool {
r := csv.NewReader(dropLastLine(in, limit))
r.Comma = comma
r.ReuseRecord = true
r.LazyQuotes = true
r.Comment = '#'
headerFields, _, hasMore := r.CountFields(false)
if headerFields < 2 || !hasMore {
return false
}
csvLines := 1 // 1 for header
lines := 0
for {
fields, _, hasMore := r.CountFields(false)
if !hasMore && fields == 0 {
_, err := r.Read()
if errors.Is(err, io.EOF) {
break
}
csvLines++
if fields != headerFields {
if err != nil {
return false
}
if csvLines >= 10 {
return true
}
lines++
}
return csvLines >= 2
return r.FieldsPerRecord > 1 && lines > 1
}
// dropLastLine drops the last incomplete line from b.
//
// mimetype limits itself to ReadLimit bytes when performing a detection.
// This means, for file formats like CSV for NDJSON, the last line of the input
// can be an incomplete line.
func dropLastLine(b []byte, cutAt uint32) io.Reader {
if cutAt == 0 {
return bytes.NewReader(b)
}
if uint32(len(b)) >= cutAt {
for i := cutAt - 1; i > 0; i-- {
if b[i] == '\n' {
return bytes.NewReader(b[:i])
}
}
// No newline was found between the 0 index and cutAt.
return bytes.NewReader(b[:cutAt])
}
return bytes.NewReader(b)
}

167
vendor/github.com/gabriel-vasile/mimetype/internal/magic/zip.go generated vendored
View File

@@ -2,8 +2,8 @@ package magic
import (
"bytes"
"github.com/gabriel-vasile/mimetype/internal/scan"
"encoding/binary"
"strings"
)
var (
@@ -41,149 +41,52 @@ func Zip(raw []byte, limit uint32) bool {
(raw[3] == 0x4 || raw[3] == 0x6 || raw[3] == 0x8)
}
// Jar matches a Java archive file. There are two types of Jar files:
// 1. the ones that can be opened with jexec and have 0xCAFE optional flag
// https://stackoverflow.com/tags/executable-jar/info
// 2. regular jars, same as above, just without the executable flag
// https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=262278#c0
// There is an argument to only check for manifest, since it's the common nominator
// for both executable and non-executable versions. But the traversing zip entries
// is unreliable because it does linear search for signatures
// (instead of relying on offsets told by the file.)
// Jar matches a Java archive file.
func Jar(raw []byte, limit uint32) bool {
return executableJar(raw) ||
zipHas(raw, zipEntries{{
name: []byte("META-INF/MANIFEST.MF"),
}, {
name: []byte("META-INF/"),
}}, 1)
return zipContains(raw, "META-INF/MANIFEST.MF")
}
// KMZ matches a zipped KML file, which is "doc.kml" by convention.
func KMZ(raw []byte, _ uint32) bool {
return zipHas(raw, zipEntries{{
name: []byte("doc.kml"),
}}, 100)
// zipTokenizer holds the source zip file and scanned index.
type zipTokenizer struct {
in []byte
i int // current index
}
// An executable Jar has a 0xCAFE flag enabled in the first zip entry.
// The rule from file/file is:
// >(26.s+30) leshort 0xcafe Java archive data (JAR)
func executableJar(b scan.Bytes) bool {
b.Advance(0x1A)
offset, ok := b.Uint16()
if !ok {
return false
// next returns the next file name from the zip headers.
// https://web.archive.org/web/20191129114319/https://users.cs.jmu.edu/buchhofp/forensics/formats/pkzip.html
func (t *zipTokenizer) next() (fileName string) {
if t.i > len(t.in) {
return
}
b.Advance(int(offset) + 2)
cafe, ok := b.Uint16()
return ok && cafe == 0xCAFE
}
// zipIterator iterates over a zip file returning the name of the zip entries
// in that file.
type zipIterator struct {
b scan.Bytes
}
type zipEntries []struct {
name []byte
dir bool // dir means checking just the prefix of the entry, not the whole path
}
func (z zipEntries) match(file []byte) bool {
for i := range z {
if z[i].dir && bytes.HasPrefix(file, z[i].name) {
return true
}
if bytes.Equal(file, z[i].name) {
return true
}
}
return false
}
func zipHas(raw scan.Bytes, searchFor zipEntries, stopAfter int) bool {
iter := zipIterator{raw}
for i := 0; i < stopAfter; i++ {
f := iter.next()
if len(f) == 0 {
break
}
if searchFor.match(f) {
return true
}
in := t.in[t.i:]
// pkSig is the signature of the zip local file header.
pkSig := []byte("PK\003\004")
pkIndex := bytes.Index(in, pkSig)
// 30 is the offset of the file name in the header.
fNameOffset := pkIndex + 30
// end if signature not found or file name offset outside of file.
if pkIndex == -1 || fNameOffset > len(in) {
return
}
return false
fNameLen := int(binary.LittleEndian.Uint16(in[pkIndex+26 : pkIndex+28]))
if fNameLen <= 0 || fNameOffset+fNameLen > len(in) {
return
}
t.i += fNameOffset + fNameLen
return string(in[fNameOffset : fNameOffset+fNameLen])
}
// msoxml behaves like zipHas, but it puts restrictions on what the first zip
// entry can be.
func msoxml(raw scan.Bytes, searchFor zipEntries, stopAfter int) bool {
iter := zipIterator{raw}
for i := 0; i < stopAfter; i++ {
f := iter.next()
if len(f) == 0 {
break
}
if searchFor.match(f) {
return true
}
// If the first is not one of the next usually expected entries,
// then abort this check.
if i == 0 {
if !bytes.Equal(f, []byte("[Content_Types].xml")) &&
!bytes.Equal(f, []byte("_rels/.rels")) &&
!bytes.Equal(f, []byte("docProps")) &&
!bytes.Equal(f, []byte("customXml")) &&
!bytes.Equal(f, []byte("[trash]")) {
return false
// zipContains returns true if the zip file headers from in contain any of the paths.
func zipContains(in []byte, paths ...string) bool {
t := zipTokenizer{in: in}
for i, tok := 0, t.next(); tok != ""; i, tok = i+1, t.next() {
for p := range paths {
if strings.HasPrefix(tok, paths[p]) {
return true
}
}
}
return false
}
// next extracts the name of the next zip entry.
func (i *zipIterator) next() []byte {
pk := []byte("PK\003\004")
n := bytes.Index(i.b, pk)
if n == -1 {
return nil
}
i.b.Advance(n)
if !i.b.Advance(0x1A) {
return nil
}
l, ok := i.b.Uint16()
if !ok {
return nil
}
if !i.b.Advance(0x02) {
return nil
}
if len(i.b) < int(l) {
return nil
}
return i.b[:l]
}
// APK matches an Android Package Archive.
// The source of signatures is https://github.com/file/file/blob/1778642b8ba3d947a779a36fcd81f8e807220a19/magic/Magdir/archive#L1820-L1887
func APK(raw []byte, _ uint32) bool {
return zipHas(raw, zipEntries{{
name: []byte("AndroidManifest.xml"),
}, {
name: []byte("META-INF/com/android/build/gradle/app-metadata.properties"),
}, {
name: []byte("classes.dex"),
}, {
name: []byte("resources.arsc"),
}, {
name: []byte("res/drawable"),
}}, 100)
}

26
vendor/github.com/gabriel-vasile/mimetype/mime.go generated vendored
View File

@@ -103,17 +103,15 @@ func (m *MIME) match(in []byte, readLimit uint32) *MIME {
"text/html": charset.FromHTML,
"text/xml": charset.FromXML,
}
charset := ""
// ps holds optional MIME parameters.
ps := map[string]string{}
if f, ok := needsCharset[m.mime]; ok {
// The charset comes from BOM, from HTML headers, from XML headers.
// Limit the number of bytes searched for to 1024.
charset = f(in[:min(len(in), 1024)])
}
if m == root {
return m
if cset := f(in); cset != "" {
ps["charset"] = cset
}
}
return m.cloneHierarchy(charset)
return m.cloneHierarchy(ps)
}
// flatten transforms an hierarchy of MIMEs into a slice of MIMEs.
@@ -127,10 +125,10 @@ func (m *MIME) flatten() []*MIME {
}
// clone creates a new MIME with the provided optional MIME parameters.
func (m *MIME) clone(charset string) *MIME {
func (m *MIME) clone(ps map[string]string) *MIME {
clonedMIME := m.mime
if charset != "" {
clonedMIME = m.mime + "; charset=" + charset
if len(ps) > 0 {
clonedMIME = mime.FormatMediaType(m.mime, ps)
}
return &MIME{
@@ -142,11 +140,11 @@ func (m *MIME) clone(charset string) *MIME {
// cloneHierarchy creates a clone of m and all its ancestors. The optional MIME
// parameters are set on the last child of the hierarchy.
func (m *MIME) cloneHierarchy(charset string) *MIME {
ret := m.clone(charset)
func (m *MIME) cloneHierarchy(ps map[string]string) *MIME {
ret := m.clone(ps)
lastChild := ret
for p := m.Parent(); p != nil; p = p.Parent() {
pClone := p.clone("")
pClone := p.clone(nil)
lastChild.parent = pClone
lastChild = pClone
}

8
vendor/github.com/gabriel-vasile/mimetype/mimetype.go generated vendored
View File

@@ -7,15 +7,14 @@ package mimetype
import (
"io"
"io/ioutil"
"mime"
"os"
"sync/atomic"
)
var defaultLimit uint32 = 3072
// readLimit is the maximum number of bytes from the input used when detecting.
var readLimit uint32 = defaultLimit
var readLimit uint32 = 3072
// Detect returns the MIME type found from the provided byte slice.
//
@@ -49,7 +48,7 @@ func DetectReader(r io.Reader) (*MIME, error) {
// Using atomic because readLimit can be written at the same time in other goroutine.
l := atomic.LoadUint32(&readLimit)
if l == 0 {
in, err = io.ReadAll(r)
in, err = ioutil.ReadAll(r)
if err != nil {
return errMIME, err
}
@@ -104,7 +103,6 @@ func EqualsAny(s string, mimes ...string) bool {
// SetLimit sets the maximum number of bytes read from input when detecting the MIME type.
// Increasing the limit provides better detection for file formats which store
// their magical numbers towards the end of the file: docx, pptx, xlsx, etc.
// During detection data is read in a single block of size limit, i.e. it is not buffered.
// A limit of 0 means the whole input file will be used.
func SetLimit(limit uint32) {
// Using atomic because readLimit can be read at the same time in other goroutine.

54
vendor/github.com/gabriel-vasile/mimetype/supported_mimes.md generated vendored
View File

@@ -1,4 +1,4 @@
## 191 Supported MIME types
## 173 Supported MIME types
This file is automatically generated when running tests. Do not edit manually.
Extension | MIME type | Aliases
@@ -7,12 +7,11 @@ Extension | MIME type | Aliases
**.xpm** | image/x-xpixmap | -
**.7z** | application/x-7z-compressed | -
**.zip** | application/zip | application/x-zip, application/x-zip-compressed
**.xlsx** | application/vnd.openxmlformats-officedocument.spreadsheetml.sheet | -
**.docx** | application/vnd.openxmlformats-officedocument.wordprocessingml.document | -
**.pptx** | application/vnd.openxmlformats-officedocument.presentationml.presentation | -
**.xlsx** | application/vnd.openxmlformats-officedocument.spreadsheetml.sheet | -
**.epub** | application/epub+zip | -
**.apk** | application/vnd.android.package-archive | -
**.jar** | application/java-archive | application/jar, application/jar-archive, application/x-java-archive
**.jar** | application/jar | -
**.odt** | application/vnd.oasis.opendocument.text | application/x-vnd.oasis.opendocument.text
**.ott** | application/vnd.oasis.opendocument.text-template | application/x-vnd.oasis.opendocument.text-template
**.ods** | application/vnd.oasis.opendocument.spreadsheet | application/x-vnd.oasis.opendocument.spreadsheet
@@ -24,8 +23,6 @@ Extension | MIME type | Aliases
**.odf** | application/vnd.oasis.opendocument.formula | application/x-vnd.oasis.opendocument.formula
**.odc** | application/vnd.oasis.opendocument.chart | application/x-vnd.oasis.opendocument.chart
**.sxc** | application/vnd.sun.xml.calc | -
**.kmz** | application/vnd.google-earth.kmz | -
**.vsdx** | application/vnd.ms-visio.drawing.main+xml | -
**.pdf** | application/pdf | application/x-pdf
**.fdf** | application/vnd.fdf | -
**n/a** | application/x-ole-storage | -
@@ -63,10 +60,9 @@ Extension | MIME type | Aliases
**.tar** | application/x-tar | -
**.xar** | application/x-xar | -
**.bz2** | application/x-bzip2 | -
**.fits** | application/fits | image/fits
**.fits** | application/fits | -
**.tiff** | image/tiff | -
**.bmp** | image/bmp | image/x-bmp, image/x-ms-bmp
**.123** | application/vnd.lotus-1-2-3 | -
**.ico** | image/x-icon | -
**.mp3** | audio/mpeg | audio/x-mpeg, audio/mp3
**.flac** | audio/flac | -
@@ -79,28 +75,21 @@ Extension | MIME type | Aliases
**.au** | audio/basic | -
**.mpeg** | video/mpeg | -
**.mov** | video/quicktime | -
**.mqv** | video/quicktime | -
**.mp4** | video/mp4 | -
**.avif** | image/avif | -
**.webm** | video/webm | audio/webm
**.3gp** | video/3gpp | video/3gp, audio/3gpp
**.3g2** | video/3gpp2 | video/3g2, audio/3gpp2
**.mp4** | audio/mp4 | audio/x-mp4a
**.mqv** | video/quicktime | -
**.m4a** | audio/x-m4a | -
**.m4v** | video/x-m4v | -
**.heic** | image/heic | -
**.heic** | image/heic-sequence | -
**.heif** | image/heif | -
**.heif** | image/heif-sequence | -
**.mj2** | video/mj2 | -
**.dvb** | video/vnd.dvb.file | -
**.webm** | video/webm | audio/webm
**.avi** | video/x-msvideo | video/avi, video/msvideo
**.flv** | video/x-flv | -
**.mkv** | video/x-matroska | -
**.asf** | video/x-ms-asf | video/asf, video/x-ms-wmv
**.aac** | audio/aac | -
**.voc** | audio/x-unknown | -
**.mp4** | audio/mp4 | audio/x-m4a, audio/x-mp4a
**.m4a** | audio/x-m4a | -
**.m3u** | application/vnd.apple.mpegurl | audio/mpegurl
**.m4v** | video/x-m4v | -
**.rmvb** | application/vnd.rn-realmedia-vbr | -
**.gz** | application/gzip | application/x-gzip, application/x-gunzip, application/gzipped, application/gzip-compressed, application/x-gzip-compressed, gzip/document
**.class** | application/x-java-applet | -
@@ -122,7 +111,6 @@ Extension | MIME type | Aliases
**.mobi** | application/x-mobipocket-ebook | -
**.lit** | application/x-ms-reader | -
**.bpg** | image/bpg | -
**.cbor** | application/cbor | -
**.sqlite** | application/vnd.sqlite3 | application/x-sqlite3
**.dwg** | image/vnd.dwg | image/x-dwg, application/acad, application/x-acad, application/autocad_dwg, application/dwg, application/x-dwg, application/x-autocad, drawing/dwg
**.nes** | application/vnd.nintendo.snes.rom | -
@@ -130,6 +118,10 @@ Extension | MIME type | Aliases
**.macho** | application/x-mach-binary | -
**.qcp** | audio/qcelp | -
**.icns** | image/x-icns | -
**.heic** | image/heic | -
**.heic** | image/heic-sequence | -
**.heif** | image/heif | -
**.heif** | image/heif-sequence | -
**.hdr** | image/vnd.radiance | -
**.mrc** | application/marc | -
**.mdb** | application/x-msaccess | -
@@ -146,15 +138,13 @@ Extension | MIME type | Aliases
**.pat** | image/x-gimp-pat | -
**.gbr** | image/x-gimp-gbr | -
**.glb** | model/gltf-binary | -
**.avif** | image/avif | -
**.cab** | application/x-installshield | -
**.jxr** | image/jxr | image/vnd.ms-photo
**.parquet** | application/vnd.apache.parquet | application/x-parquet
**.one** | application/onenote | -
**.chm** | application/vnd.ms-htmlhelp | -
**.txt** | text/plain | -
**.svg** | image/svg+xml | -
**.html** | text/html | -
**.xml** | text/xml | application/xml
**.svg** | image/svg+xml | -
**.xml** | text/xml | -
**.rss** | application/rss+xml | text/rss
**.atom** | application/atom+xml | -
**.x3d** | model/x3d+xml | -
@@ -168,19 +158,16 @@ Extension | MIME type | Aliases
**.3mf** | application/vnd.ms-package.3dmanufacturing-3dmodel+xml | -
**.xfdf** | application/vnd.adobe.xfdf | -
**.owl** | application/owl+xml | -
**.html** | application/xhtml+xml | -
**.php** | text/x-php | -
**.js** | text/javascript | application/x-javascript, application/javascript
**.js** | application/javascript | application/x-javascript, text/javascript
**.lua** | text/x-lua | -
**.pl** | text/x-perl | -
**.py** | text/x-python | text/x-script.python, application/x-python
**.rb** | text/x-ruby | application/x-ruby
**.json** | application/json | -
**.geojson** | application/geo+json | -
**.har** | application/json | -
**.gltf** | model/gltf+json | -
**.ndjson** | application/x-ndjson | -
**.rtf** | text/rtf | application/rtf
**.rtf** | text/rtf | -
**.srt** | application/x-subrip | application/x-srt, text/x-srt
**.tcl** | text/x-tcl | application/x-tcl
**.csv** | text/csv | -
@@ -189,8 +176,3 @@ Extension | MIME type | Aliases
**.ics** | text/calendar | -
**.warc** | application/warc | -
**.vtt** | text/vtt | -
**.sh** | text/x-shellscript | text/x-sh, application/x-shellscript, application/x-sh
**.pbm** | image/x-portable-bitmap | -
**.pgm** | image/x-portable-graymap | -
**.ppm** | image/x-portable-pixmap | -
**.pam** | image/x-portable-arbitrarymap | -

105
vendor/github.com/gabriel-vasile/mimetype/tree.go generated vendored
View File

@@ -18,14 +18,14 @@ import (
var root = newMIME("application/octet-stream", "",
func([]byte, uint32) bool { return true },
xpm, sevenZ, zip, pdf, fdf, ole, ps, psd, p7s, ogg, png, jpg, jxl, jp2, jpx,
jpm, jxs, gif, webp, exe, elf, ar, tar, xar, bz2, fits, tiff, bmp, lotus, ico,
mp3, flac, midi, ape, musePack, amr, wav, aiff, au, mpeg, quickTime, mp4, webM,
avi, flv, mkv, asf, aac, voc, m3u, rmvb, gzip, class, swf, crx, ttf, woff,
woff2, otf, ttc, eot, wasm, shx, dbf, dcm, rar, djvu, mobi, lit, bpg, cbor,
sqlite3, dwg, nes, lnk, macho, qcp, icns, hdr, mrc, mdb, accdb, zstd, cab,
rpm, xz, lzip, torrent, cpio, tzif, xcf, pat, gbr, glb, cabIS, jxr, parquet,
oneNote, chm,
// Keep text last because it is the slowest check.
jpm, jxs, gif, webp, exe, elf, ar, tar, xar, bz2, fits, tiff, bmp, ico, mp3, flac,
midi, ape, musePack, amr, wav, aiff, au, mpeg, quickTime, mqv, mp4, webM,
threeGP, threeG2, avi, flv, mkv, asf, aac, voc, aMp4, m4a, m3u, m4v, rmvb,
gzip, class, swf, crx, ttf, woff, woff2, otf, ttc, eot, wasm, shx, dbf, dcm, rar,
djvu, mobi, lit, bpg, sqlite3, dwg, nes, lnk, macho, qcp, icns, heic,
heicSeq, heif, heifSeq, hdr, mrc, mdb, accdb, zstd, cab, rpm, xz, lzip,
torrent, cpio, tzif, xcf, pat, gbr, glb, avif, cabIS, jxr,
// Keep text last because it is the slowest check
text,
)
@@ -45,28 +45,21 @@ var (
"application/gzip-compressed", "application/x-gzip-compressed",
"gzip/document")
sevenZ = newMIME("application/x-7z-compressed", ".7z", magic.SevenZ)
// APK must be checked before JAR because APK is a subset of JAR.
// This means APK should be a child of JAR detector, but in practice,
// the decisive signature for JAR might be located at the end of the file
// and not reachable because of library readLimit.
zip = newMIME("application/zip", ".zip", magic.Zip, docx, pptx, xlsx, epub, apk, jar, odt, ods, odp, odg, odf, odc, sxc, kmz, visio).
zip = newMIME("application/zip", ".zip", magic.Zip, xlsx, docx, pptx, epub, jar, odt, ods, odp, odg, odf, odc, sxc).
alias("application/x-zip", "application/x-zip-compressed")
tar = newMIME("application/x-tar", ".tar", magic.Tar)
xar = newMIME("application/x-xar", ".xar", magic.Xar)
bz2 = newMIME("application/x-bzip2", ".bz2", magic.Bz2)
pdf = newMIME("application/pdf", ".pdf", magic.PDF).
pdf = newMIME("application/pdf", ".pdf", magic.Pdf).
alias("application/x-pdf")
fdf = newMIME("application/vnd.fdf", ".fdf", magic.Fdf)
xlsx = newMIME("application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", ".xlsx", magic.Xlsx)
docx = newMIME("application/vnd.openxmlformats-officedocument.wordprocessingml.document", ".docx", magic.Docx)
pptx = newMIME("application/vnd.openxmlformats-officedocument.presentationml.presentation", ".pptx", magic.Pptx)
visio = newMIME("application/vnd.ms-visio.drawing.main+xml", ".vsdx", magic.Visio)
epub = newMIME("application/epub+zip", ".epub", magic.Epub)
jar = newMIME("application/java-archive", ".jar", magic.Jar).
alias("application/jar", "application/jar-archive", "application/x-java-archive")
apk = newMIME("application/vnd.android.package-archive", ".apk", magic.APK)
ole = newMIME("application/x-ole-storage", "", magic.Ole, msi, aaf, msg, xls, pub, ppt, doc)
msi = newMIME("application/x-ms-installer", ".msi", magic.Msi).
fdf = newMIME("application/vnd.fdf", ".fdf", magic.Fdf)
xlsx = newMIME("application/vnd.openxmlformats-officedocument.spreadsheetml.sheet", ".xlsx", magic.Xlsx)
docx = newMIME("application/vnd.openxmlformats-officedocument.wordprocessingml.document", ".docx", magic.Docx)
pptx = newMIME("application/vnd.openxmlformats-officedocument.presentationml.presentation", ".pptx", magic.Pptx)
epub = newMIME("application/epub+zip", ".epub", magic.Epub)
jar = newMIME("application/jar", ".jar", magic.Jar)
ole = newMIME("application/x-ole-storage", "", magic.Ole, msi, aaf, msg, xls, pub, ppt, doc)
msi = newMIME("application/x-ms-installer", ".msi", magic.Msi).
alias("application/x-windows-installer", "application/x-msi")
aaf = newMIME("application/octet-stream", ".aaf", magic.Aaf)
doc = newMIME("application/msword", ".doc", magic.Doc).
@@ -78,26 +71,24 @@ var (
alias("application/msexcel")
msg = newMIME("application/vnd.ms-outlook", ".msg", magic.Msg)
ps = newMIME("application/postscript", ".ps", magic.Ps)
fits = newMIME("application/fits", ".fits", magic.Fits).alias("image/fits")
fits = newMIME("application/fits", ".fits", magic.Fits)
ogg = newMIME("application/ogg", ".ogg", magic.Ogg, oggAudio, oggVideo).
alias("application/x-ogg")
oggAudio = newMIME("audio/ogg", ".oga", magic.OggAudio)
oggVideo = newMIME("video/ogg", ".ogv", magic.OggVideo)
text = newMIME("text/plain", ".txt", magic.Text, svg, html, xml, php, js, lua, perl, python, ruby, json, ndJSON, rtf, srt, tcl, csv, tsv, vCard, iCalendar, warc, vtt, shell, netpbm, netpgm, netppm, netpam)
xml = newMIME("text/xml", ".xml", magic.XML, rss, atom, x3d, kml, xliff, collada, gml, gpx, tcx, amf, threemf, xfdf, owl2, xhtml).
alias("application/xml")
xhtml = newMIME("application/xhtml+xml", ".html", magic.XHTML)
json = newMIME("application/json", ".json", magic.JSON, geoJSON, har, gltf)
har = newMIME("application/json", ".har", magic.HAR)
csv = newMIME("text/csv", ".csv", magic.CSV)
tsv = newMIME("text/tab-separated-values", ".tsv", magic.TSV)
geoJSON = newMIME("application/geo+json", ".geojson", magic.GeoJSON)
ndJSON = newMIME("application/x-ndjson", ".ndjson", magic.NdJSON)
html = newMIME("text/html", ".html", magic.HTML)
php = newMIME("text/x-php", ".php", magic.Php)
rtf = newMIME("text/rtf", ".rtf", magic.Rtf).alias("application/rtf")
js = newMIME("text/javascript", ".js", magic.Js).
alias("application/x-javascript", "application/javascript")
text = newMIME("text/plain", ".txt", magic.Text, html, svg, xml, php, js, lua, perl, python, json, ndJSON, rtf, srt, tcl, csv, tsv, vCard, iCalendar, warc, vtt)
xml = newMIME("text/xml", ".xml", magic.XML, rss, atom, x3d, kml, xliff, collada, gml, gpx, tcx, amf, threemf, xfdf, owl2)
json = newMIME("application/json", ".json", magic.JSON, geoJSON, har)
har = newMIME("application/json", ".har", magic.HAR)
csv = newMIME("text/csv", ".csv", magic.Csv)
tsv = newMIME("text/tab-separated-values", ".tsv", magic.Tsv)
geoJSON = newMIME("application/geo+json", ".geojson", magic.GeoJSON)
ndJSON = newMIME("application/x-ndjson", ".ndjson", magic.NdJSON)
html = newMIME("text/html", ".html", magic.HTML)
php = newMIME("text/x-php", ".php", magic.Php)
rtf = newMIME("text/rtf", ".rtf", magic.Rtf)
js = newMIME("application/javascript", ".js", magic.Js).
alias("application/x-javascript", "text/javascript")
srt = newMIME("application/x-subrip", ".srt", magic.Srt).
alias("application/x-srt", "text/x-srt")
vtt = newMIME("text/vtt", ".vtt", magic.Vtt)
@@ -105,10 +96,6 @@ var (
perl = newMIME("text/x-perl", ".pl", magic.Perl)
python = newMIME("text/x-python", ".py", magic.Python).
alias("text/x-script.python", "application/x-python")
ruby = newMIME("text/x-ruby", ".rb", magic.Ruby).
alias("application/x-ruby")
shell = newMIME("text/x-shellscript", ".sh", magic.Shell).
alias("text/x-sh", "application/x-shellscript", "application/x-sh")
tcl = newMIME("text/x-tcl", ".tcl", magic.Tcl).
alias("application/x-tcl")
vCard = newMIME("text/vcard", ".vcf", magic.VCard)
@@ -120,7 +107,6 @@ var (
atom = newMIME("application/atom+xml", ".atom", magic.Atom)
x3d = newMIME("model/x3d+xml", ".x3d", magic.X3d)
kml = newMIME("application/vnd.google-earth.kml+xml", ".kml", magic.Kml)
kmz = newMIME("application/vnd.google-earth.kmz", ".kmz", magic.KMZ)
xliff = newMIME("application/x-xliff+xml", ".xlf", magic.Xliff)
collada = newMIME("model/vnd.collada+xml", ".dae", magic.Collada)
gml = newMIME("application/gml+xml", ".gml", magic.Gml)
@@ -144,12 +130,9 @@ var (
tiff = newMIME("image/tiff", ".tiff", magic.Tiff)
bmp = newMIME("image/bmp", ".bmp", magic.Bmp).
alias("image/x-bmp", "image/x-ms-bmp")
// lotus check must be done before ico because some ico detection is a bit
// relaxed and some lotus files are wrongfully identified as ico otherwise.
lotus = newMIME("application/vnd.lotus-1-2-3", ".123", magic.Lotus123)
ico = newMIME("image/x-icon", ".ico", magic.Ico)
icns = newMIME("image/x-icns", ".icns", magic.Icns)
psd = newMIME("image/vnd.adobe.photoshop", ".psd", magic.Psd).
ico = newMIME("image/x-icon", ".ico", magic.Ico)
icns = newMIME("image/x-icns", ".icns", magic.Icns)
psd = newMIME("image/vnd.adobe.photoshop", ".psd", magic.Psd).
alias("image/x-psd", "application/photoshop")
heic = newMIME("image/heic", ".heic", magic.Heic)
heicSeq = newMIME("image/heic-sequence", ".heic", magic.HeicSequence)
@@ -173,14 +156,12 @@ var (
aac = newMIME("audio/aac", ".aac", magic.AAC)
voc = newMIME("audio/x-unknown", ".voc", magic.Voc)
aMp4 = newMIME("audio/mp4", ".mp4", magic.AMp4).
alias("audio/x-mp4a")
alias("audio/x-m4a", "audio/x-mp4a")
m4a = newMIME("audio/x-m4a", ".m4a", magic.M4a)
m3u = newMIME("application/vnd.apple.mpegurl", ".m3u", magic.M3u).
alias("audio/mpegurl")
m4v = newMIME("video/x-m4v", ".m4v", magic.M4v)
mj2 = newMIME("video/mj2", ".mj2", magic.Mj2)
dvb = newMIME("video/vnd.dvb.file", ".dvb", magic.Dvb)
mp4 = newMIME("video/mp4", ".mp4", magic.Mp4, avif, threeGP, threeG2, aMp4, mqv, m4a, m4v, heic, heicSeq, heif, heifSeq, mj2, dvb)
mp4 = newMIME("video/mp4", ".mp4", magic.Mp4)
webM = newMIME("video/webm", ".webm", magic.WebM).
alias("audio/webm")
mpeg = newMIME("video/mpeg", ".mpeg", magic.Mpeg)
@@ -274,16 +255,6 @@ var (
pat = newMIME("image/x-gimp-pat", ".pat", magic.Pat)
gbr = newMIME("image/x-gimp-gbr", ".gbr", magic.Gbr)
xfdf = newMIME("application/vnd.adobe.xfdf", ".xfdf", magic.Xfdf)
glb = newMIME("model/gltf-binary", ".glb", magic.GLB)
gltf = newMIME("model/gltf+json", ".gltf", magic.GLTF)
glb = newMIME("model/gltf-binary", ".glb", magic.Glb)
jxr = newMIME("image/jxr", ".jxr", magic.Jxr).alias("image/vnd.ms-photo")
parquet = newMIME("application/vnd.apache.parquet", ".parquet", magic.Par1).
alias("application/x-parquet")
netpbm = newMIME("image/x-portable-bitmap", ".pbm", magic.NetPBM)
netpgm = newMIME("image/x-portable-graymap", ".pgm", magic.NetPGM)
netppm = newMIME("image/x-portable-pixmap", ".ppm", magic.NetPPM)
netpam = newMIME("image/x-portable-arbitrarymap", ".pam", magic.NetPAM)
cbor = newMIME("application/cbor", ".cbor", magic.CBOR)
oneNote = newMIME("application/onenote", ".one", magic.One)
chm = newMIME("application/vnd.ms-htmlhelp", ".chm", magic.CHM)
)

42
vendor/github.com/gin-contrib/sse/README.md generated vendored
View File

@@ -1,7 +1,7 @@
# Server-Sent Events
[![Go Reference](https://pkg.go.dev/badge/github.com/gin-contrib/sse.svg)](https://pkg.go.dev/github.com/gin-contrib/sse)
[![Run Tests](https://github.com/gin-contrib/sse/actions/workflows/go.yml/badge.svg)](https://github.com/gin-contrib/sse/actions/workflows/go.yml)
[![GoDoc](https://godoc.org/github.com/gin-contrib/sse?status.svg)](https://godoc.org/github.com/gin-contrib/sse)
[![Build Status](https://travis-ci.org/gin-contrib/sse.svg)](https://travis-ci.org/gin-contrib/sse)
[![codecov](https://codecov.io/gh/gin-contrib/sse/branch/master/graph/badge.svg)](https://codecov.io/gh/gin-contrib/sse)
[![Go Report Card](https://goreportcard.com/badge/github.com/gin-contrib/sse)](https://goreportcard.com/report/github.com/gin-contrib/sse)
@@ -16,33 +16,32 @@ Server-sent events (SSE) is a technology where a browser receives automatic upda
import "github.com/gin-contrib/sse"
func httpHandler(w http.ResponseWriter, req *http.Request) {
// data can be a primitive like a string, an integer or a float
sse.Encode(w, sse.Event{
Event: "message",
Data: "some data\nmore data",
})
// data can be a primitive like a string, an integer or a float
sse.Encode(w, sse.Event{
Event: "message",
Data: "some data\nmore data",
})
// also a complex type, like a map, a struct or a slice
sse.Encode(w, sse.Event{
Id: "124",
Event: "message",
Data: map[string]interface{}{
"user": "manu",
"date": time.Now().Unix(),
"content": "hi!",
},
})
// also a complex type, like a map, a struct or a slice
sse.Encode(w, sse.Event{
Id: "124",
Event: "message",
Data: map[string]interface{}{
"user": "manu",
"date": time.Now().Unix(),
"content": "hi!",
},
})
}
```
```sh
```
event: message
data: some data\\nmore data
id: 124
event: message
data: {"content":"hi!","date":1431540810,"user":"manu"}
```
## Content-Type
@@ -50,8 +49,7 @@ data: {"content":"hi!","date":1431540810,"user":"manu"}
```go
fmt.Println(sse.ContentType)
```
```sh
```
text/event-stream
```

13
vendor/github.com/gin-contrib/sse/sse-decoder.go generated vendored
View File

@@ -7,6 +7,7 @@ package sse
import (
"bytes"
"io"
"io/ioutil"
)
type decoder struct {
@@ -21,8 +22,7 @@ func Decode(r io.Reader) ([]Event, error) {
func (d *decoder) dispatchEvent(event Event, data string) {
dataLength := len(data)
if dataLength > 0 {
// If the data buffer's last character is a U+000A LINE FEED (LF) character,
// then remove the last character from the data buffer.
//If the data buffer's last character is a U+000A LINE FEED (LF) character, then remove the last character from the data buffer.
data = data[:dataLength-1]
dataLength--
}
@@ -37,13 +37,13 @@ func (d *decoder) dispatchEvent(event Event, data string) {
}
func (d *decoder) decode(r io.Reader) ([]Event, error) {
buf, err := io.ReadAll(r)
buf, err := ioutil.ReadAll(r)
if err != nil {
return nil, err
}
var currentEvent Event
dataBuffer := new(bytes.Buffer)
var dataBuffer *bytes.Buffer = new(bytes.Buffer)
// TODO (and unit tests)
// Lines must be separated by either a U+000D CARRIAGE RETURN U+000A LINE FEED (CRLF) character pair,
// a single U+000A LINE FEED (LF) character,
@@ -96,8 +96,7 @@ func (d *decoder) decode(r io.Reader) ([]Event, error) {
currentEvent.Id = string(value)
case "retry":
// If the field value consists of only characters in the range U+0030 DIGIT ZERO (0) to U+0039 DIGIT NINE (9),
// then interpret the field value as an integer in base ten, and set the event stream's
// reconnection time to that integer.
// then interpret the field value as an integer in base ten, and set the event stream's reconnection time to that integer.
// Otherwise, ignore the field.
currentEvent.Id = string(value)
case "data":
@@ -106,7 +105,7 @@ func (d *decoder) decode(r io.Reader) ([]Event, error) {
// then append a single U+000A LINE FEED (LF) character to the data buffer.
dataBuffer.WriteString("\n")
default:
// Otherwise. The field is ignored.
//Otherwise. The field is ignored.
continue
}
}

44
vendor/github.com/gin-contrib/sse/sse-encoder.go generated vendored
View File

@@ -18,12 +18,10 @@ import (
// W3C Working Draft 29 October 2009
// http://www.w3.org/TR/2009/WD-eventsource-20091029/
const ContentType = "text/event-stream;charset=utf-8"
const ContentType = "text/event-stream"
var (
contentType = []string{ContentType}
noCache = []string{"no-cache"}
)
var contentType = []string{ContentType}
var noCache = []string{"no-cache"}
var fieldReplacer = strings.NewReplacer(
"\n", "\\n",
@@ -50,48 +48,40 @@ func Encode(writer io.Writer, event Event) error {
func writeId(w stringWriter, id string) {
if len(id) > 0 {
_, _ = w.WriteString("id:")
_, _ = fieldReplacer.WriteString(w, id)
_, _ = w.WriteString("\n")
w.WriteString("id:")
fieldReplacer.WriteString(w, id)
w.WriteString("\n")
}
}
func writeEvent(w stringWriter, event string) {
if len(event) > 0 {
_, _ = w.WriteString("event:")
_, _ = fieldReplacer.WriteString(w, event)
_, _ = w.WriteString("\n")
w.WriteString("event:")
fieldReplacer.WriteString(w, event)
w.WriteString("\n")
}
}
func writeRetry(w stringWriter, retry uint) {
if retry > 0 {
_, _ = w.WriteString("retry:")
_, _ = w.WriteString(strconv.FormatUint(uint64(retry), 10))
_, _ = w.WriteString("\n")
w.WriteString("retry:")
w.WriteString(strconv.FormatUint(uint64(retry), 10))
w.WriteString("\n")
}
}
func writeData(w stringWriter, data interface{}) error {
_, _ = w.WriteString("data:")
bData, ok := data.([]byte)
if ok {
_, _ = dataReplacer.WriteString(w, string(bData))
_, _ = w.WriteString("\n\n")
return nil
}
switch kindOfData(data) { //nolint:exhaustive
w.WriteString("data:")
switch kindOfData(data) {
case reflect.Struct, reflect.Slice, reflect.Map:
err := json.NewEncoder(w).Encode(data)
if err != nil {
return err
}
_, _ = w.WriteString("\n")
w.WriteString("\n")
default:
_, _ = dataReplacer.WriteString(w, fmt.Sprint(data))
_, _ = w.WriteString("\n\n")
dataReplacer.WriteString(w, fmt.Sprint(data))
w.WriteString("\n\n")
}
return nil
}

2
vendor/github.com/gin-contrib/sse/writer.go generated vendored
View File

@@ -12,7 +12,7 @@ type stringWrapper struct {
}
func (w stringWrapper) WriteString(str string) (int, error) {
return w.Write([]byte(str))
return w.Writer.Write([]byte(str))
}
func checkWriter(writer io.Writer) stringWriter {

17
vendor/github.com/gin-gonic/gin/gin.go generated vendored
View File

@@ -5,7 +5,6 @@
package gin
import (
"crypto/tls"
"fmt"
"html/template"
"net"
@@ -42,10 +41,8 @@ var defaultTrustedCIDRs = []*net.IPNet{
},
}
var (
regSafePrefix = regexp.MustCompile("[^a-zA-Z0-9/-]+")
regRemoveRepeatedChar = regexp.MustCompile("/{2,}")
)
var regSafePrefix = regexp.MustCompile("[^a-zA-Z0-9/-]+")
var regRemoveRepeatedChar = regexp.MustCompile("/{2,}")
// HandlerFunc defines the handler used by gin middleware as return value.
type HandlerFunc func(*Context)
@@ -518,15 +515,7 @@ func (engine *Engine) RunTLS(addr, certFile, keyFile string) (err error) {
"Please check https://pkg.go.dev/github.com/gin-gonic/gin#readme-don-t-trust-all-proxies for details.")
}
server := &http.Server{
Addr: addr,
Handler: engine.Handler(),
TLSConfig: &tls.Config{
MinVersion: tls.VersionTLS12, // TLS 1.2 or higher
},
}
err = server.ListenAndServeTLS(certFile, keyFile)
err = http.ListenAndServeTLS(addr, certFile, keyFile, engine.Handler())
return
}

2
vendor/github.com/go-playground/validator/v10/Makefile generated vendored
View File

@@ -3,7 +3,7 @@ GOCMD=go
linters-install:
@golangci-lint --version >/dev/null 2>&1 || { \
echo "installing linting tools..."; \
curl -sfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh| sh -s v2.0.2; \
curl -sfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh| sh -s v1.41.1; \
}
lint: linters-install

157
vendor/github.com/go-playground/validator/v10/README.md generated vendored
View File

@@ -1,7 +1,8 @@
Package validator
=================
<img align="right" src="logo.png">[![GitHub release (latest SemVer)](https://img.shields.io/github/v/release/go-playground/validator)](https://github.com/go-playground/validator/releases)
[![Build Status](https://github.com/go-playground/validator/actions/workflows/workflow.yml/badge.svg)](https://github.com/go-playground/validator/actions)
<img align="right" src="logo.png">[![Join the chat at https://gitter.im/go-playground/validator](https://badges.gitter.im/Join%20Chat.svg)](https://gitter.im/go-playground/validator?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
![Project status](https://img.shields.io/badge/version-10.20.0-green.svg)
[![Build Status](https://travis-ci.org/go-playground/validator.svg?branch=master)](https://travis-ci.org/go-playground/validator)
[![Coverage Status](https://coveralls.io/repos/go-playground/validator/badge.svg?branch=master&service=github)](https://coveralls.io/github/go-playground/validator?branch=master)
[![Go Report Card](https://goreportcard.com/badge/github.com/go-playground/validator)](https://goreportcard.com/report/github.com/go-playground/validator)
[![GoDoc](https://godoc.org/github.com/go-playground/validator?status.svg)](https://pkg.go.dev/github.com/go-playground/validator/v10)
@@ -21,11 +22,6 @@ It has the following **unique** features:
- Customizable i18n aware error messages.
- Default validator for the [gin](https://github.com/gin-gonic/gin) web framework; upgrading from v8 to v9 in gin see [here](https://github.com/go-playground/validator/tree/master/_examples/gin-upgrading-overriding)
A Call for Maintainers
----------------------
Please read the discussiong started [here](https://github.com/go-playground/validator/discussions/1330) if you are interested in contributing/helping maintain this package.
Installation
------------
@@ -167,12 +163,10 @@ validate := validator.New(validator.WithRequiredStructEnabled())
| btc_addr_bech32 | Bitcoin Bech32 Address (segwit) |
| credit_card | Credit Card Number |
| mongodb | MongoDB ObjectID |
| mongodb_connection_string | MongoDB Connection String |
| cron | Cron |
| spicedb | SpiceDb ObjectID/Permission/Type |
| datetime | Datetime |
| e164 | e164 formatted phone number |
| ein | U.S. Employeer Identification Number |
| email | E-mail String
| eth_addr | Ethereum Address |
| hexadecimal | Hexadecimal String |
@@ -262,8 +256,6 @@ validate := validator.New(validator.WithRequiredStructEnabled())
| excluded_without | Excluded Without |
| excluded_without_all | Excluded Without All |
| unique | Unique |
| validateFn | Verify if the method `Validate() error` does not return an error (or any specified method) |
#### Aliases:
| Tag | Description |
@@ -273,75 +265,74 @@ validate := validator.New(validator.WithRequiredStructEnabled())
Benchmarks
------
###### Run on MacBook Pro Max M3
###### Run on MacBook Pro (15-inch, 2017) go version go1.10.2 darwin/amd64
```go
go version go1.23.3 darwin/arm64
go version go1.21.0 darwin/arm64
goos: darwin
goarch: arm64
cpu: Apple M3 Max
pkg: github.com/go-playground/validator/v10
BenchmarkFieldSuccess-16 42461943 27.88 ns/op 0 B/op 0 allocs/op
BenchmarkFieldSuccessParallel-16 486632887 2.289 ns/op 0 B/op 0 allocs/op
BenchmarkFieldFailure-16 9566167 121.3 ns/op 200 B/op 4 allocs/op
BenchmarkFieldFailureParallel-16 17551471 83.68 ns/op 200 B/op 4 allocs/op
BenchmarkFieldArrayDiveSuccess-16 7602306 155.6 ns/op 97 B/op 5 allocs/op
BenchmarkFieldArrayDiveSuccessParallel-16 20664610 59.80 ns/op 97 B/op 5 allocs/op
BenchmarkFieldArrayDiveFailure-16 4659756 252.9 ns/op 301 B/op 10 allocs/op
BenchmarkFieldArrayDiveFailureParallel-16 8010116 152.9 ns/op 301 B/op 10 allocs/op
BenchmarkFieldMapDiveSuccess-16 2834575 421.2 ns/op 288 B/op 14 allocs/op
BenchmarkFieldMapDiveSuccessParallel-16 7179700 171.8 ns/op 288 B/op 14 allocs/op
BenchmarkFieldMapDiveFailure-16 3081728 384.4 ns/op 376 B/op 13 allocs/op
BenchmarkFieldMapDiveFailureParallel-16 6058137 204.0 ns/op 377 B/op 13 allocs/op
BenchmarkFieldMapDiveWithKeysSuccess-16 2544975 464.8 ns/op 288 B/op 14 allocs/op
BenchmarkFieldMapDiveWithKeysSuccessParallel-16 6661954 181.4 ns/op 288 B/op 14 allocs/op
BenchmarkFieldMapDiveWithKeysFailure-16 2435484 490.7 ns/op 553 B/op 16 allocs/op
BenchmarkFieldMapDiveWithKeysFailureParallel-16 4249617 282.0 ns/op 554 B/op 16 allocs/op
BenchmarkFieldCustomTypeSuccess-16 14943525 77.35 ns/op 32 B/op 2 allocs/op
BenchmarkFieldCustomTypeSuccessParallel-16 64051954 20.61 ns/op 32 B/op 2 allocs/op
BenchmarkFieldCustomTypeFailure-16 10721384 107.1 ns/op 184 B/op 3 allocs/op
BenchmarkFieldCustomTypeFailureParallel-16 18714495 69.77 ns/op 184 B/op 3 allocs/op
BenchmarkFieldOrTagSuccess-16 4063124 294.3 ns/op 16 B/op 1 allocs/op
BenchmarkFieldOrTagSuccessParallel-16 31903756 41.22 ns/op 18 B/op 1 allocs/op
BenchmarkFieldOrTagFailure-16 7748558 146.8 ns/op 216 B/op 5 allocs/op
BenchmarkFieldOrTagFailureParallel-16 13139854 92.05 ns/op 216 B/op 5 allocs/op
BenchmarkStructLevelValidationSuccess-16 16808389 70.25 ns/op 16 B/op 1 allocs/op
BenchmarkStructLevelValidationSuccessParallel-16 90686955 14.47 ns/op 16 B/op 1 allocs/op
BenchmarkStructLevelValidationFailure-16 5818791 200.2 ns/op 264 B/op 7 allocs/op
BenchmarkStructLevelValidationFailureParallel-16 11115874 107.5 ns/op 264 B/op 7 allocs/op
BenchmarkStructSimpleCustomTypeSuccess-16 7764956 151.9 ns/op 32 B/op 2 allocs/op
BenchmarkStructSimpleCustomTypeSuccessParallel-16 52316265 30.37 ns/op 32 B/op 2 allocs/op
BenchmarkStructSimpleCustomTypeFailure-16 4195429 277.2 ns/op 416 B/op 9 allocs/op
BenchmarkStructSimpleCustomTypeFailureParallel-16 7305661 164.6 ns/op 432 B/op 10 allocs/op
BenchmarkStructFilteredSuccess-16 6312625 186.1 ns/op 216 B/op 5 allocs/op
BenchmarkStructFilteredSuccessParallel-16 13684459 93.42 ns/op 216 B/op 5 allocs/op
BenchmarkStructFilteredFailure-16 6751482 171.2 ns/op 216 B/op 5 allocs/op
BenchmarkStructFilteredFailureParallel-16 14146070 86.93 ns/op 216 B/op 5 allocs/op
BenchmarkStructPartialSuccess-16 6544448 177.3 ns/op 224 B/op 4 allocs/op
BenchmarkStructPartialSuccessParallel-16 13951946 88.73 ns/op 224 B/op 4 allocs/op
BenchmarkStructPartialFailure-16 4075833 287.5 ns/op 440 B/op 9 allocs/op
BenchmarkStructPartialFailureParallel-16 7490805 161.3 ns/op 440 B/op 9 allocs/op
BenchmarkStructExceptSuccess-16 4107187 281.4 ns/op 424 B/op 8 allocs/op
BenchmarkStructExceptSuccessParallel-16 15979173 80.86 ns/op 208 B/op 3 allocs/op
BenchmarkStructExceptFailure-16 4434372 264.3 ns/op 424 B/op 8 allocs/op
BenchmarkStructExceptFailureParallel-16 8081367 154.1 ns/op 424 B/op 8 allocs/op
BenchmarkStructSimpleCrossFieldSuccess-16 6459542 183.4 ns/op 56 B/op 3 allocs/op
BenchmarkStructSimpleCrossFieldSuccessParallel-16 41013781 37.95 ns/op 56 B/op 3 allocs/op
BenchmarkStructSimpleCrossFieldFailure-16 4034998 292.1 ns/op 272 B/op 8 allocs/op
BenchmarkStructSimpleCrossFieldFailureParallel-16 11348446 115.3 ns/op 272 B/op 8 allocs/op
BenchmarkStructSimpleCrossStructCrossFieldSuccess-16 4448528 267.7 ns/op 64 B/op 4 allocs/op
BenchmarkStructSimpleCrossStructCrossFieldSuccessParallel-16 26813619 48.33 ns/op 64 B/op 4 allocs/op
BenchmarkStructSimpleCrossStructCrossFieldFailure-16 3090646 384.5 ns/op 288 B/op 9 allocs/op
BenchmarkStructSimpleCrossStructCrossFieldFailureParallel-16 9870906 129.5 ns/op 288 B/op 9 allocs/op
BenchmarkStructSimpleSuccess-16 10675562 109.5 ns/op 0 B/op 0 allocs/op
BenchmarkStructSimpleSuccessParallel-16 131159784 8.932 ns/op 0 B/op 0 allocs/op
BenchmarkStructSimpleFailure-16 4094979 286.6 ns/op 416 B/op 9 allocs/op
BenchmarkStructSimpleFailureParallel-16 7606663 157.9 ns/op 416 B/op 9 allocs/op
BenchmarkStructComplexSuccess-16 2073470 576.0 ns/op 224 B/op 5 allocs/op
BenchmarkStructComplexSuccessParallel-16 7821831 161.3 ns/op 224 B/op 5 allocs/op
BenchmarkStructComplexFailure-16 576358 2001 ns/op 3042 B/op 48 allocs/op
BenchmarkStructComplexFailureParallel-16 1000000 1171 ns/op 3041 B/op 48 allocs/op
BenchmarkOneof-16 22503973 52.82 ns/op 0 B/op 0 allocs/op
BenchmarkOneofParallel-16 8538474 140.4 ns/op 0 B/op 0 allocs/op
BenchmarkFieldSuccess-8 33142266 35.94 ns/op 0 B/op 0 allocs/op
BenchmarkFieldSuccessParallel-8 200816191 6.568 ns/op 0 B/op 0 allocs/op
BenchmarkFieldFailure-8 6779707 175.1 ns/op 200 B/op 4 allocs/op
BenchmarkFieldFailureParallel-8 11044147 108.4 ns/op 200 B/op 4 allocs/op
BenchmarkFieldArrayDiveSuccess-8 6054232 194.4 ns/op 97 B/op 5 allocs/op
BenchmarkFieldArrayDiveSuccessParallel-8 12523388 94.07 ns/op 97 B/op 5 allocs/op
BenchmarkFieldArrayDiveFailure-8 3587043 334.3 ns/op 300 B/op 10 allocs/op
BenchmarkFieldArrayDiveFailureParallel-8 5816665 200.8 ns/op 300 B/op 10 allocs/op
BenchmarkFieldMapDiveSuccess-8 2217910 540.1 ns/op 288 B/op 14 allocs/op
BenchmarkFieldMapDiveSuccessParallel-8 4446698 258.7 ns/op 288 B/op 14 allocs/op
BenchmarkFieldMapDiveFailure-8 2392759 504.6 ns/op 376 B/op 13 allocs/op
BenchmarkFieldMapDiveFailureParallel-8 4244199 286.9 ns/op 376 B/op 13 allocs/op
BenchmarkFieldMapDiveWithKeysSuccess-8 2005857 592.1 ns/op 288 B/op 14 allocs/op
BenchmarkFieldMapDiveWithKeysSuccessParallel-8 4400850 296.9 ns/op 288 B/op 14 allocs/op
BenchmarkFieldMapDiveWithKeysFailure-8 1850227 643.8 ns/op 553 B/op 16 allocs/op
BenchmarkFieldMapDiveWithKeysFailureParallel-8 3293233 375.1 ns/op 553 B/op 16 allocs/op
BenchmarkFieldCustomTypeSuccess-8 12174412 98.25 ns/op 32 B/op 2 allocs/op
BenchmarkFieldCustomTypeSuccessParallel-8 34389907 35.49 ns/op 32 B/op 2 allocs/op
BenchmarkFieldCustomTypeFailure-8 7582524 156.6 ns/op 184 B/op 3 allocs/op
BenchmarkFieldCustomTypeFailureParallel-8 13019902 92.79 ns/op 184 B/op 3 allocs/op
BenchmarkFieldOrTagSuccess-8 3427260 349.4 ns/op 16 B/op 1 allocs/op
BenchmarkFieldOrTagSuccessParallel-8 15144128 81.25 ns/op 16 B/op 1 allocs/op
BenchmarkFieldOrTagFailure-8 5913546 201.9 ns/op 216 B/op 5 allocs/op
BenchmarkFieldOrTagFailureParallel-8 9810212 113.7 ns/op 216 B/op 5 allocs/op
BenchmarkStructLevelValidationSuccess-8 13456327 87.66 ns/op 16 B/op 1 allocs/op
BenchmarkStructLevelValidationSuccessParallel-8 41818888 27.77 ns/op 16 B/op 1 allocs/op
BenchmarkStructLevelValidationFailure-8 4166284 272.6 ns/op 264 B/op 7 allocs/op
BenchmarkStructLevelValidationFailureParallel-8 7594581 152.1 ns/op 264 B/op 7 allocs/op
BenchmarkStructSimpleCustomTypeSuccess-8 6508082 182.6 ns/op 32 B/op 2 allocs/op
BenchmarkStructSimpleCustomTypeSuccessParallel-8 23078605 54.78 ns/op 32 B/op 2 allocs/op
BenchmarkStructSimpleCustomTypeFailure-8 3118352 381.0 ns/op 416 B/op 9 allocs/op
BenchmarkStructSimpleCustomTypeFailureParallel-8 5300738 224.1 ns/op 432 B/op 10 allocs/op
BenchmarkStructFilteredSuccess-8 4761807 251.1 ns/op 216 B/op 5 allocs/op
BenchmarkStructFilteredSuccessParallel-8 8792598 128.6 ns/op 216 B/op 5 allocs/op
BenchmarkStructFilteredFailure-8 5202573 232.1 ns/op 216 B/op 5 allocs/op
BenchmarkStructFilteredFailureParallel-8 9591267 121.4 ns/op 216 B/op 5 allocs/op
BenchmarkStructPartialSuccess-8 5188512 231.6 ns/op 224 B/op 4 allocs/op
BenchmarkStructPartialSuccessParallel-8 9179776 123.1 ns/op 224 B/op 4 allocs/op
BenchmarkStructPartialFailure-8 3071212 392.5 ns/op 440 B/op 9 allocs/op
BenchmarkStructPartialFailureParallel-8 5344261 223.7 ns/op 440 B/op 9 allocs/op
BenchmarkStructExceptSuccess-8 3184230 375.0 ns/op 424 B/op 8 allocs/op
BenchmarkStructExceptSuccessParallel-8 10090130 108.9 ns/op 208 B/op 3 allocs/op
BenchmarkStructExceptFailure-8 3347226 357.7 ns/op 424 B/op 8 allocs/op
BenchmarkStructExceptFailureParallel-8 5654923 209.5 ns/op 424 B/op 8 allocs/op
BenchmarkStructSimpleCrossFieldSuccess-8 5232265 229.1 ns/op 56 B/op 3 allocs/op
BenchmarkStructSimpleCrossFieldSuccessParallel-8 17436674 64.75 ns/op 56 B/op 3 allocs/op
BenchmarkStructSimpleCrossFieldFailure-8 3128613 383.6 ns/op 272 B/op 8 allocs/op
BenchmarkStructSimpleCrossFieldFailureParallel-8 6994113 168.8 ns/op 272 B/op 8 allocs/op
BenchmarkStructSimpleCrossStructCrossFieldSuccess-8 3506487 340.9 ns/op 64 B/op 4 allocs/op
BenchmarkStructSimpleCrossStructCrossFieldSuccessParallel-8 13431300 91.77 ns/op 64 B/op 4 allocs/op
BenchmarkStructSimpleCrossStructCrossFieldFailure-8 2410566 500.9 ns/op 288 B/op 9 allocs/op
BenchmarkStructSimpleCrossStructCrossFieldFailureParallel-8 6344510 188.2 ns/op 288 B/op 9 allocs/op
BenchmarkStructSimpleSuccess-8 8922726 133.8 ns/op 0 B/op 0 allocs/op
BenchmarkStructSimpleSuccessParallel-8 55291153 23.63 ns/op 0 B/op 0 allocs/op
BenchmarkStructSimpleFailure-8 3171553 378.4 ns/op 416 B/op 9 allocs/op
BenchmarkStructSimpleFailureParallel-8 5571692 212.0 ns/op 416 B/op 9 allocs/op
BenchmarkStructComplexSuccess-8 1683750 714.5 ns/op 224 B/op 5 allocs/op
BenchmarkStructComplexSuccessParallel-8 4578046 257.0 ns/op 224 B/op 5 allocs/op
BenchmarkStructComplexFailure-8 481585 2547 ns/op 3041 B/op 48 allocs/op
BenchmarkStructComplexFailureParallel-8 965764 1577 ns/op 3040 B/op 48 allocs/op
BenchmarkOneof-8 17380881 68.50 ns/op 0 B/op 0 allocs/op
BenchmarkOneofParallel-8 8084733 153.5 ns/op 0 B/op 0 allocs/op
```
Complementary Software
@@ -357,20 +348,6 @@ How to Contribute
Make a pull request...
Maintenance and support for SDK major versions
----------------------------------------------
See prior discussion [here](https://github.com/go-playground/validator/discussions/1342) for more details.
This package is aligned with the [Go release policy](https://go.dev/doc/devel/release) in that support is guaranteed for
the two most recent major versions.
This does not mean the package will not work with older versions of Go, only that we reserve the right to increase the
MSGV(Minimum Supported Go Version) when the need arises to address Security issues/patches, OS issues & support or newly
introduced functionality that would greatly benefit the maintenance and/or usage of this package.
If and when the MSGV is increased it will be done so in a minimum of a `Minor` release bump.
License
-------
Distributed under MIT License, please see license file within the code for more details.

516
vendor/github.com/go-playground/validator/v10/baked_in.go generated vendored
View File

File diff suppressed because it is too large Load Diff

20
vendor/github.com/go-playground/validator/v10/cache.go generated vendored
View File

@@ -21,7 +21,6 @@ const (
typeKeys
typeEndKeys
typeOmitNil
typeOmitZero
)
const (
@@ -124,6 +123,7 @@ func (v *Validate) extractStructCache(current reflect.Value, sName string) *cStr
var customName string
for i := 0; i < numFields; i++ {
fld = typ.Field(i)
if !v.privateFieldValidation && !fld.Anonymous && len(fld.PkgPath) > 0 {
@@ -190,6 +190,7 @@ func (v *Validate) parseFieldTagsRecursive(tag string, fieldName string, alias s
} else {
next, curr := v.parseFieldTagsRecursive(tagsVal, fieldName, t, true)
current.next, current = next, curr
}
continue
}
@@ -208,6 +209,7 @@ func (v *Validate) parseFieldTagsRecursive(tag string, fieldName string, alias s
switch t {
case diveTag:
current.typeof = typeDive
continue
case keysTag:
current.typeof = typeKeys
@@ -216,6 +218,8 @@ func (v *Validate) parseFieldTagsRecursive(tag string, fieldName string, alias s
panic(fmt.Sprintf("'%s' tag must be immediately preceded by the '%s' tag", keysTag, diveTag))
}
current.typeof = typeKeys
// need to pass along only keys tag
// need to increment i to skip over the keys tags
b := make([]byte, 0, 64)
@@ -223,6 +227,7 @@ func (v *Validate) parseFieldTagsRecursive(tag string, fieldName string, alias s
i++
for ; i < len(tags); i++ {
b = append(b, tags[i]...)
b = append(b, ',')
@@ -232,6 +237,7 @@ func (v *Validate) parseFieldTagsRecursive(tag string, fieldName string, alias s
}
current.keys, _ = v.parseFieldTagsRecursive(string(b[:len(b)-1]), fieldName, "", false)
continue
case endKeysTag:
current.typeof = typeEndKeys
@@ -243,21 +249,21 @@ func (v *Validate) parseFieldTagsRecursive(tag string, fieldName string, alias s
}
return
case omitzero:
current.typeof = typeOmitZero
continue
case omitempty:
current.typeof = typeOmitEmpty
continue
case omitnil:
current.typeof = typeOmitNil
continue
case structOnlyTag:
current.typeof = typeStructOnly
continue
case noStructLevelTag:
current.typeof = typeNoStructLevel
continue
default:
if t == isdefault {
@@ -288,7 +294,7 @@ func (v *Validate) parseFieldTagsRecursive(tag string, fieldName string, alias s
if wrapper, ok := v.validations[current.tag]; ok {
current.fn = wrapper.fn
current.runValidationWhenNil = wrapper.runValidationOnNil
current.runValidationWhenNil = wrapper.runValidatinOnNil
} else {
panic(strings.TrimSpace(fmt.Sprintf(undefinedValidation, current.tag, fieldName)))
}
@@ -298,7 +304,7 @@ func (v *Validate) parseFieldTagsRecursive(tag string, fieldName string, alias s
}
if len(vals) > 1 {
current.param = strings.ReplaceAll(strings.ReplaceAll(vals[1], utf8HexComma, ","), utf8Pipe, "|")
current.param = strings.Replace(strings.Replace(vals[1], utf8HexComma, ",", -1), utf8Pipe, "|", -1)
}
}
current.isBlockEnd = true

2320
vendor/github.com/go-playground/validator/v10/country_codes.go generated vendored
View File

File diff suppressed because it is too large Load Diff

148
vendor/github.com/go-playground/validator/v10/currency_codes.go generated vendored
View File

@@ -1,79 +1,79 @@
package validator
var iso4217 = map[string]struct{}{
"AFN": {}, "EUR": {}, "ALL": {}, "DZD": {}, "USD": {},
"AOA": {}, "XCD": {}, "ARS": {}, "AMD": {}, "AWG": {},
"AUD": {}, "AZN": {}, "BSD": {}, "BHD": {}, "BDT": {},
"BBD": {}, "BYN": {}, "BZD": {}, "XOF": {}, "BMD": {},
"INR": {}, "BTN": {}, "BOB": {}, "BOV": {}, "BAM": {},
"BWP": {}, "NOK": {}, "BRL": {}, "BND": {}, "BGN": {},
"BIF": {}, "CVE": {}, "KHR": {}, "XAF": {}, "CAD": {},
"KYD": {}, "CLP": {}, "CLF": {}, "CNY": {}, "COP": {},
"COU": {}, "KMF": {}, "CDF": {}, "NZD": {}, "CRC": {},
"HRK": {}, "CUP": {}, "CUC": {}, "ANG": {}, "CZK": {},
"DKK": {}, "DJF": {}, "DOP": {}, "EGP": {}, "SVC": {},
"ERN": {}, "SZL": {}, "ETB": {}, "FKP": {}, "FJD": {},
"XPF": {}, "GMD": {}, "GEL": {}, "GHS": {}, "GIP": {},
"GTQ": {}, "GBP": {}, "GNF": {}, "GYD": {}, "HTG": {},
"HNL": {}, "HKD": {}, "HUF": {}, "ISK": {}, "IDR": {},
"XDR": {}, "IRR": {}, "IQD": {}, "ILS": {}, "JMD": {},
"JPY": {}, "JOD": {}, "KZT": {}, "KES": {}, "KPW": {},
"KRW": {}, "KWD": {}, "KGS": {}, "LAK": {}, "LBP": {},
"LSL": {}, "ZAR": {}, "LRD": {}, "LYD": {}, "CHF": {},
"MOP": {}, "MKD": {}, "MGA": {}, "MWK": {}, "MYR": {},
"MVR": {}, "MRU": {}, "MUR": {}, "XUA": {}, "MXN": {},
"MXV": {}, "MDL": {}, "MNT": {}, "MAD": {}, "MZN": {},
"MMK": {}, "NAD": {}, "NPR": {}, "NIO": {}, "NGN": {},
"OMR": {}, "PKR": {}, "PAB": {}, "PGK": {}, "PYG": {},
"PEN": {}, "PHP": {}, "PLN": {}, "QAR": {}, "RON": {},
"RUB": {}, "RWF": {}, "SHP": {}, "WST": {}, "STN": {},
"SAR": {}, "RSD": {}, "SCR": {}, "SLL": {}, "SGD": {},
"XSU": {}, "SBD": {}, "SOS": {}, "SSP": {}, "LKR": {},
"SDG": {}, "SRD": {}, "SEK": {}, "CHE": {}, "CHW": {},
"SYP": {}, "TWD": {}, "TJS": {}, "TZS": {}, "THB": {},
"TOP": {}, "TTD": {}, "TND": {}, "TRY": {}, "TMT": {},
"UGX": {}, "UAH": {}, "AED": {}, "USN": {}, "UYU": {},
"UYI": {}, "UYW": {}, "UZS": {}, "VUV": {}, "VES": {},
"VND": {}, "YER": {}, "ZMW": {}, "ZWL": {}, "XBA": {},
"XBB": {}, "XBC": {}, "XBD": {}, "XTS": {}, "XXX": {},
"XAU": {}, "XPD": {}, "XPT": {}, "XAG": {},
var iso4217 = map[string]bool{
"AFN": true, "EUR": true, "ALL": true, "DZD": true, "USD": true,
"AOA": true, "XCD": true, "ARS": true, "AMD": true, "AWG": true,
"AUD": true, "AZN": true, "BSD": true, "BHD": true, "BDT": true,
"BBD": true, "BYN": true, "BZD": true, "XOF": true, "BMD": true,
"INR": true, "BTN": true, "BOB": true, "BOV": true, "BAM": true,
"BWP": true, "NOK": true, "BRL": true, "BND": true, "BGN": true,
"BIF": true, "CVE": true, "KHR": true, "XAF": true, "CAD": true,
"KYD": true, "CLP": true, "CLF": true, "CNY": true, "COP": true,
"COU": true, "KMF": true, "CDF": true, "NZD": true, "CRC": true,
"HRK": true, "CUP": true, "CUC": true, "ANG": true, "CZK": true,
"DKK": true, "DJF": true, "DOP": true, "EGP": true, "SVC": true,
"ERN": true, "SZL": true, "ETB": true, "FKP": true, "FJD": true,
"XPF": true, "GMD": true, "GEL": true, "GHS": true, "GIP": true,
"GTQ": true, "GBP": true, "GNF": true, "GYD": true, "HTG": true,
"HNL": true, "HKD": true, "HUF": true, "ISK": true, "IDR": true,
"XDR": true, "IRR": true, "IQD": true, "ILS": true, "JMD": true,
"JPY": true, "JOD": true, "KZT": true, "KES": true, "KPW": true,
"KRW": true, "KWD": true, "KGS": true, "LAK": true, "LBP": true,
"LSL": true, "ZAR": true, "LRD": true, "LYD": true, "CHF": true,
"MOP": true, "MKD": true, "MGA": true, "MWK": true, "MYR": true,
"MVR": true, "MRU": true, "MUR": true, "XUA": true, "MXN": true,
"MXV": true, "MDL": true, "MNT": true, "MAD": true, "MZN": true,
"MMK": true, "NAD": true, "NPR": true, "NIO": true, "NGN": true,
"OMR": true, "PKR": true, "PAB": true, "PGK": true, "PYG": true,
"PEN": true, "PHP": true, "PLN": true, "QAR": true, "RON": true,
"RUB": true, "RWF": true, "SHP": true, "WST": true, "STN": true,
"SAR": true, "RSD": true, "SCR": true, "SLL": true, "SGD": true,
"XSU": true, "SBD": true, "SOS": true, "SSP": true, "LKR": true,
"SDG": true, "SRD": true, "SEK": true, "CHE": true, "CHW": true,
"SYP": true, "TWD": true, "TJS": true, "TZS": true, "THB": true,
"TOP": true, "TTD": true, "TND": true, "TRY": true, "TMT": true,
"UGX": true, "UAH": true, "AED": true, "USN": true, "UYU": true,
"UYI": true, "UYW": true, "UZS": true, "VUV": true, "VES": true,
"VND": true, "YER": true, "ZMW": true, "ZWL": true, "XBA": true,
"XBB": true, "XBC": true, "XBD": true, "XTS": true, "XXX": true,
"XAU": true, "XPD": true, "XPT": true, "XAG": true,
}
var iso4217_numeric = map[int]struct{}{
8: {}, 12: {}, 32: {}, 36: {}, 44: {},
48: {}, 50: {}, 51: {}, 52: {}, 60: {},
64: {}, 68: {}, 72: {}, 84: {}, 90: {},
96: {}, 104: {}, 108: {}, 116: {}, 124: {},
132: {}, 136: {}, 144: {}, 152: {}, 156: {},
170: {}, 174: {}, 188: {}, 191: {}, 192: {},
203: {}, 208: {}, 214: {}, 222: {}, 230: {},
232: {}, 238: {}, 242: {}, 262: {}, 270: {},
292: {}, 320: {}, 324: {}, 328: {}, 332: {},
340: {}, 344: {}, 348: {}, 352: {}, 356: {},
360: {}, 364: {}, 368: {}, 376: {}, 388: {},
392: {}, 398: {}, 400: {}, 404: {}, 408: {},
410: {}, 414: {}, 417: {}, 418: {}, 422: {},
426: {}, 430: {}, 434: {}, 446: {}, 454: {},
458: {}, 462: {}, 480: {}, 484: {}, 496: {},
498: {}, 504: {}, 512: {}, 516: {}, 524: {},
532: {}, 533: {}, 548: {}, 554: {}, 558: {},
566: {}, 578: {}, 586: {}, 590: {}, 598: {},
600: {}, 604: {}, 608: {}, 634: {}, 643: {},
646: {}, 654: {}, 682: {}, 690: {}, 694: {},
702: {}, 704: {}, 706: {}, 710: {}, 728: {},
748: {}, 752: {}, 756: {}, 760: {}, 764: {},
776: {}, 780: {}, 784: {}, 788: {}, 800: {},
807: {}, 818: {}, 826: {}, 834: {}, 840: {},
858: {}, 860: {}, 882: {}, 886: {}, 901: {},
927: {}, 928: {}, 929: {}, 930: {}, 931: {},
932: {}, 933: {}, 934: {}, 936: {}, 938: {},
940: {}, 941: {}, 943: {}, 944: {}, 946: {},
947: {}, 948: {}, 949: {}, 950: {}, 951: {},
952: {}, 953: {}, 955: {}, 956: {}, 957: {},
958: {}, 959: {}, 960: {}, 961: {}, 962: {},
963: {}, 964: {}, 965: {}, 967: {}, 968: {},
969: {}, 970: {}, 971: {}, 972: {}, 973: {},
975: {}, 976: {}, 977: {}, 978: {}, 979: {},
980: {}, 981: {}, 984: {}, 985: {}, 986: {},
990: {}, 994: {}, 997: {}, 999: {},
var iso4217_numeric = map[int]bool{
8: true, 12: true, 32: true, 36: true, 44: true,
48: true, 50: true, 51: true, 52: true, 60: true,
64: true, 68: true, 72: true, 84: true, 90: true,
96: true, 104: true, 108: true, 116: true, 124: true,
132: true, 136: true, 144: true, 152: true, 156: true,
170: true, 174: true, 188: true, 191: true, 192: true,
203: true, 208: true, 214: true, 222: true, 230: true,
232: true, 238: true, 242: true, 262: true, 270: true,
292: true, 320: true, 324: true, 328: true, 332: true,
340: true, 344: true, 348: true, 352: true, 356: true,
360: true, 364: true, 368: true, 376: true, 388: true,
392: true, 398: true, 400: true, 404: true, 408: true,
410: true, 414: true, 417: true, 418: true, 422: true,
426: true, 430: true, 434: true, 446: true, 454: true,
458: true, 462: true, 480: true, 484: true, 496: true,
498: true, 504: true, 512: true, 516: true, 524: true,
532: true, 533: true, 548: true, 554: true, 558: true,
566: true, 578: true, 586: true, 590: true, 598: true,
600: true, 604: true, 608: true, 634: true, 643: true,
646: true, 654: true, 682: true, 690: true, 694: true,
702: true, 704: true, 706: true, 710: true, 728: true,
748: true, 752: true, 756: true, 760: true, 764: true,
776: true, 780: true, 784: true, 788: true, 800: true,
807: true, 818: true, 826: true, 834: true, 840: true,
858: true, 860: true, 882: true, 886: true, 901: true,
927: true, 928: true, 929: true, 930: true, 931: true,
932: true, 933: true, 934: true, 936: true, 938: true,
940: true, 941: true, 943: true, 944: true, 946: true,
947: true, 948: true, 949: true, 950: true, 951: true,
952: true, 953: true, 955: true, 956: true, 957: true,
958: true, 959: true, 960: true, 961: true, 962: true,
963: true, 964: true, 965: true, 967: true, 968: true,
969: true, 970: true, 971: true, 972: true, 973: true,
975: true, 976: true, 977: true, 978: true, 979: true,
980: true, 981: true, 984: true, 985: true, 986: true,
990: true, 994: true, 997: true, 999: true,
}

53
vendor/github.com/go-playground/validator/v10/doc.go generated vendored
View File

@@ -188,7 +188,7 @@ Same as structonly tag except that any struct level validations will not run.
# Omit Empty
Allows conditional validation, for example, if a field is not set with
Allows conditional validation, for example if a field is not set with
a value (Determined by the "required" validator) then other validation
such as min or max won't run, but if a value is set validation will run.
@@ -253,7 +253,7 @@ Example #2
This validates that the value is not the data types default zero value.
For numbers ensures value is not zero. For strings ensures value is
not "". For booleans ensures value is not false. For slices, maps, pointers, interfaces, channels and functions
not "". For slices, maps, pointers, interfaces, channels and functions
ensures the value is not nil. For structs ensures value is not the zero value when using WithRequiredStructEnabled.
Usage: required
@@ -489,19 +489,12 @@ For strings, ints, and uints, oneof will ensure that the value
is one of the values in the parameter. The parameter should be
a list of values separated by whitespace. Values may be
strings or numbers. To match strings with spaces in them, include
the target string between single quotes. Kind of like an 'enum'.
the target string between single quotes.
Usage: oneof=red green
oneof='red green' 'blue yellow'
oneof=5 7 9
# One Of Case Insensitive
Works the same as oneof but is case insensitive and therefore only accepts strings.
Usage: oneofci=red green
oneofci='red green' 'blue yellow'
# Greater Than
For numbers, this will ensure that the value is greater than the
@@ -756,20 +749,6 @@ in a field of the struct specified via a parameter.
// For slices of struct:
Usage: unique=field
# ValidateFn
This validates that an object responds to a method that can return error or bool.
By default it expects an interface `Validate() error` and check that the method
does not return an error. Other methods can be specified using two signatures:
If the method returns an error, it check if the return value is nil.
If the method returns a boolean, it checks if the value is true.
// to use the default method Validate() error
Usage: validateFn
// to use the custom method IsValid() bool (or error)
Usage: validateFn=IsValid
# Alpha Only
This validates that a string value contains ASCII alpha characters only
@@ -932,7 +911,7 @@ This will accept any uri the golang request uri accepts
# Urn RFC 2141 String
This validates that a string value contains a valid URN
This validataes that a string value contains a valid URN
according to the RFC 2141 spec.
Usage: urn_rfc2141
@@ -973,7 +952,7 @@ Although an empty string is a valid base64 URL safe value, this will report
an empty string as an error, if you wish to accept an empty string as valid
you can use this with the omitempty tag.
Usage: base64rawurl
Usage: base64url
# Bitcoin Address
@@ -987,7 +966,7 @@ Bitcoin Bech32 Address (segwit)
This validates that a string value contains a valid bitcoin Bech32 address as defined
by bip-0173 (https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki)
Special thanks to Pieter Wuille for providing reference implementations.
Special thanks to Pieter Wuille for providng reference implementations.
Usage: btc_addr_bech32
@@ -1148,12 +1127,6 @@ This validates that a string value contains a valid longitude.
Usage: longitude
# Employeer Identification Number EIN
This validates that a string value contains a valid U.S. Employer Identification Number.
Usage: ein
# Social Security Number SSN
This validates that a string value contains a valid U.S. Social Security Number.
@@ -1326,7 +1299,7 @@ may not exist at the time of validation.
# HostPort
This validates that a string value contains a valid DNS hostname and port that
can be used to validate fields typically passed to sockets and connections.
can be used to valiate fields typically passed to sockets and connections.
Usage: hostname_port
@@ -1413,19 +1386,11 @@ This validates that a string value contains a valid credit card number using Luh
This validates that a string or (u)int value contains a valid checksum using the Luhn algorithm.
# MongoDB
# MongoDb ObjectID
This validates that a string is a valid 24 character hexadecimal string or valid connection string.
This validates that a string is a valid 24 character hexadecimal string.
Usage: mongodb
mongodb_connection_string
Example:
type Test struct {
ObjectIdField string `validate:"mongodb"`
ConnectionStringField string `validate:"mongodb_connection_string"`
}
# Cron

15
vendor/github.com/go-playground/validator/v10/errors.go generated vendored
View File

@@ -24,6 +24,7 @@ type InvalidValidationError struct {
// Error returns InvalidValidationError message
func (e *InvalidValidationError) Error() string {
if e.Type == nil {
return "validator: (nil)"
}
@@ -40,9 +41,11 @@ type ValidationErrors []FieldError
// All information to create an error message specific to your application is contained within
// the FieldError found within the ValidationErrors array
func (ve ValidationErrors) Error() string {
buff := bytes.NewBufferString("")
for i := 0; i < len(ve); i++ {
buff.WriteString(ve[i].Error())
buff.WriteString("\n")
}
@@ -52,6 +55,7 @@ func (ve ValidationErrors) Error() string {
// Translate translates all of the ValidationErrors
func (ve ValidationErrors) Translate(ut ut.Translator) ValidationErrorsTranslations {
trans := make(ValidationErrorsTranslations)
var fe *fieldError
@@ -105,24 +109,22 @@ type FieldError interface {
// StructNamespace returns the namespace for the field error, with the field's
// actual name.
//
// eg. "User.FirstName" see Namespace for comparison
// eq. "User.FirstName" see Namespace for comparison
//
// NOTE: this field can be blank when validating a single primitive field
// using validate.Field(...) as there is no way to extract its name
StructNamespace() string
// Field returns the field's name with the tag name taking precedence over the
// Field returns the fields name with the tag name taking precedence over the
// field's actual name.
//
// `RegisterTagNameFunc` must be registered to get tag value.
//
// eg. JSON name "fname"
// eq. JSON name "fname"
// see StructField for comparison
Field() string
// StructField returns the field's actual name from the struct, when able to determine.
//
// eg. "FirstName"
// eq. "FirstName"
// see Field for comparison
StructField() string
@@ -202,6 +204,7 @@ func (fe *fieldError) StructNamespace() string {
// Field returns the field's name with the tag name taking precedence over the
// field's actual name.
func (fe *fieldError) Field() string {
return fe.ns[len(fe.ns)-int(fe.fieldLen):]
// // return fe.field
// fld := fe.ns[len(fe.ns)-int(fe.fieldLen):]

12
vendor/github.com/go-playground/validator/v10/postcode_regexes.go generated vendored
View File

@@ -1,9 +1,6 @@
package validator
import (
"regexp"
"sync"
)
import "regexp"
var postCodePatternDict = map[string]string{
"GB": `^GIR[ ]?0AA|((AB|AL|B|BA|BB|BD|BH|BL|BN|BR|BS|BT|CA|CB|CF|CH|CM|CO|CR|CT|CV|CW|DA|DD|DE|DG|DH|DL|DN|DT|DY|E|EC|EH|EN|EX|FK|FY|G|GL|GY|GU|HA|HD|HG|HP|HR|HS|HU|HX|IG|IM|IP|IV|JE|KA|KT|KW|KY|L|LA|LD|LE|LL|LN|LS|LU|M|ME|MK|ML|N|NE|NG|NN|NP|NR|NW|OL|OX|PA|PE|PH|PL|PO|PR|RG|RH|RM|S|SA|SE|SG|SK|SL|SM|SN|SO|SP|SR|SS|ST|SW|SY|TA|TD|TF|TN|TQ|TR|TS|TW|UB|W|WA|WC|WD|WF|WN|WR|WS|WV|YO|ZE)(\d[\dA-Z]?[ ]?\d[ABD-HJLN-UW-Z]{2}))|BFPO[ ]?\d{1,4}$`,
@@ -167,12 +164,9 @@ var postCodePatternDict = map[string]string{
"YT": `^976\d{2}$`,
}
var (
postcodeRegexInit sync.Once
postCodeRegexDict = map[string]*regexp.Regexp{}
)
var postCodeRegexDict = map[string]*regexp.Regexp{}
func initPostcodes() {
func init() {
for countryCode, pattern := range postCodePatternDict {
postCodeRegexDict[countryCode] = regexp.MustCompile(pattern)
}

162
vendor/github.com/go-playground/validator/v10/regexes.go generated vendored
View File

@@ -1,9 +1,6 @@
package validator
import (
"regexp"
"sync"
)
import "regexp"
const (
alphaRegexString = "^[a-zA-Z]+$"
@@ -69,97 +66,82 @@ const (
splitParamsRegexString = `'[^']*'|\S+`
bicRegexString = `^[A-Za-z]{6}[A-Za-z0-9]{2}([A-Za-z0-9]{3})?$`
semverRegexString = `^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?$` // numbered capture groups https://semver.org/
dnsRegexStringRFC1035Label = "^[a-z]([-a-z0-9]*[a-z0-9])?$"
dnsRegexStringRFC1035Label = "^[a-z]([-a-z0-9]*[a-z0-9]){0,62}$"
cveRegexString = `^CVE-(1999|2\d{3})-(0[^0]\d{2}|0\d[^0]\d{1}|0\d{2}[^0]|[1-9]{1}\d{3,})$` // CVE Format Id https://cve.mitre.org/cve/identifiers/syntaxchange.html
mongodbIdRegexString = "^[a-f\\d]{24}$"
mongodbConnStringRegexString = "^mongodb(\\+srv)?:\\/\\/(([a-zA-Z\\d]+):([a-zA-Z\\d$:\\/?#\\[\\]@]+)@)?(([a-z\\d.-]+)(:[\\d]+)?)((,(([a-z\\d.-]+)(:(\\d+))?))*)?(\\/[a-zA-Z-_]{1,64})?(\\?(([a-zA-Z]+)=([a-zA-Z\\d]+))(&(([a-zA-Z\\d]+)=([a-zA-Z\\d]+))?)*)?$"
cronRegexString = `(@(annually|yearly|monthly|weekly|daily|hourly|reboot))|(@every (\d+(ns|us|µs|ms|s|m|h))+)|((((\d+,)+\d+|((\*|\d+)(\/|-)\d+)|\d+|\*) ?){5,7})`
mongodbRegexString = "^[a-f\\d]{24}$"
cronRegexString = `(@(annually|yearly|monthly|weekly|daily|hourly|reboot))|(@every (\d+(ns|us|µs|ms|s|m|h))+)|((((\d+,)+\d+|(\d+(\/|-)\d+)|\d+|\*) ?){5,7})`
spicedbIDRegexString = `^(([a-zA-Z0-9/_|\-=+]{1,})|\*)$`
spicedbPermissionRegexString = "^([a-z][a-z0-9_]{1,62}[a-z0-9])?$"
spicedbTypeRegexString = "^([a-z][a-z0-9_]{1,61}[a-z0-9]/)?[a-z][a-z0-9_]{1,62}[a-z0-9]$"
einRegexString = "^(\\d{2}-\\d{7})$"
)
func lazyRegexCompile(str string) func() *regexp.Regexp {
var regex *regexp.Regexp
var once sync.Once
return func() *regexp.Regexp {
once.Do(func() {
regex = regexp.MustCompile(str)
})
return regex
}
}
var (
alphaRegex = lazyRegexCompile(alphaRegexString)
alphaNumericRegex = lazyRegexCompile(alphaNumericRegexString)
alphaUnicodeRegex = lazyRegexCompile(alphaUnicodeRegexString)
alphaUnicodeNumericRegex = lazyRegexCompile(alphaUnicodeNumericRegexString)
numericRegex = lazyRegexCompile(numericRegexString)
numberRegex = lazyRegexCompile(numberRegexString)
hexadecimalRegex = lazyRegexCompile(hexadecimalRegexString)
hexColorRegex = lazyRegexCompile(hexColorRegexString)
rgbRegex = lazyRegexCompile(rgbRegexString)
rgbaRegex = lazyRegexCompile(rgbaRegexString)
hslRegex = lazyRegexCompile(hslRegexString)
hslaRegex = lazyRegexCompile(hslaRegexString)
e164Regex = lazyRegexCompile(e164RegexString)
emailRegex = lazyRegexCompile(emailRegexString)
base32Regex = lazyRegexCompile(base32RegexString)
base64Regex = lazyRegexCompile(base64RegexString)
base64URLRegex = lazyRegexCompile(base64URLRegexString)
base64RawURLRegex = lazyRegexCompile(base64RawURLRegexString)
iSBN10Regex = lazyRegexCompile(iSBN10RegexString)
iSBN13Regex = lazyRegexCompile(iSBN13RegexString)
iSSNRegex = lazyRegexCompile(iSSNRegexString)
uUID3Regex = lazyRegexCompile(uUID3RegexString)
uUID4Regex = lazyRegexCompile(uUID4RegexString)
uUID5Regex = lazyRegexCompile(uUID5RegexString)
uUIDRegex = lazyRegexCompile(uUIDRegexString)
uUID3RFC4122Regex = lazyRegexCompile(uUID3RFC4122RegexString)
uUID4RFC4122Regex = lazyRegexCompile(uUID4RFC4122RegexString)
uUID5RFC4122Regex = lazyRegexCompile(uUID5RFC4122RegexString)
uUIDRFC4122Regex = lazyRegexCompile(uUIDRFC4122RegexString)
uLIDRegex = lazyRegexCompile(uLIDRegexString)
md4Regex = lazyRegexCompile(md4RegexString)
md5Regex = lazyRegexCompile(md5RegexString)
sha256Regex = lazyRegexCompile(sha256RegexString)
sha384Regex = lazyRegexCompile(sha384RegexString)
sha512Regex = lazyRegexCompile(sha512RegexString)
ripemd128Regex = lazyRegexCompile(ripemd128RegexString)
ripemd160Regex = lazyRegexCompile(ripemd160RegexString)
tiger128Regex = lazyRegexCompile(tiger128RegexString)
tiger160Regex = lazyRegexCompile(tiger160RegexString)
tiger192Regex = lazyRegexCompile(tiger192RegexString)
aSCIIRegex = lazyRegexCompile(aSCIIRegexString)
printableASCIIRegex = lazyRegexCompile(printableASCIIRegexString)
multibyteRegex = lazyRegexCompile(multibyteRegexString)
dataURIRegex = lazyRegexCompile(dataURIRegexString)
latitudeRegex = lazyRegexCompile(latitudeRegexString)
longitudeRegex = lazyRegexCompile(longitudeRegexString)
sSNRegex = lazyRegexCompile(sSNRegexString)
hostnameRegexRFC952 = lazyRegexCompile(hostnameRegexStringRFC952)
hostnameRegexRFC1123 = lazyRegexCompile(hostnameRegexStringRFC1123)
fqdnRegexRFC1123 = lazyRegexCompile(fqdnRegexStringRFC1123)
btcAddressRegex = lazyRegexCompile(btcAddressRegexString)
btcUpperAddressRegexBech32 = lazyRegexCompile(btcAddressUpperRegexStringBech32)
btcLowerAddressRegexBech32 = lazyRegexCompile(btcAddressLowerRegexStringBech32)
ethAddressRegex = lazyRegexCompile(ethAddressRegexString)
uRLEncodedRegex = lazyRegexCompile(uRLEncodedRegexString)
hTMLEncodedRegex = lazyRegexCompile(hTMLEncodedRegexString)
hTMLRegex = lazyRegexCompile(hTMLRegexString)
jWTRegex = lazyRegexCompile(jWTRegexString)
splitParamsRegex = lazyRegexCompile(splitParamsRegexString)
bicRegex = lazyRegexCompile(bicRegexString)
semverRegex = lazyRegexCompile(semverRegexString)
dnsRegexRFC1035Label = lazyRegexCompile(dnsRegexStringRFC1035Label)
cveRegex = lazyRegexCompile(cveRegexString)
mongodbIdRegex = lazyRegexCompile(mongodbIdRegexString)
mongodbConnectionRegex = lazyRegexCompile(mongodbConnStringRegexString)
cronRegex = lazyRegexCompile(cronRegexString)
spicedbIDRegex = lazyRegexCompile(spicedbIDRegexString)
spicedbPermissionRegex = lazyRegexCompile(spicedbPermissionRegexString)
spicedbTypeRegex = lazyRegexCompile(spicedbTypeRegexString)
einRegex = lazyRegexCompile(einRegexString)
alphaRegex = regexp.MustCompile(alphaRegexString)
alphaNumericRegex = regexp.MustCompile(alphaNumericRegexString)
alphaUnicodeRegex = regexp.MustCompile(alphaUnicodeRegexString)
alphaUnicodeNumericRegex = regexp.MustCompile(alphaUnicodeNumericRegexString)
numericRegex = regexp.MustCompile(numericRegexString)
numberRegex = regexp.MustCompile(numberRegexString)
hexadecimalRegex = regexp.MustCompile(hexadecimalRegexString)
hexColorRegex = regexp.MustCompile(hexColorRegexString)
rgbRegex = regexp.MustCompile(rgbRegexString)
rgbaRegex = regexp.MustCompile(rgbaRegexString)
hslRegex = regexp.MustCompile(hslRegexString)
hslaRegex = regexp.MustCompile(hslaRegexString)
e164Regex = regexp.MustCompile(e164RegexString)
emailRegex = regexp.MustCompile(emailRegexString)
base32Regex = regexp.MustCompile(base32RegexString)
base64Regex = regexp.MustCompile(base64RegexString)
base64URLRegex = regexp.MustCompile(base64URLRegexString)
base64RawURLRegex = regexp.MustCompile(base64RawURLRegexString)
iSBN10Regex = regexp.MustCompile(iSBN10RegexString)
iSBN13Regex = regexp.MustCompile(iSBN13RegexString)
iSSNRegex = regexp.MustCompile(iSSNRegexString)
uUID3Regex = regexp.MustCompile(uUID3RegexString)
uUID4Regex = regexp.MustCompile(uUID4RegexString)
uUID5Regex = regexp.MustCompile(uUID5RegexString)
uUIDRegex = regexp.MustCompile(uUIDRegexString)
uUID3RFC4122Regex = regexp.MustCompile(uUID3RFC4122RegexString)
uUID4RFC4122Regex = regexp.MustCompile(uUID4RFC4122RegexString)
uUID5RFC4122Regex = regexp.MustCompile(uUID5RFC4122RegexString)
uUIDRFC4122Regex = regexp.MustCompile(uUIDRFC4122RegexString)
uLIDRegex = regexp.MustCompile(uLIDRegexString)
md4Regex = regexp.MustCompile(md4RegexString)
md5Regex = regexp.MustCompile(md5RegexString)
sha256Regex = regexp.MustCompile(sha256RegexString)
sha384Regex = regexp.MustCompile(sha384RegexString)
sha512Regex = regexp.MustCompile(sha512RegexString)
ripemd128Regex = regexp.MustCompile(ripemd128RegexString)
ripemd160Regex = regexp.MustCompile(ripemd160RegexString)
tiger128Regex = regexp.MustCompile(tiger128RegexString)
tiger160Regex = regexp.MustCompile(tiger160RegexString)
tiger192Regex = regexp.MustCompile(tiger192RegexString)
aSCIIRegex = regexp.MustCompile(aSCIIRegexString)
printableASCIIRegex = regexp.MustCompile(printableASCIIRegexString)
multibyteRegex = regexp.MustCompile(multibyteRegexString)
dataURIRegex = regexp.MustCompile(dataURIRegexString)
latitudeRegex = regexp.MustCompile(latitudeRegexString)
longitudeRegex = regexp.MustCompile(longitudeRegexString)
sSNRegex = regexp.MustCompile(sSNRegexString)
hostnameRegexRFC952 = regexp.MustCompile(hostnameRegexStringRFC952)
hostnameRegexRFC1123 = regexp.MustCompile(hostnameRegexStringRFC1123)
fqdnRegexRFC1123 = regexp.MustCompile(fqdnRegexStringRFC1123)
btcAddressRegex = regexp.MustCompile(btcAddressRegexString)
btcUpperAddressRegexBech32 = regexp.MustCompile(btcAddressUpperRegexStringBech32)
btcLowerAddressRegexBech32 = regexp.MustCompile(btcAddressLowerRegexStringBech32)
ethAddressRegex = regexp.MustCompile(ethAddressRegexString)
uRLEncodedRegex = regexp.MustCompile(uRLEncodedRegexString)
hTMLEncodedRegex = regexp.MustCompile(hTMLEncodedRegexString)
hTMLRegex = regexp.MustCompile(hTMLRegexString)
jWTRegex = regexp.MustCompile(jWTRegexString)
splitParamsRegex = regexp.MustCompile(splitParamsRegexString)
bicRegex = regexp.MustCompile(bicRegexString)
semverRegex = regexp.MustCompile(semverRegexString)
dnsRegexRFC1035Label = regexp.MustCompile(dnsRegexStringRFC1035Label)
cveRegex = regexp.MustCompile(cveRegexString)
mongodbRegex = regexp.MustCompile(mongodbRegexString)
cronRegex = regexp.MustCompile(cronRegexString)
spicedbIDRegex = regexp.MustCompile(spicedbIDRegexString)
spicedbPermissionRegex = regexp.MustCompile(spicedbPermissionRegexString)
spicedbTypeRegex = regexp.MustCompile(spicedbTypeRegexString)
)

12
vendor/github.com/go-playground/validator/v10/struct_level.go generated vendored
View File

@@ -46,9 +46,9 @@ type StructLevel interface {
//
// NOTES:
//
// fieldName and structFieldName get appended to the existing
// namespace that validator is on. e.g. pass 'FirstName' or
// 'Names[0]' depending on the nesting
// fieldName and altName get appended to the existing namespace that
// validator is on. e.g. pass 'FirstName' or 'Names[0]' depending
// on the nesting
//
// tag can be an existing validation tag or just something you make up
// and process on the flip side it's up to you.
@@ -107,6 +107,7 @@ func (v *validate) ExtractType(field reflect.Value) (reflect.Value, reflect.Kind
// ReportError reports an error just by passing the field and tag information
func (v *validate) ReportError(field interface{}, fieldName, structFieldName, tag, param string) {
fv, kind, _ := v.extractTypeInternal(reflect.ValueOf(field), false)
if len(structFieldName) == 0 {
@@ -122,6 +123,7 @@ func (v *validate) ReportError(field interface{}, fieldName, structFieldName, ta
}
if kind == reflect.Invalid {
v.errs = append(v.errs,
&fieldError{
v: v.v,
@@ -147,7 +149,7 @@ func (v *validate) ReportError(field interface{}, fieldName, structFieldName, ta
structNs: v.str2,
fieldLen: uint8(len(fieldName)),
structfieldLen: uint8(len(structFieldName)),
value: getValue(fv),
value: fv.Interface(),
param: param,
kind: kind,
typ: fv.Type(),
@@ -159,9 +161,11 @@ func (v *validate) ReportError(field interface{}, fieldName, structFieldName, ta
//
// NOTE: this function prepends the current namespace to the relative ones.
func (v *validate) ReportValidationErrors(relativeNamespace, relativeStructNamespace string, errs ValidationErrors) {
var err *fieldError
for i := 0; i < len(errs); i++ {
err = errs[i].(*fieldError)
err.ns = string(append(append(v.ns, relativeNamespace...), err.ns...))
err.structNs = string(append(append(v.actualNs, relativeStructNamespace...), err.structNs...))

16
vendor/github.com/go-playground/validator/v10/util.go generated vendored
View File

@@ -13,6 +13,7 @@ import (
// It will dive into pointers, customTypes and return you the
// underlying value and it's kind.
func (v *validate) extractTypeInternal(current reflect.Value, nullable bool) (reflect.Value, reflect.Kind, bool) {
BEGIN:
switch current.Kind() {
case reflect.Ptr:
@@ -43,6 +44,7 @@ BEGIN:
default:
if v.v.hasCustomFuncs {
if fn, ok := v.v.customFuncs[current.Type()]; ok {
current = reflect.ValueOf(fn(current))
goto BEGIN
@@ -59,6 +61,7 @@ BEGIN:
// NOTE: when not successful ok will be false, this can happen when a nested struct is nil and so the field
// could not be retrieved because it didn't exist.
func (v *validate) getStructFieldOKInternal(val reflect.Value, namespace string) (current reflect.Value, kind reflect.Kind, nullable bool, found bool) {
BEGIN:
current, kind, nullable = v.ExtractType(val)
if kind == reflect.Invalid {
@@ -71,6 +74,7 @@ BEGIN:
}
switch kind {
case reflect.Ptr, reflect.Interface:
return
@@ -81,6 +85,7 @@ BEGIN:
var ns string
if !typ.ConvertibleTo(timeType) {
idx := strings.Index(namespace, namespaceSeparator)
if idx != -1 {
@@ -217,7 +222,7 @@ BEGIN:
panic("Invalid field namespace")
}
// asInt returns the parameter as an int64
// asInt returns the parameter as a int64
// or panics if it can't convert
func asInt(param string) int64 {
i, err := strconv.ParseInt(param, 0, 64)
@@ -251,6 +256,7 @@ func asIntFromType(t reflect.Type, param string) int64 {
// asUint returns the parameter as a uint64
// or panics if it can't convert
func asUint(param string) uint64 {
i, err := strconv.ParseUint(param, 0, 64)
panicIf(err)
@@ -265,7 +271,7 @@ func asFloat64(param string) float64 {
return i
}
// asFloat32 returns the parameter as a float32
// asFloat64 returns the parameter as a float64
// or panics if it can't convert
func asFloat32(param string) float64 {
i, err := strconv.ParseFloat(param, 32)
@@ -276,6 +282,7 @@ func asFloat32(param string) float64 {
// asBool returns the parameter as a bool
// or panics if it can't convert
func asBool(param string) bool {
i, err := strconv.ParseBool(param)
panicIf(err)
@@ -290,13 +297,12 @@ func panicIf(err error) {
// Checks if field value matches regex. If fl.Field can be cast to Stringer, it uses the Stringer interfaces
// String() return value. Otherwise, it uses fl.Field's String() value.
func fieldMatchesRegexByStringerValOrString(regexFn func() *regexp.Regexp, fl FieldLevel) bool {
regex := regexFn()
func fieldMatchesRegexByStringerValOrString(regex *regexp.Regexp, fl FieldLevel) bool {
switch fl.Field().Kind() {
case reflect.String:
return regex.MatchString(fl.Field().String())
default:
if stringer, ok := getValue(fl.Field()).(fmt.Stringer); ok {
if stringer, ok := fl.Field().Interface().(fmt.Stringer); ok {
return regex.MatchString(stringer.String())
} else {
return regex.MatchString(fl.Field().String())

49
vendor/github.com/go-playground/validator/v10/validator.go generated vendored
View File

@@ -32,12 +32,14 @@ type validate struct {
// parent and current will be the same the first run of validateStruct
func (v *validate) validateStruct(ctx context.Context, parent reflect.Value, current reflect.Value, typ reflect.Type, ns []byte, structNs []byte, ct *cTag) {
cs, ok := v.v.structCache.Get(typ)
if !ok {
cs = v.v.extractStructCache(current, typ.Name())
}
if len(ns) == 0 && len(cs.name) != 0 {
ns = append(ns, cs.name...)
ns = append(ns, '.')
@@ -48,17 +50,21 @@ func (v *validate) validateStruct(ctx context.Context, parent reflect.Value, cur
// ct is nil on top level struct, and structs as fields that have no tag info
// so if nil or if not nil and the structonly tag isn't present
if ct == nil || ct.typeof != typeStructOnly {
var f *cField
for i := 0; i < len(cs.fields); i++ {
f = cs.fields[i]
if v.isPartial {
if v.ffn != nil {
// used with StructFiltered
if v.ffn(append(structNs, f.name...)) {
continue
}
} else {
// used with StructPartial & StructExcept
_, ok = v.includeExclude[string(append(structNs, f.name...))]
@@ -77,6 +83,7 @@ func (v *validate) validateStruct(ctx context.Context, parent reflect.Value, cur
// first iteration will have no info about nostructlevel tag, and is checked prior to
// calling the next iteration of validateStruct called from traverseField.
if cs.fn != nil {
v.slflParent = parent
v.slCurrent = current
v.ns = ns
@@ -110,10 +117,6 @@ func (v *validate) traverseField(ctx context.Context, parent reflect.Value, curr
return
}
if ct.typeof == typeOmitZero {
return
}
if ct.hasTag {
if kind == reflect.Invalid {
v.str1 = string(append(ns, cf.altName...))
@@ -235,19 +238,6 @@ OUTER:
ct = ct.next
continue
case typeOmitZero:
v.slflParent = parent
v.flField = current
v.cf = cf
v.ct = ct
if !hasNotZeroValue(v) {
return
}
ct = ct.next
continue
case typeOmitNil:
v.slflParent = parent
v.flField = current
@@ -260,7 +250,7 @@ OUTER:
return
}
default:
if v.fldIsPointer && getValue(field) == nil {
if v.fldIsPointer && field.Interface() == nil {
return
}
}
@@ -284,6 +274,7 @@ OUTER:
reusableCF := &cField{}
for i := 0; i < current.Len(); i++ {
i64 = int64(i)
v.misc = append(v.misc[0:0], cf.name...)
@@ -296,6 +287,7 @@ OUTER:
if cf.namesEqual {
reusableCF.altName = reusableCF.name
} else {
v.misc = append(v.misc[0:0], cf.altName...)
v.misc = append(v.misc, '[')
v.misc = strconv.AppendInt(v.misc, i64, 10)
@@ -312,7 +304,8 @@ OUTER:
reusableCF := &cField{}
for _, key := range current.MapKeys() {
pv = fmt.Sprintf("%v", key)
pv = fmt.Sprintf("%v", key.Interface())
v.misc = append(v.misc[0:0], cf.name...)
v.misc = append(v.misc, '[')
@@ -337,18 +330,6 @@ OUTER:
// can be nil when just keys being validated
if ct.next != nil {
v.traverseField(ctx, parent, current.MapIndex(key), ns, structNs, reusableCF, ct.next)
} else {
// Struct fallback when map values are structs
val := current.MapIndex(key)
switch val.Kind() {
case reflect.Ptr:
if val.Elem().Kind() == reflect.Struct {
// Dive into the struct so its own tags run
v.traverseField(ctx, parent, val, ns, structNs, reusableCF, nil)
}
case reflect.Struct:
v.traverseField(ctx, parent, val, ns, structNs, reusableCF, nil)
}
}
} else {
v.traverseField(ctx, parent, current.MapIndex(key), ns, structNs, reusableCF, ct)
@@ -368,6 +349,7 @@ OUTER:
v.misc = v.misc[0:0]
for {
// set Field Level fields
v.slflParent = parent
v.flField = current
@@ -382,6 +364,7 @@ OUTER:
// drain rest of the 'or' values, then continue or leave
for {
ct = ct.next
if ct == nil {
@@ -418,6 +401,7 @@ OUTER:
}
if ct.hasAlias {
v.errs = append(v.errs,
&fieldError{
v: v.v,
@@ -433,7 +417,9 @@ OUTER:
typ: typ,
},
)
} else {
tVal := string(v.misc)[1:]
v.errs = append(v.errs,
@@ -497,6 +483,7 @@ OUTER:
ct = ct.next
}
}
}
func getValue(val reflect.Value) interface{} {

53
vendor/github.com/go-playground/validator/v10/validator_instance.go generated vendored
View File

@@ -21,7 +21,6 @@ const (
tagKeySeparator = "="
structOnlyTag = "structonly"
noStructLevelTag = "nostructlevel"
omitzero = "omitzero"
omitempty = "omitempty"
omitnil = "omitnil"
isdefault = "isdefault"
@@ -75,8 +74,8 @@ type CustomTypeFunc func(field reflect.Value) interface{}
type TagNameFunc func(field reflect.StructField) string
type internalValidationFuncWrapper struct {
fn FuncCtx
runValidationOnNil bool
fn FuncCtx
runValidatinOnNil bool
}
// Validate contains the validator settings and cache
@@ -104,6 +103,7 @@ type Validate struct {
// in essence only parsing your validation tags once per struct type.
// Using multiple instances neglects the benefit of caching.
func New(options ...Option) *Validate {
tc := new(tagCache)
tc.m.Store(make(map[string]*cTag))
@@ -125,6 +125,7 @@ func New(options ...Option) *Validate {
// must copy validators for separate validations to be used in each instance
for k, val := range bakedInValidators {
switch k {
// these require that even if the value is nil that the validation should run, omitempty still overrides this behaviour
case requiredIfTag, requiredUnlessTag, requiredWithTag, requiredWithAllTag, requiredWithoutTag, requiredWithoutAllTag,
@@ -231,12 +232,30 @@ func (v *Validate) RegisterValidationCtx(tag string, fn FuncCtx, callValidationE
return v.registerValidation(tag, fn, false, nilCheckable)
}
func (v *Validate) registerValidation(tag string, fn FuncCtx, bakedIn bool, nilCheckable bool) error {
if len(tag) == 0 {
return errors.New("function Key cannot be empty")
}
if fn == nil {
return errors.New("function cannot be empty")
}
_, ok := restrictedTags[tag]
if !bakedIn && (ok || strings.ContainsAny(tag, restrictedTagChars)) {
panic(fmt.Sprintf(restrictedTagErr, tag))
}
v.validations[tag] = internalValidationFuncWrapper{fn: fn, runValidatinOnNil: nilCheckable}
return nil
}
// RegisterAlias registers a mapping of a single validation tag that
// defines a common or complex set of validation(s) to simplify adding validation
// to structs.
//
// NOTE: this function is not thread-safe it is intended that these all be registered prior to any validation
func (v *Validate) RegisterAlias(alias, tags string) {
_, ok := restrictedTags[alias]
if ok || strings.ContainsAny(alias, restrictedTagChars) {
@@ -260,6 +279,7 @@ func (v *Validate) RegisterStructValidation(fn StructLevelFunc, types ...interfa
// NOTE:
// - this method is not thread-safe it is intended that these all be registered prior to any validation
func (v *Validate) RegisterStructValidationCtx(fn StructLevelFuncCtx, types ...interface{}) {
if v.structLevelFuncs == nil {
v.structLevelFuncs = make(map[reflect.Type]StructLevelFuncCtx)
}
@@ -306,6 +326,7 @@ func (v *Validate) RegisterStructValidationMapRules(rules map[string]string, typ
//
// NOTE: this method is not thread-safe it is intended that these all be registered prior to any validation
func (v *Validate) RegisterCustomTypeFunc(fn CustomTypeFunc, types ...interface{}) {
if v.customFuncs == nil {
v.customFuncs = make(map[reflect.Type]CustomTypeFunc)
}
@@ -319,6 +340,7 @@ func (v *Validate) RegisterCustomTypeFunc(fn CustomTypeFunc, types ...interface{
// RegisterTranslation registers translations against the provided tag.
func (v *Validate) RegisterTranslation(tag string, trans ut.Translator, registerFn RegisterTranslationsFunc, translationFn TranslationFunc) (err error) {
if v.transTagFunc == nil {
v.transTagFunc = make(map[ut.Translator]map[string]TranslationFunc)
}
@@ -352,6 +374,7 @@ func (v *Validate) Struct(s interface{}) error {
// It returns InvalidValidationError for bad values passed in and nil or ValidationErrors as error otherwise.
// You will need to assert the error if it's not nil eg. err.(validator.ValidationErrors) to access the array of errors.
func (v *Validate) StructCtx(ctx context.Context, s interface{}) (err error) {
val := reflect.ValueOf(s)
top := val
@@ -468,8 +491,10 @@ func (v *Validate) StructPartialCtx(ctx context.Context, s interface{}, fields .
name := typ.Name()
for _, k := range fields {
flds := strings.Split(k, namespaceSeparator)
if len(flds) > 0 {
vd.misc = append(vd.misc[0:0], name...)
// Don't append empty name for unnamed structs
if len(vd.misc) != 0 {
@@ -477,6 +502,7 @@ func (v *Validate) StructPartialCtx(ctx context.Context, s interface{}, fields .
}
for _, s := range flds {
idx := strings.Index(s, leftBracket)
if idx != -1 {
@@ -492,6 +518,7 @@ func (v *Validate) StructPartialCtx(ctx context.Context, s interface{}, fields .
idx = strings.Index(s, leftBracket)
}
} else {
vd.misc = append(vd.misc, s...)
vd.includeExclude[string(vd.misc)] = struct{}{}
}
@@ -554,6 +581,7 @@ func (v *Validate) StructExceptCtx(ctx context.Context, s interface{}, fields ..
name := typ.Name()
for _, key := range fields {
vd.misc = vd.misc[0:0]
if len(name) > 0 {
@@ -648,7 +676,7 @@ func (v *Validate) VarWithValue(field interface{}, other interface{}, tag string
}
// VarWithValueCtx validates a single variable, against another variable/field's value using tag style validation and
// allows passing of contextual validation information via context.Context.
// allows passing of contextual validation validation information via context.Context.
// eg.
// s1 := "abcd"
// s2 := "abcd"
@@ -680,20 +708,3 @@ func (v *Validate) VarWithValueCtx(ctx context.Context, field interface{}, other
v.pool.Put(vd)
return
}
func (v *Validate) registerValidation(tag string, fn FuncCtx, bakedIn bool, nilCheckable bool) error {
if len(tag) == 0 {
return errors.New("function Key cannot be empty")
}
if fn == nil {
return errors.New("function cannot be empty")
}
_, ok := restrictedTags[tag]
if !bakedIn && (ok || strings.ContainsAny(tag, restrictedTagChars)) {
panic(fmt.Sprintf(restrictedTagErr, tag))
}
v.validations[tag] = internalValidationFuncWrapper{fn: fn, runValidationOnNil: nilCheckable}
return nil
}

3
vendor/github.com/goccy/go-json/.golangci.yml generated vendored
View File

@@ -56,9 +56,6 @@ linters:
- cyclop
- containedctx
- revive
- nosnakecase
- exhaustruct
- depguard
issues:
exclude-rules:

2
vendor/github.com/goccy/go-json/Makefile generated vendored
View File

@@ -30,7 +30,7 @@ golangci-lint: | $(BIN_DIR)
GOLANGCI_LINT_TMP_DIR=$$(mktemp -d); \
cd $$GOLANGCI_LINT_TMP_DIR; \
go mod init tmp; \
GOBIN=$(BIN_DIR) go install github.com/golangci/golangci-lint/cmd/golangci-lint@v1.54.2; \
GOBIN=$(BIN_DIR) go install github.com/golangci/golangci-lint/cmd/golangci-lint@v1.48.0; \
rm -rf $$GOLANGCI_LINT_TMP_DIR; \
}

4
vendor/github.com/goccy/go-json/encode.go generated vendored
View File

@@ -52,7 +52,7 @@ func (e *Encoder) EncodeContext(ctx context.Context, v interface{}, optFuncs ...
rctx.Option.Flag |= encoder.ContextOption
rctx.Option.Context = ctx
err := e.encodeWithOption(rctx, v, optFuncs...) //nolint: contextcheck
err := e.encodeWithOption(rctx, v, optFuncs...)
encoder.ReleaseRuntimeContext(rctx)
return err
@@ -120,7 +120,7 @@ func marshalContext(ctx context.Context, v interface{}, optFuncs ...EncodeOption
optFunc(rctx.Option)
}
buf, err := encode(rctx, v) //nolint: contextcheck
buf, err := encode(rctx, v)
if err != nil {
encoder.ReleaseRuntimeContext(rctx)
return nil, err

18
vendor/github.com/goccy/go-json/internal/decoder/compile.go generated vendored
View File

@@ -5,7 +5,6 @@ import (
"fmt"
"reflect"
"strings"
"sync"
"sync/atomic"
"unicode"
"unsafe"
@@ -18,27 +17,22 @@ var (
typeAddr *runtime.TypeAddr
cachedDecoderMap unsafe.Pointer // map[uintptr]decoder
cachedDecoder []Decoder
initOnce sync.Once
)
func initDecoder() {
initOnce.Do(func() {
typeAddr = runtime.AnalyzeTypeAddr()
if typeAddr == nil {
typeAddr = &runtime.TypeAddr{}
}
cachedDecoder = make([]Decoder, typeAddr.AddrRange>>typeAddr.AddrShift+1)
})
func init() {
typeAddr = runtime.AnalyzeTypeAddr()
if typeAddr == nil {
typeAddr = &runtime.TypeAddr{}
}
cachedDecoder = make([]Decoder, typeAddr.AddrRange>>typeAddr.AddrShift+1)
}
func loadDecoderMap() map[uintptr]Decoder {
initDecoder()
p := atomic.LoadPointer(&cachedDecoderMap)
return *(*map[uintptr]Decoder)(unsafe.Pointer(&p))
}
func storeDecoder(typ uintptr, dec Decoder, m map[uintptr]Decoder) {
initDecoder()
newDecoderMap := make(map[uintptr]Decoder, len(m)+1)
newDecoderMap[typ] = dec

1
vendor/github.com/goccy/go-json/internal/decoder/compile_norace.go generated vendored
View File

@@ -10,7 +10,6 @@ import (
)
func CompileToGetDecoder(typ *runtime.Type) (Decoder, error) {
initDecoder()
typeptr := uintptr(unsafe.Pointer(typ))
if typeptr > typeAddr.MaxTypeAddr {
return compileToGetDecoderSlowPath(typeptr, typ)

1
vendor/github.com/goccy/go-json/internal/decoder/compile_race.go generated vendored
View File

@@ -13,7 +13,6 @@ import (
var decMu sync.RWMutex
func CompileToGetDecoder(typ *runtime.Type) (Decoder, error) {
initDecoder()
typeptr := uintptr(unsafe.Pointer(typ))
if typeptr > typeAddr.MaxTypeAddr {
return compileToGetDecoderSlowPath(typeptr, typ)

1
vendor/github.com/goccy/go-json/internal/decoder/ptr.go generated vendored
View File

@@ -85,7 +85,6 @@ func (d *ptrDecoder) Decode(ctx *RuntimeContext, cursor, depth int64, p unsafe.P
}
c, err := d.dec.Decode(ctx, cursor, depth, newptr)
if err != nil {
*(*unsafe.Pointer)(p) = nil
return 0, err
}
cursor = c

2
vendor/github.com/goccy/go-json/internal/decoder/unmarshal_text.go generated vendored
View File

@@ -147,7 +147,7 @@ func (d *unmarshalTextDecoder) DecodePath(ctx *RuntimeContext, cursor, depth int
return nil, 0, fmt.Errorf("json: unmarshal text decoder does not support decode path")
}
func unquoteBytes(s []byte) (t []byte, ok bool) { //nolint: nonamedreturns
func unquoteBytes(s []byte) (t []byte, ok bool) {
length := len(s)
if length < 2 || s[0] != '"' || s[length-1] != '"' {
return

2
vendor/github.com/goccy/go-json/internal/encoder/compact.go generated vendored
View File

@@ -213,8 +213,8 @@ func compactString(dst, src []byte, cursor int64, escape bool) ([]byte, int64, e
dst = append(dst, src[start:cursor]...)
dst = append(dst, `\u202`...)
dst = append(dst, hex[src[cursor+2]&0xF])
start = cursor + 3
cursor += 2
start = cursor + 3
}
}
switch c {

18
vendor/github.com/goccy/go-json/internal/encoder/compiler.go generated vendored
View File

@@ -5,7 +5,6 @@ import (
"encoding"
"encoding/json"
"reflect"
"sync"
"sync/atomic"
"unsafe"
@@ -25,17 +24,14 @@ var (
cachedOpcodeSets []*OpcodeSet
cachedOpcodeMap unsafe.Pointer // map[uintptr]*OpcodeSet
typeAddr *runtime.TypeAddr
initEncoderOnce sync.Once
)
func initEncoder() {
initEncoderOnce.Do(func() {
typeAddr = runtime.AnalyzeTypeAddr()
if typeAddr == nil {
typeAddr = &runtime.TypeAddr{}
}
cachedOpcodeSets = make([]*OpcodeSet, typeAddr.AddrRange>>typeAddr.AddrShift+1)
})
func init() {
typeAddr = runtime.AnalyzeTypeAddr()
if typeAddr == nil {
typeAddr = &runtime.TypeAddr{}
}
cachedOpcodeSets = make([]*OpcodeSet, typeAddr.AddrRange>>typeAddr.AddrShift+1)
}
func loadOpcodeMap() map[uintptr]*OpcodeSet {
@@ -484,7 +480,7 @@ func (c *Compiler) mapCode(typ *runtime.Type) (*MapCode, error) {
func (c *Compiler) listElemCode(typ *runtime.Type) (Code, error) {
switch {
case c.implementsMarshalJSONType(typ) || c.implementsMarshalJSONType(runtime.PtrTo(typ)):
case c.isPtrMarshalJSONType(typ):
return c.marshalJSONCode(typ)
case !typ.Implements(marshalTextType) && runtime.PtrTo(typ).Implements(marshalTextType):
return c.marshalTextCode(typ)

1
vendor/github.com/goccy/go-json/internal/encoder/compiler_norace.go generated vendored
View File

@@ -4,7 +4,6 @@
package encoder
func CompileToGetCodeSet(ctx *RuntimeContext, typeptr uintptr) (*OpcodeSet, error) {
initEncoder()
if typeptr > typeAddr.MaxTypeAddr || typeptr < typeAddr.BaseTypeAddr {
codeSet, err := compileToGetCodeSetSlowPath(typeptr)
if err != nil {

1
vendor/github.com/goccy/go-json/internal/encoder/compiler_race.go generated vendored
View File

@@ -10,7 +10,6 @@ import (
var setsMu sync.RWMutex
func CompileToGetCodeSet(ctx *RuntimeContext, typeptr uintptr) (*OpcodeSet, error) {
initEncoder()
if typeptr > typeAddr.MaxTypeAddr || typeptr < typeAddr.BaseTypeAddr {
codeSet, err := compileToGetCodeSetSlowPath(typeptr)
if err != nil {

5
vendor/github.com/goccy/go-json/internal/encoder/encoder.go generated vendored
View File

@@ -406,11 +406,6 @@ func AppendMarshalJSON(ctx *RuntimeContext, code *Opcode, b []byte, v interface{
rv = newV
}
}
if rv.Kind() == reflect.Ptr && rv.IsNil() {
return AppendNull(ctx, b), nil
}
v = rv.Interface()
var bb []byte
if (code.Flags & MarshalerContextFlags) != 0 {

24
vendor/github.com/goccy/go-json/internal/encoder/int.go generated vendored
View File

@@ -1,27 +1,3 @@
// This files's processing codes are inspired by https://github.com/segmentio/encoding.
// The license notation is as follows.
//
// # MIT License
//
// Copyright (c) 2019 Segment.io, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
package encoder
import (

24
vendor/github.com/goccy/go-json/internal/encoder/string.go generated vendored
View File

@@ -1,27 +1,3 @@
// This files's string processing codes are inspired by https://github.com/segmentio/encoding.
// The license notation is as follows.
//
// # MIT License
//
// Copyright (c) 2019 Segment.io, Inc.
//
// Permission is hereby granted, free of charge, to any person obtaining a copy
// of this software and associated documentation files (the "Software"), to deal
// in the Software without restriction, including without limitation the rights
// to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
// copies of the Software, and to permit persons to whom the Software is
// furnished to do so, subject to the following conditions:
//
// The above copyright notice and this permission notice shall be included in all
// copies or substantial portions of the Software.
//
// THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
// IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
// FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
// AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
// LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
// OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
// SOFTWARE.
package encoder
import (

1
vendor/github.com/goccy/go-json/internal/runtime/rtype.go generated vendored
View File

@@ -252,6 +252,7 @@ func IfaceIndir(*Type) bool
//go:noescape
func RType2Type(t *Type) reflect.Type
//go:nolint structcheck
type emptyInterface struct {
_ *Type
ptr unsafe.Pointer

108
vendor/github.com/goccy/go-json/internal/runtime/type.go generated vendored
View File

@@ -2,7 +2,6 @@ package runtime
import (
"reflect"
"sync"
"unsafe"
)
@@ -24,8 +23,8 @@ type TypeAddr struct {
}
var (
typeAddr *TypeAddr
once sync.Once
typeAddr *TypeAddr
alreadyAnalyzed bool
)
//go:linkname typelinks reflect.typelinks
@@ -35,64 +34,67 @@ func typelinks() ([]unsafe.Pointer, [][]int32)
func rtypeOff(unsafe.Pointer, int32) unsafe.Pointer
func AnalyzeTypeAddr() *TypeAddr {
once.Do(func() {
sections, offsets := typelinks()
if len(sections) != 1 {
return
defer func() {
alreadyAnalyzed = true
}()
if alreadyAnalyzed {
return typeAddr
}
sections, offsets := typelinks()
if len(sections) != 1 {
return nil
}
if len(offsets) != 1 {
return nil
}
section := sections[0]
offset := offsets[0]
var (
min uintptr = uintptr(^uint(0))
max uintptr = 0
isAligned64 = true
isAligned32 = true
)
for i := 0; i < len(offset); i++ {
typ := (*Type)(rtypeOff(section, offset[i]))
addr := uintptr(unsafe.Pointer(typ))
if min > addr {
min = addr
}
if len(offsets) != 1 {
return
if max < addr {
max = addr
}
section := sections[0]
offset := offsets[0]
var (
min uintptr = uintptr(^uint(0))
max uintptr = 0
isAligned64 = true
isAligned32 = true
)
for i := 0; i < len(offset); i++ {
typ := (*Type)(rtypeOff(section, offset[i]))
addr := uintptr(unsafe.Pointer(typ))
if typ.Kind() == reflect.Ptr {
addr = uintptr(unsafe.Pointer(typ.Elem()))
if min > addr {
min = addr
}
if max < addr {
max = addr
}
if typ.Kind() == reflect.Ptr {
addr = uintptr(unsafe.Pointer(typ.Elem()))
if min > addr {
min = addr
}
if max < addr {
max = addr
}
}
isAligned64 = isAligned64 && (addr-min)&63 == 0
isAligned32 = isAligned32 && (addr-min)&31 == 0
}
addrRange := max - min
if addrRange == 0 {
return
}
var addrShift uintptr
if isAligned64 {
addrShift = 6
} else if isAligned32 {
addrShift = 5
}
cacheSize := addrRange >> addrShift
if cacheSize > maxAcceptableTypeAddrRange {
return
}
typeAddr = &TypeAddr{
BaseTypeAddr: min,
MaxTypeAddr: max,
AddrRange: addrRange,
AddrShift: addrShift,
}
})
isAligned64 = isAligned64 && (addr-min)&63 == 0
isAligned32 = isAligned32 && (addr-min)&31 == 0
}
addrRange := max - min
if addrRange == 0 {
return nil
}
var addrShift uintptr
if isAligned64 {
addrShift = 6
} else if isAligned32 {
addrShift = 5
}
cacheSize := addrRange >> addrShift
if cacheSize > maxAcceptableTypeAddrRange {
return nil
}
typeAddr = &TypeAddr{
BaseTypeAddr: min,
MaxTypeAddr: max,
AddrRange: addrRange,
AddrShift: addrShift,
}
return typeAddr
}

35
vendor/github.com/goccy/go-json/json.go generated vendored
View File

@@ -89,31 +89,31 @@ type UnmarshalerContext interface {
//
// Examples of struct field tags and their meanings:
//
// // Field appears in JSON as key "myName".
// Field int `json:"myName"`
// // Field appears in JSON as key "myName".
// Field int `json:"myName"`
//
// // Field appears in JSON as key "myName" and
// // the field is omitted from the object if its value is empty,
// // as defined above.
// Field int `json:"myName,omitempty"`
// // Field appears in JSON as key "myName" and
// // the field is omitted from the object if its value is empty,
// // as defined above.
// Field int `json:"myName,omitempty"`
//
// // Field appears in JSON as key "Field" (the default), but
// // the field is skipped if empty.
// // Note the leading comma.
// Field int `json:",omitempty"`
// // Field appears in JSON as key "Field" (the default), but
// // the field is skipped if empty.
// // Note the leading comma.
// Field int `json:",omitempty"`
//
// // Field is ignored by this package.
// Field int `json:"-"`
// // Field is ignored by this package.
// Field int `json:"-"`
//
// // Field appears in JSON as key "-".
// Field int `json:"-,"`
// // Field appears in JSON as key "-".
// Field int `json:"-,"`
//
// The "string" option signals that a field is stored as JSON inside a
// JSON-encoded string. It applies only to fields of string, floating point,
// integer, or boolean types. This extra level of encoding is sometimes used
// when communicating with JavaScript programs:
//
// Int64String int64 `json:",string"`
// Int64String int64 `json:",string"`
//
// The key name will be used if it's a non-empty string consisting of
// only Unicode letters, digits, and ASCII punctuation except quotation
@@ -166,6 +166,7 @@ type UnmarshalerContext interface {
// JSON cannot represent cyclic data structures and Marshal does not
// handle them. Passing cyclic structures to Marshal will result in
// an infinite recursion.
//
func Marshal(v interface{}) ([]byte, error) {
return MarshalWithOption(v)
}
@@ -263,13 +264,14 @@ func MarshalIndentWithOption(v interface{}, prefix, indent string, optFuncs ...E
//
// The JSON null value unmarshals into an interface, map, pointer, or slice
// by setting that Go value to nil. Because null is often used in JSON to mean
// “not present,” unmarshaling a JSON null into any other Go type has no effect
// ``not present,'' unmarshaling a JSON null into any other Go type has no effect
// on the value and produces no error.
//
// When unmarshaling quoted strings, invalid UTF-8 or
// invalid UTF-16 surrogate pairs are not treated as an error.
// Instead, they are replaced by the Unicode replacement
// character U+FFFD.
//
func Unmarshal(data []byte, v interface{}) error {
return unmarshal(data, v)
}
@@ -297,6 +299,7 @@ func UnmarshalNoEscape(data []byte, v interface{}, optFuncs ...DecodeOptionFunc)
// Number, for JSON numbers
// string, for JSON string literals
// nil, for JSON null
//
type Token = json.Token
// A Number represents a JSON number literal.

9
vendor/github.com/jackc/pgservicefile/.travis.yml generated vendored
View File

@@ -1,9 +0,0 @@
language: go
go:
- 1.x
- tip
matrix:
allow_failures:
- go: tip

5
vendor/github.com/jackc/pgservicefile/README.md generated vendored
View File

@@ -1,5 +1,6 @@
[![](https://godoc.org/github.com/jackc/pgservicefile?status.svg)](https://godoc.org/github.com/jackc/pgservicefile)
[![Build Status](https://travis-ci.org/jackc/pgservicefile.svg)](https://travis-ci.org/jackc/pgservicefile)
[![Go Reference](https://pkg.go.dev/badge/github.com/jackc/pgservicefile.svg)](https://pkg.go.dev/github.com/jackc/pgservicefile)
[![Build Status](https://github.com/jackc/pgservicefile/actions/workflows/ci.yml/badge.svg)](https://github.com/jackc/pgservicefile/actions/workflows/ci.yml)
# pgservicefile

4
vendor/github.com/jackc/pgservicefile/pgservicefile.go generated vendored
View File

@@ -57,7 +57,7 @@ func ParseServicefile(r io.Reader) (*Servicefile, error) {
} else if strings.HasPrefix(line, "[") && strings.HasSuffix(line, "]") {
service = &Service{Name: line[1 : len(line)-1], Settings: make(map[string]string)}
servicefile.Services = append(servicefile.Services, service)
} else {
} else if service != nil {
parts := strings.SplitN(line, "=", 2)
if len(parts) != 2 {
return nil, fmt.Errorf("unable to parse line %d", lineNum)
@@ -67,6 +67,8 @@ func ParseServicefile(r io.Reader) (*Servicefile, error) {
value := strings.TrimSpace(parts[1])
service.Settings[key] = value
} else {
return nil, fmt.Errorf("line %d is not in a section", lineNum)
}
}

111
vendor/github.com/jackc/pgx/v5/CHANGELOG.md generated vendored
View File

@@ -1,3 +1,114 @@
# 5.7.1 (September 10, 2024)
* Fix data race in tracelog.TraceLog
* Update puddle to v2.2.2. This removes the import of nanotime via linkname.
* Update golang.org/x/crypto and golang.org/x/text
# 5.7.0 (September 7, 2024)
* Add support for sslrootcert=system (Yann Soubeyrand)
* Add LoadTypes to load multiple types in a single SQL query (Nick Farrell)
* Add XMLCodec supports encoding + scanning XML column type like json (nickcruess-soda)
* Add MultiTrace (Stepan Rabotkin)
* Add TraceLogConfig with customizable TimeKey (stringintech)
* pgx.ErrNoRows wraps sql.ErrNoRows to aid in database/sql compatibility with native pgx functions (merlin)
* Support scanning binary formatted uint32 into string / TextScanner (jennifersp)
* Fix interval encoding to allow 0s and avoid extra spaces (Carlos Pérez-Aradros Herce)
* Update pgservicefile - fixes panic when parsing invalid file
* Better error message when reading past end of batch
* Don't print url when url.Parse returns an error (Kevin Biju)
* Fix snake case name normalization collision in RowToStructByName with db tag (nolandseigler)
* Fix: Scan and encode types with underlying types of arrays
# 5.6.0 (May 25, 2024)
* Add StrictNamedArgs (Tomas Zahradnicek)
* Add support for macaddr8 type (Carlos Pérez-Aradros Herce)
* Add SeverityUnlocalized field to PgError / Notice
* Performance optimization of RowToStructByPos/Name (Zach Olstein)
* Allow customizing context canceled behavior for pgconn
* Add ScanLocation to pgtype.Timestamp[tz]Codec
* Add custom data to pgconn.PgConn
* Fix ResultReader.Read() to handle nil values
* Do not encode interval microseconds when they are 0 (Carlos Pérez-Aradros Herce)
* pgconn.SafeToRetry checks for wrapped errors (tjasko)
* Failed connection attempts include all errors
* Optimize LargeObject.Read (Mitar)
* Add tracing for connection acquire and release from pool (ngavinsir)
* Fix encode driver.Valuer not called when nil
* Add support for custom JSON marshal and unmarshal (Mitar)
* Use Go default keepalive for TCP connections (Hans-Joachim Kliemeck)
# 5.5.5 (March 9, 2024)
Use spaces instead of parentheses for SQL sanitization.
This still solves the problem of negative numbers creating a line comment, but this avoids breaking edge cases such as
`set foo to $1` where the substitution is taking place in a location where an arbitrary expression is not allowed.
# 5.5.4 (March 4, 2024)
Fix CVE-2024-27304
SQL injection can occur if an attacker can cause a single query or bind message to exceed 4 GB in size. An integer
overflow in the calculated message size can cause the one large message to be sent as multiple messages under the
attacker's control.
Thanks to Paul Gerste for reporting this issue.
* Fix behavior of CollectRows to return empty slice if Rows are empty (Felix)
* Fix simple protocol encoding of json.RawMessage
* Fix *Pipeline.getResults should close pipeline on error
* Fix panic in TryFindUnderlyingTypeScanPlan (David Kurman)
* Fix deallocation of invalidated cached statements in a transaction
* Handle invalid sslkey file
* Fix scan float4 into sql.Scanner
* Fix pgtype.Bits not making copy of data from read buffer. This would cause the data to be corrupted by future reads.
# 5.5.3 (February 3, 2024)
* Fix: prepared statement already exists
* Improve CopyFrom auto-conversion of text-ish values
* Add ltree type support (Florent Viel)
* Make some properties of Batch and QueuedQuery public (Pavlo Golub)
* Add AppendRows function (Edoardo Spadolini)
* Optimize convert UUID [16]byte to string (Kirill Malikov)
* Fix: LargeObject Read and Write of more than ~1GB at a time (Mitar)
# 5.5.2 (January 13, 2024)
* Allow NamedArgs to start with underscore
* pgproto3: Maximum message body length support (jeremy.spriet)
* Upgrade golang.org/x/crypto to v0.17.0
* Add snake_case support to RowToStructByName (Tikhon Fedulov)
* Fix: update description cache after exec prepare (James Hartig)
* Fix: pipeline checks if it is closed (James Hartig and Ryan Fowler)
* Fix: normalize timeout / context errors during TLS startup (Samuel Stauffer)
* Add OnPgError for easier centralized error handling (James Hartig)
# 5.5.1 (December 9, 2023)
* Add CopyFromFunc helper function. (robford)
* Add PgConn.Deallocate method that uses PostgreSQL protocol Close message.
* pgx uses new PgConn.Deallocate method. This allows deallocating statements to work in a failed transaction. This fixes a case where the prepared statement map could become invalid.
* Fix: Prefer driver.Valuer over json.Marshaler for json fields. (Jacopo)
* Fix: simple protocol SQL sanitizer previously panicked if an invalid $0 placeholder was used. This now returns an error instead. (maksymnevajdev)
* Add pgtype.Numeric.ScanScientific (Eshton Robateau)
# 5.5.0 (November 4, 2023)
* Add CollectExactlyOneRow. (Julien GOTTELAND)
* Add OpenDBFromPool to create *database/sql.DB from *pgxpool.Pool. (Lev Zakharov)
* Prepare can automatically choose statement name based on sql. This makes it easier to explicitly manage prepared statements.
* Statement cache now uses deterministic, stable statement names.
* database/sql prepared statement names are deterministically generated.
* Fix: SendBatch wasn't respecting context cancellation.
* Fix: Timeout error from pipeline is now normalized.
* Fix: database/sql encoding json.RawMessage to []byte.
* CancelRequest: Wait for the cancel request to be acknowledged by the server. This should improve PgBouncer compatibility. (Anton Levakin)
* stdlib: Use Ping instead of CheckConn in ResetSession
* Add json.Marshaler and json.Unmarshaler for Float4, Float8 (Kirill Mironov)
# 5.4.3 (August 5, 2023)
* Fix: QCharArrayOID was defined with the wrong OID (Christoph Engelbert)

19
vendor/github.com/jackc/pgx/v5/CONTRIBUTING.md generated vendored
View File

@@ -29,6 +29,7 @@ Create and setup a test database:
export PGDATABASE=pgx_test
createdb
psql -c 'create extension hstore;'
psql -c 'create extension ltree;'
psql -c 'create domain uint64 as numeric(20,0);'
```
@@ -79,20 +80,11 @@ echo "listen_addresses = '127.0.0.1'" >> .testdb/$POSTGRESQL_DATA_DIR/postgresql
echo "port = $PGPORT" >> .testdb/$POSTGRESQL_DATA_DIR/postgresql.conf
cat testsetup/postgresql_ssl.conf >> .testdb/$POSTGRESQL_DATA_DIR/postgresql.conf
cp testsetup/pg_hba.conf .testdb/$POSTGRESQL_DATA_DIR/pg_hba.conf
cp testsetup/ca.cnf .testdb
cp testsetup/localhost.cnf .testdb
cp testsetup/pgx_sslcert.cnf .testdb
cd .testdb
# Generate a CA public / private key pair.
openssl genrsa -out ca.key 4096
openssl req -x509 -config ca.cnf -new -nodes -key ca.key -sha256 -days 365 -subj '/O=pgx-test-root' -out ca.pem
# Generate the certificate for localhost (the server).
openssl genrsa -out localhost.key 2048
openssl req -new -config localhost.cnf -key localhost.key -out localhost.csr
openssl x509 -req -in localhost.csr -CA ca.pem -CAkey ca.key -CAcreateserial -out localhost.crt -days 364 -sha256 -extfile localhost.cnf -extensions v3_req
# Generate CA, server, and encrypted client certificates.
go run ../testsetup/generate_certs.go
# Copy certificates to server directory and set permissions.
cp ca.pem $POSTGRESQL_DATA_DIR/root.crt
@@ -100,11 +92,6 @@ cp localhost.key $POSTGRESQL_DATA_DIR/server.key
chmod 600 $POSTGRESQL_DATA_DIR/server.key
cp localhost.crt $POSTGRESQL_DATA_DIR/server.crt
# Generate the certificate for client authentication.
openssl genrsa -des3 -out pgx_sslcert.key -passout pass:certpw 2048
openssl req -new -config pgx_sslcert.cnf -key pgx_sslcert.key -passin pass:certpw -out pgx_sslcert.csr
openssl x509 -req -in pgx_sslcert.csr -CA ca.pem -CAkey ca.key -CAcreateserial -out pgx_sslcert.crt -days 363 -sha256 -extfile pgx_sslcert.cnf -extensions v3_req
cd ..
```

7
vendor/github.com/jackc/pgx/v5/README.md generated vendored
View File

@@ -86,9 +86,13 @@ It is also possible to use the `database/sql` interface and convert a connection
See CONTRIBUTING.md for setup instructions.
## Architecture
See the presentation at Golang Estonia, [PGX Top to Bottom](https://www.youtube.com/watch?v=sXMSWhcHCf8) for a description of pgx architecture.
## Supported Go and PostgreSQL Versions
pgx supports the same versions of Go and PostgreSQL that are supported by their respective teams. For [Go](https://golang.org/doc/devel/release.html#policy) that is the two most recent major releases and for [PostgreSQL](https://www.postgresql.org/support/versioning/) the major releases in the last 5 years. This means pgx supports Go 1.19 and higher and PostgreSQL 11 and higher. pgx also is tested against the latest version of [CockroachDB](https://www.cockroachlabs.com/product/).
pgx supports the same versions of Go and PostgreSQL that are supported by their respective teams. For [Go](https://golang.org/doc/devel/release.html#policy) that is the two most recent major releases and for [PostgreSQL](https://www.postgresql.org/support/versioning/) the major releases in the last 5 years. This means pgx supports Go 1.21 and higher and PostgreSQL 12 and higher. pgx also is tested against the latest version of [CockroachDB](https://www.cockroachlabs.com/product/).
## Version Policy
@@ -116,6 +120,7 @@ pgerrcode contains constants for the PostgreSQL error codes.
* [github.com/jackc/pgx-gofrs-uuid](https://github.com/jackc/pgx-gofrs-uuid)
* [github.com/jackc/pgx-shopspring-decimal](https://github.com/jackc/pgx-shopspring-decimal)
* [github.com/twpayne/pgx-geos](https://github.com/twpayne/pgx-geos) ([PostGIS](https://postgis.net/) and [GEOS](https://libgeos.org/) via [go-geos](https://github.com/twpayne/go-geos))
* [github.com/vgarvardt/pgx-google-uuid](https://github.com/vgarvardt/pgx-google-uuid)

84
vendor/github.com/jackc/pgx/v5/batch.go generated vendored
View File

@@ -10,9 +10,9 @@ import (
// QueuedQuery is a query that has been queued for execution via a Batch.
type QueuedQuery struct {
query string
arguments []any
fn batchItemFunc
SQL string
Arguments []any
Fn batchItemFunc
sd *pgconn.StatementDescription
}
@@ -20,7 +20,7 @@ type batchItemFunc func(br BatchResults) error
// Query sets fn to be called when the response to qq is received.
func (qq *QueuedQuery) Query(fn func(rows Rows) error) {
qq.fn = func(br BatchResults) error {
qq.Fn = func(br BatchResults) error {
rows, _ := br.Query()
defer rows.Close()
@@ -36,7 +36,7 @@ func (qq *QueuedQuery) Query(fn func(rows Rows) error) {
// Query sets fn to be called when the response to qq is received.
func (qq *QueuedQuery) QueryRow(fn func(row Row) error) {
qq.fn = func(br BatchResults) error {
qq.Fn = func(br BatchResults) error {
row := br.QueryRow()
return fn(row)
}
@@ -44,7 +44,7 @@ func (qq *QueuedQuery) QueryRow(fn func(row Row) error) {
// Exec sets fn to be called when the response to qq is received.
func (qq *QueuedQuery) Exec(fn func(ct pgconn.CommandTag) error) {
qq.fn = func(br BatchResults) error {
qq.Fn = func(br BatchResults) error {
ct, err := br.Exec()
if err != nil {
return err
@@ -57,22 +57,28 @@ func (qq *QueuedQuery) Exec(fn func(ct pgconn.CommandTag) error) {
// Batch queries are a way of bundling multiple queries together to avoid
// unnecessary network round trips. A Batch must only be sent once.
type Batch struct {
queuedQueries []*QueuedQuery
QueuedQueries []*QueuedQuery
}
// Queue queues a query to batch b. query can be an SQL query or the name of a prepared statement.
// Queue queues a query to batch b. query can be an SQL query or the name of a prepared statement. The only pgx option
// argument that is supported is QueryRewriter. Queries are executed using the connection's DefaultQueryExecMode.
//
// While query can contain multiple statements if the connection's DefaultQueryExecMode is QueryModeSimple, this should
// be avoided. QueuedQuery.Fn must not be set as it will only be called for the first query. That is, QueuedQuery.Query,
// QueuedQuery.QueryRow, and QueuedQuery.Exec must not be called. In addition, any error messages or tracing that
// include the current query may reference the wrong query.
func (b *Batch) Queue(query string, arguments ...any) *QueuedQuery {
qq := &QueuedQuery{
query: query,
arguments: arguments,
SQL: query,
Arguments: arguments,
}
b.queuedQueries = append(b.queuedQueries, qq)
b.QueuedQueries = append(b.QueuedQueries, qq)
return qq
}
// Len returns number of queries that have been queued so far.
func (b *Batch) Len() int {
return len(b.queuedQueries)
return len(b.QueuedQueries)
}
type BatchResults interface {
@@ -126,7 +132,7 @@ func (br *batchResults) Exec() (pgconn.CommandTag, error) {
if !br.mrr.NextResult() {
err := br.mrr.Close()
if err == nil {
err = errors.New("no result")
err = errors.New("no more results in batch")
}
if br.conn.batchTracer != nil {
br.conn.batchTracer.TraceBatchQuery(br.ctx, br.conn, TraceBatchQueryData{
@@ -178,7 +184,7 @@ func (br *batchResults) Query() (Rows, error) {
if !br.mrr.NextResult() {
rows.err = br.mrr.Close()
if rows.err == nil {
rows.err = errors.New("no result")
rows.err = errors.New("no more results in batch")
}
rows.closed = true
@@ -225,9 +231,9 @@ func (br *batchResults) Close() error {
}
// Read and run fn for all remaining items
for br.err == nil && !br.closed && br.b != nil && br.qqIdx < len(br.b.queuedQueries) {
if br.b.queuedQueries[br.qqIdx].fn != nil {
err := br.b.queuedQueries[br.qqIdx].fn(br)
for br.err == nil && !br.closed && br.b != nil && br.qqIdx < len(br.b.QueuedQueries) {
if br.b.QueuedQueries[br.qqIdx].Fn != nil {
err := br.b.QueuedQueries[br.qqIdx].Fn(br)
if err != nil {
br.err = err
}
@@ -251,10 +257,10 @@ func (br *batchResults) earlyError() error {
}
func (br *batchResults) nextQueryAndArgs() (query string, args []any, ok bool) {
if br.b != nil && br.qqIdx < len(br.b.queuedQueries) {
bi := br.b.queuedQueries[br.qqIdx]
query = bi.query
args = bi.arguments
if br.b != nil && br.qqIdx < len(br.b.QueuedQueries) {
bi := br.b.QueuedQueries[br.qqIdx]
query = bi.SQL
args = bi.Arguments
ok = true
br.qqIdx++
}
@@ -285,7 +291,10 @@ func (br *pipelineBatchResults) Exec() (pgconn.CommandTag, error) {
return pgconn.CommandTag{}, br.err
}
query, arguments, _ := br.nextQueryAndArgs()
query, arguments, err := br.nextQueryAndArgs()
if err != nil {
return pgconn.CommandTag{}, err
}
results, err := br.pipeline.GetResults()
if err != nil {
@@ -328,9 +337,9 @@ func (br *pipelineBatchResults) Query() (Rows, error) {
return &baseRows{err: br.err, closed: true}, br.err
}
query, arguments, ok := br.nextQueryAndArgs()
if !ok {
query = "batch query"
query, arguments, err := br.nextQueryAndArgs()
if err != nil {
return &baseRows{err: err, closed: true}, err
}
rows := br.conn.getRows(br.ctx, query, arguments)
@@ -394,9 +403,9 @@ func (br *pipelineBatchResults) Close() error {
}
// Read and run fn for all remaining items
for br.err == nil && !br.closed && br.b != nil && br.qqIdx < len(br.b.queuedQueries) {
if br.b.queuedQueries[br.qqIdx].fn != nil {
err := br.b.queuedQueries[br.qqIdx].fn(br)
for br.err == nil && !br.closed && br.b != nil && br.qqIdx < len(br.b.QueuedQueries) {
if br.b.QueuedQueries[br.qqIdx].Fn != nil {
err := br.b.QueuedQueries[br.qqIdx].Fn(br)
if err != nil {
br.err = err
}
@@ -419,13 +428,16 @@ func (br *pipelineBatchResults) earlyError() error {
return br.err
}
func (br *pipelineBatchResults) nextQueryAndArgs() (query string, args []any, ok bool) {
if br.b != nil && br.qqIdx < len(br.b.queuedQueries) {
bi := br.b.queuedQueries[br.qqIdx]
query = bi.query
args = bi.arguments
ok = true
br.qqIdx++
func (br *pipelineBatchResults) nextQueryAndArgs() (query string, args []any, err error) {
if br.b == nil {
return "", nil, errors.New("no reference to batch")
}
return
if br.qqIdx >= len(br.b.QueuedQueries) {
return "", nil, errors.New("no more results in batch")
}
bi := br.b.QueuedQueries[br.qqIdx]
br.qqIdx++
return bi.SQL, bi.Arguments, nil
}

207
vendor/github.com/jackc/pgx/v5/conn.go generated vendored
View File

@@ -2,13 +2,15 @@ package pgx
import (
"context"
"crypto/sha256"
"database/sql"
"encoding/hex"
"errors"
"fmt"
"strconv"
"strings"
"time"
"github.com/jackc/pgx/v5/internal/anynil"
"github.com/jackc/pgx/v5/internal/sanitize"
"github.com/jackc/pgx/v5/internal/stmtcache"
"github.com/jackc/pgx/v5/pgconn"
@@ -35,7 +37,7 @@ type ConnConfig struct {
// DefaultQueryExecMode controls the default mode for executing queries. By default pgx uses the extended protocol
// and automatically prepares and caches prepared statements. However, this may be incompatible with proxies such as
// PGBouncer. In this case it may be preferrable to use QueryExecModeExec or QueryExecModeSimpleProtocol. The same
// PGBouncer. In this case it may be preferable to use QueryExecModeExec or QueryExecModeSimpleProtocol. The same
// functionality can be controlled on a per query basis by passing a QueryExecMode as the first query argument.
DefaultQueryExecMode QueryExecMode
@@ -99,11 +101,33 @@ func (ident Identifier) Sanitize() string {
return strings.Join(parts, ".")
}
// ErrNoRows occurs when rows are expected but none are returned.
var ErrNoRows = errors.New("no rows in result set")
var (
// ErrNoRows occurs when rows are expected but none are returned.
ErrNoRows = newProxyErr(sql.ErrNoRows, "no rows in result set")
// ErrTooManyRows occurs when more rows than expected are returned.
ErrTooManyRows = errors.New("too many rows in result set")
)
var errDisabledStatementCache = fmt.Errorf("cannot use QueryExecModeCacheStatement with disabled statement cache")
var errDisabledDescriptionCache = fmt.Errorf("cannot use QueryExecModeCacheDescribe with disabled description cache")
func newProxyErr(background error, msg string) error {
return &proxyError{
msg: msg,
background: background,
}
}
type proxyError struct {
msg string
background error
}
func (err *proxyError) Error() string { return err.msg }
func (err *proxyError) Unwrap() error { return err.background }
var (
errDisabledStatementCache = fmt.Errorf("cannot use QueryExecModeCacheStatement with disabled statement cache")
errDisabledDescriptionCache = fmt.Errorf("cannot use QueryExecModeCacheDescribe with disabled description cache")
)
// Connect establishes a connection with a PostgreSQL server with a connection string. See
// pgconn.Connect for details.
@@ -269,7 +293,7 @@ func connect(ctx context.Context, config *ConnConfig) (c *Conn, err error) {
return c, nil
}
// Close closes a connection. It is safe to call Close on a already closed
// Close closes a connection. It is safe to call Close on an already closed
// connection.
func (c *Conn) Close(ctx context.Context) error {
if c.IsClosed() {
@@ -280,12 +304,15 @@ func (c *Conn) Close(ctx context.Context) error {
return err
}
// Prepare creates a prepared statement with name and sql. sql can contain placeholders
// for bound parameters. These placeholders are referenced positional as $1, $2, etc.
// Prepare creates a prepared statement with name and sql. sql can contain placeholders for bound parameters. These
// placeholders are referenced positionally as $1, $2, etc. name can be used instead of sql with Query, QueryRow, and
// Exec to execute the statement. It can also be used with Batch.Queue.
//
// Prepare is idempotent; i.e. it is safe to call Prepare multiple times with the same
// name and sql arguments. This allows a code path to Prepare and Query/Exec without
// concern for if the statement has already been prepared.
// The underlying PostgreSQL identifier for the prepared statement will be name if name != sql or a digest of sql if
// name == sql.
//
// Prepare is idempotent; i.e. it is safe to call Prepare multiple times with the same name and sql arguments. This
// allows a code path to Prepare and Query/Exec without concern for if the statement has already been prepared.
func (c *Conn) Prepare(ctx context.Context, name, sql string) (sd *pgconn.StatementDescription, err error) {
if c.prepareTracer != nil {
ctx = c.prepareTracer.TracePrepareStart(ctx, c, TracePrepareStartData{Name: name, SQL: sql})
@@ -307,23 +334,48 @@ func (c *Conn) Prepare(ctx context.Context, name, sql string) (sd *pgconn.Statem
}()
}
sd, err = c.pgConn.Prepare(ctx, name, sql, nil)
var psName, psKey string
if name == sql {
digest := sha256.Sum256([]byte(sql))
psName = "stmt_" + hex.EncodeToString(digest[0:24])
psKey = sql
} else {
psName = name
psKey = name
}
sd, err = c.pgConn.Prepare(ctx, psName, sql, nil)
if err != nil {
return nil, err
}
if name != "" {
c.preparedStatements[name] = sd
if psKey != "" {
c.preparedStatements[psKey] = sd
}
return sd, nil
}
// Deallocate released a prepared statement
// Deallocate releases a prepared statement. Calling Deallocate on a non-existent prepared statement will succeed.
func (c *Conn) Deallocate(ctx context.Context, name string) error {
delete(c.preparedStatements, name)
_, err := c.pgConn.Exec(ctx, "deallocate "+quoteIdentifier(name)).ReadAll()
return err
var psName string
sd := c.preparedStatements[name]
if sd != nil {
psName = sd.Name
} else {
psName = name
}
err := c.pgConn.Deallocate(ctx, psName)
if err != nil {
return err
}
if sd != nil {
delete(c.preparedStatements, name)
}
return nil
}
// DeallocateAll releases all previously prepared statements from the server and client, where it also resets the statement and description cache.
@@ -441,7 +493,7 @@ optionLoop:
if queryRewriter != nil {
sql, arguments, err = queryRewriter.RewriteQuery(ctx, c, sql, arguments)
if err != nil {
return pgconn.CommandTag{}, fmt.Errorf("rewrite query failed: %v", err)
return pgconn.CommandTag{}, fmt.Errorf("rewrite query failed: %w", err)
}
}
@@ -461,7 +513,7 @@ optionLoop:
}
sd := c.statementCache.Get(sql)
if sd == nil {
sd, err = c.Prepare(ctx, stmtcache.NextStatementName(), sql)
sd, err = c.Prepare(ctx, stmtcache.StatementName(sql), sql)
if err != nil {
return pgconn.CommandTag{}, err
}
@@ -479,6 +531,7 @@ optionLoop:
if err != nil {
return pgconn.CommandTag{}, err
}
c.descriptionCache.Put(sd)
}
return c.execParams(ctx, sd, arguments)
@@ -573,32 +626,35 @@ type QueryExecMode int32
const (
_ QueryExecMode = iota
// Automatically prepare and cache statements. This uses the extended protocol. Queries are executed in a single
// round trip after the statement is cached. This is the default.
// Automatically prepare and cache statements. This uses the extended protocol. Queries are executed in a single round
// trip after the statement is cached. This is the default. If the database schema is modified or the search_path is
// changed after a statement is cached then the first execution of a previously cached query may fail. e.g. If the
// number of columns returned by a "SELECT *" changes or the type of a column is changed.
QueryExecModeCacheStatement
// Cache statement descriptions (i.e. argument and result types) and assume they do not change. This uses the
// extended protocol. Queries are executed in a single round trip after the description is cached. If the database
// schema is modified or the search_path is changed this may result in undetected result decoding errors.
// Cache statement descriptions (i.e. argument and result types) and assume they do not change. This uses the extended
// protocol. Queries are executed in a single round trip after the description is cached. If the database schema is
// modified or the search_path is changed after a statement is cached then the first execution of a previously cached
// query may fail. e.g. If the number of columns returned by a "SELECT *" changes or the type of a column is changed.
QueryExecModeCacheDescribe
// Get the statement description on every execution. This uses the extended protocol. Queries require two round trips
// to execute. It does not use named prepared statements. But it does use the unnamed prepared statement to get the
// statement description on the first round trip and then uses it to execute the query on the second round trip. This
// may cause problems with connection poolers that switch the underlying connection between round trips. It is safe
// even when the the database schema is modified concurrently.
// even when the database schema is modified concurrently.
QueryExecModeDescribeExec
// Assume the PostgreSQL query parameter types based on the Go type of the arguments. This uses the extended protocol
// with text formatted parameters and results. Queries are executed in a single round trip. Type mappings can be
// registered with pgtype.Map.RegisterDefaultPgType. Queries will be rejected that have arguments that are
// unregistered or ambigious. e.g. A map[string]string may have the PostgreSQL type json or hstore. Modes that know
// unregistered or ambiguous. e.g. A map[string]string may have the PostgreSQL type json or hstore. Modes that know
// the PostgreSQL type can use a map[string]string directly as an argument. This mode cannot.
QueryExecModeExec
// Use the simple protocol. Assume the PostgreSQL query parameter types based on the Go type of the arguments.
// Queries are executed in a single round trip. Type mappings can be registered with
// pgtype.Map.RegisterDefaultPgType. Queries will be rejected that have arguments that are unregistered or ambigious.
// pgtype.Map.RegisterDefaultPgType. Queries will be rejected that have arguments that are unregistered or ambiguous.
// e.g. A map[string]string may have the PostgreSQL type json or hstore. Modes that know the PostgreSQL type can use
// a map[string]string directly as an argument. This mode cannot.
//
@@ -705,7 +761,7 @@ optionLoop:
sql, args, err = queryRewriter.RewriteQuery(ctx, c, sql, args)
if err != nil {
rows := c.getRows(ctx, originalSQL, originalArgs)
err = fmt.Errorf("rewrite query failed: %v", err)
err = fmt.Errorf("rewrite query failed: %w", err)
rows.fatal(err)
return rows, err
}
@@ -717,7 +773,6 @@ optionLoop:
}
c.eqb.reset()
anynil.NormalizeSlice(args)
rows := c.getRows(ctx, sql, args)
var err error
@@ -807,7 +862,6 @@ func (c *Conn) getStatementDescription(
mode QueryExecMode,
sql string,
) (sd *pgconn.StatementDescription, err error) {
switch mode {
case QueryExecModeCacheStatement:
if c.statementCache == nil {
@@ -815,7 +869,7 @@ func (c *Conn) getStatementDescription(
}
sd = c.statementCache.Get(sql)
if sd == nil {
sd, err = c.Prepare(ctx, stmtcache.NextStatementName(), sql)
sd, err = c.Prepare(ctx, stmtcache.StatementName(sql), sql)
if err != nil {
return nil, err
}
@@ -865,15 +919,14 @@ func (c *Conn) SendBatch(ctx context.Context, b *Batch) (br BatchResults) {
return &batchResults{ctx: ctx, conn: c, err: err}
}
mode := c.config.DefaultQueryExecMode
for _, bi := range b.queuedQueries {
for _, bi := range b.QueuedQueries {
var queryRewriter QueryRewriter
sql := bi.query
arguments := bi.arguments
sql := bi.SQL
arguments := bi.Arguments
optionLoop:
for len(arguments) > 0 {
// Update Batch.Queue function comment when additional options are implemented
switch arg := arguments[0].(type) {
case QueryRewriter:
queryRewriter = arg
@@ -887,21 +940,23 @@ func (c *Conn) SendBatch(ctx context.Context, b *Batch) (br BatchResults) {
var err error
sql, arguments, err = queryRewriter.RewriteQuery(ctx, c, sql, arguments)
if err != nil {
return &batchResults{ctx: ctx, conn: c, err: fmt.Errorf("rewrite query failed: %v", err)}
return &batchResults{ctx: ctx, conn: c, err: fmt.Errorf("rewrite query failed: %w", err)}
}
}
bi.query = sql
bi.arguments = arguments
bi.SQL = sql
bi.Arguments = arguments
}
// TODO: changing mode per batch? Update Batch.Queue function comment when implemented
mode := c.config.DefaultQueryExecMode
if mode == QueryExecModeSimpleProtocol {
return c.sendBatchQueryExecModeSimpleProtocol(ctx, b)
}
// All other modes use extended protocol and thus can use prepared statements.
for _, bi := range b.queuedQueries {
if sd, ok := c.preparedStatements[bi.query]; ok {
for _, bi := range b.QueuedQueries {
if sd, ok := c.preparedStatements[bi.SQL]; ok {
bi.sd = sd
}
}
@@ -922,11 +977,11 @@ func (c *Conn) SendBatch(ctx context.Context, b *Batch) (br BatchResults) {
func (c *Conn) sendBatchQueryExecModeSimpleProtocol(ctx context.Context, b *Batch) *batchResults {
var sb strings.Builder
for i, bi := range b.queuedQueries {
for i, bi := range b.QueuedQueries {
if i > 0 {
sb.WriteByte(';')
}
sql, err := c.sanitizeForSimpleQuery(bi.query, bi.arguments...)
sql, err := c.sanitizeForSimpleQuery(bi.SQL, bi.Arguments...)
if err != nil {
return &batchResults{ctx: ctx, conn: c, err: err}
}
@@ -945,21 +1000,21 @@ func (c *Conn) sendBatchQueryExecModeSimpleProtocol(ctx context.Context, b *Batc
func (c *Conn) sendBatchQueryExecModeExec(ctx context.Context, b *Batch) *batchResults {
batch := &pgconn.Batch{}
for _, bi := range b.queuedQueries {
for _, bi := range b.QueuedQueries {
sd := bi.sd
if sd != nil {
err := c.eqb.Build(c.typeMap, sd, bi.arguments)
err := c.eqb.Build(c.typeMap, sd, bi.Arguments)
if err != nil {
return &batchResults{ctx: ctx, conn: c, err: err}
}
batch.ExecPrepared(sd.Name, c.eqb.ParamValues, c.eqb.ParamFormats, c.eqb.ResultFormats)
} else {
err := c.eqb.Build(c.typeMap, nil, bi.arguments)
err := c.eqb.Build(c.typeMap, nil, bi.Arguments)
if err != nil {
return &batchResults{ctx: ctx, conn: c, err: err}
}
batch.ExecParams(bi.query, c.eqb.ParamValues, nil, c.eqb.ParamFormats, c.eqb.ResultFormats)
batch.ExecParams(bi.SQL, c.eqb.ParamValues, nil, c.eqb.ParamFormats, c.eqb.ResultFormats)
}
}
@@ -984,18 +1039,18 @@ func (c *Conn) sendBatchQueryExecModeCacheStatement(ctx context.Context, b *Batc
distinctNewQueries := []*pgconn.StatementDescription{}
distinctNewQueriesIdxMap := make(map[string]int)
for _, bi := range b.queuedQueries {
for _, bi := range b.QueuedQueries {
if bi.sd == nil {
sd := c.statementCache.Get(bi.query)
sd := c.statementCache.Get(bi.SQL)
if sd != nil {
bi.sd = sd
} else {
if idx, present := distinctNewQueriesIdxMap[bi.query]; present {
if idx, present := distinctNewQueriesIdxMap[bi.SQL]; present {
bi.sd = distinctNewQueries[idx]
} else {
sd = &pgconn.StatementDescription{
Name: stmtcache.NextStatementName(),
SQL: bi.query,
Name: stmtcache.StatementName(bi.SQL),
SQL: bi.SQL,
}
distinctNewQueriesIdxMap[sd.SQL] = len(distinctNewQueries)
distinctNewQueries = append(distinctNewQueries, sd)
@@ -1016,17 +1071,17 @@ func (c *Conn) sendBatchQueryExecModeCacheDescribe(ctx context.Context, b *Batch
distinctNewQueries := []*pgconn.StatementDescription{}
distinctNewQueriesIdxMap := make(map[string]int)
for _, bi := range b.queuedQueries {
for _, bi := range b.QueuedQueries {
if bi.sd == nil {
sd := c.descriptionCache.Get(bi.query)
sd := c.descriptionCache.Get(bi.SQL)
if sd != nil {
bi.sd = sd
} else {
if idx, present := distinctNewQueriesIdxMap[bi.query]; present {
if idx, present := distinctNewQueriesIdxMap[bi.SQL]; present {
bi.sd = distinctNewQueries[idx]
} else {
sd = &pgconn.StatementDescription{
SQL: bi.query,
SQL: bi.SQL,
}
distinctNewQueriesIdxMap[sd.SQL] = len(distinctNewQueries)
distinctNewQueries = append(distinctNewQueries, sd)
@@ -1043,13 +1098,13 @@ func (c *Conn) sendBatchQueryExecModeDescribeExec(ctx context.Context, b *Batch)
distinctNewQueries := []*pgconn.StatementDescription{}
distinctNewQueriesIdxMap := make(map[string]int)
for _, bi := range b.queuedQueries {
for _, bi := range b.QueuedQueries {
if bi.sd == nil {
if idx, present := distinctNewQueriesIdxMap[bi.query]; present {
if idx, present := distinctNewQueriesIdxMap[bi.SQL]; present {
bi.sd = distinctNewQueries[idx]
} else {
sd := &pgconn.StatementDescription{
SQL: bi.query,
SQL: bi.SQL,
}
distinctNewQueriesIdxMap[sd.SQL] = len(distinctNewQueries)
distinctNewQueries = append(distinctNewQueries, sd)
@@ -1062,7 +1117,7 @@ func (c *Conn) sendBatchQueryExecModeDescribeExec(ctx context.Context, b *Batch)
}
func (c *Conn) sendBatchExtendedWithDescription(ctx context.Context, b *Batch, distinctNewQueries []*pgconn.StatementDescription, sdCache stmtcache.Cache) (pbr *pipelineBatchResults) {
pipeline := c.pgConn.StartPipeline(context.Background())
pipeline := c.pgConn.StartPipeline(ctx)
defer func() {
if pbr != nil && pbr.err != nil {
pipeline.Close()
@@ -1115,11 +1170,11 @@ func (c *Conn) sendBatchExtendedWithDescription(ctx context.Context, b *Batch, d
}
// Queue the queries.
for _, bi := range b.queuedQueries {
err := c.eqb.Build(c.typeMap, bi.sd, bi.arguments)
for _, bi := range b.QueuedQueries {
err := c.eqb.Build(c.typeMap, bi.sd, bi.Arguments)
if err != nil {
// we wrap the error so we the user can understand which query failed inside the batch
err = fmt.Errorf("error building query %s: %w", bi.query, err)
err = fmt.Errorf("error building query %s: %w", bi.SQL, err)
return &pipelineBatchResults{ctx: ctx, conn: c, err: err, closed: true}
}
@@ -1164,7 +1219,15 @@ func (c *Conn) sanitizeForSimpleQuery(sql string, args ...any) (string, error) {
return sanitize.SanitizeSQL(sql, valueArgs...)
}
// LoadType inspects the database for typeName and produces a pgtype.Type suitable for registration.
// LoadType inspects the database for typeName and produces a pgtype.Type suitable for registration. typeName must be
// the name of a type where the underlying type(s) is already understood by pgx. It is for derived types. In particular,
// typeName must be one of the following:
// - An array type name of a type that is already registered. e.g. "_foo" when "foo" is registered.
// - A composite type name where all field types are already registered.
// - A domain type name where the base type is already registered.
// - An enum type name.
// - A range type name where the element type is already registered.
// - A multirange type name where the element type is already registered.
func (c *Conn) LoadType(ctx context.Context, typeName string) (*pgtype.Type, error) {
var oid uint32
@@ -1307,17 +1370,17 @@ order by attnum`,
}
func (c *Conn) deallocateInvalidatedCachedStatements(ctx context.Context) error {
if c.pgConn.TxStatus() != 'I' {
if txStatus := c.pgConn.TxStatus(); txStatus != 'I' && txStatus != 'T' {
return nil
}
if c.descriptionCache != nil {
c.descriptionCache.HandleInvalidated()
c.descriptionCache.RemoveInvalidated()
}
var invalidatedStatements []*pgconn.StatementDescription
if c.statementCache != nil {
invalidatedStatements = c.statementCache.HandleInvalidated()
invalidatedStatements = c.statementCache.GetInvalidated()
}
if len(invalidatedStatements) == 0 {
@@ -1329,7 +1392,6 @@ func (c *Conn) deallocateInvalidatedCachedStatements(ctx context.Context) error
for _, sd := range invalidatedStatements {
pipeline.SendDeallocate(sd.Name)
delete(c.preparedStatements, sd.Name)
}
err := pipeline.Sync()
@@ -1342,5 +1404,10 @@ func (c *Conn) deallocateInvalidatedCachedStatements(ctx context.Context) error
return fmt.Errorf("failed to deallocate cached statement(s): %w", err)
}
c.statementCache.RemoveInvalidated()
for _, sd := range invalidatedStatements {
delete(c.preparedStatements, sd.Name)
}
return nil
}

27
vendor/github.com/jackc/pgx/v5/copy_from.go generated vendored
View File

@@ -64,6 +64,33 @@ func (cts *copyFromSlice) Err() error {
return cts.err
}
// CopyFromFunc returns a CopyFromSource interface that relies on nxtf for values.
// nxtf returns rows until it either signals an 'end of data' by returning row=nil and err=nil,
// or it returns an error. If nxtf returns an error, the copy is aborted.
func CopyFromFunc(nxtf func() (row []any, err error)) CopyFromSource {
return &copyFromFunc{next: nxtf}
}
type copyFromFunc struct {
next func() ([]any, error)
valueRow []any
err error
}
func (g *copyFromFunc) Next() bool {
g.valueRow, g.err = g.next()
// only return true if valueRow exists and no error
return g.valueRow != nil && g.err == nil
}
func (g *copyFromFunc) Values() ([]any, error) {
return g.valueRow, g.err
}
func (g *copyFromFunc) Err() error {
return g.err
}
// CopyFromSource is the interface used by *Conn.CopyFrom as the source for copy data.
type CopyFromSource interface {
// Next returns true if there is another row and makes the next row data

15
vendor/github.com/jackc/pgx/v5/doc.go generated vendored
View File

@@ -11,9 +11,10 @@ The primary way of establishing a connection is with [pgx.Connect]:
conn, err := pgx.Connect(context.Background(), os.Getenv("DATABASE_URL"))
The database connection string can be in URL or DSN format. Both PostgreSQL settings and pgx settings can be specified
here. In addition, a config struct can be created by [ParseConfig] and modified before establishing the connection with
[ConnectConfig] to configure settings such as tracing that cannot be configured with a connection string.
The database connection string can be in URL or key/value format. Both PostgreSQL settings and pgx settings can be
specified here. In addition, a config struct can be created by [ParseConfig] and modified before establishing the
connection with [ConnectConfig] to configure settings such as tracing that cannot be configured with a connection
string.
Connection Pool
@@ -23,8 +24,8 @@ github.com/jackc/pgx/v5/pgxpool for a concurrency safe connection pool.
Query Interface
pgx implements Query in the familiar database/sql style. However, pgx provides generic functions such as CollectRows and
ForEachRow that are a simpler and safer way of processing rows than manually calling rows.Next(), rows.Scan, and
rows.Err().
ForEachRow that are a simpler and safer way of processing rows than manually calling defer rows.Close(), rows.Next(),
rows.Scan, and rows.Err().
CollectRows can be used collect all returned rows into a slice.
@@ -174,7 +175,7 @@ notification is received or the context is canceled.
Tracing and Logging
pgx supports tracing by setting ConnConfig.Tracer.
pgx supports tracing by setting ConnConfig.Tracer. To combine several tracers you can use the multitracer.Tracer.
In addition, the tracelog package provides the TraceLog type which lets a traditional logger act as a Tracer.
@@ -187,7 +188,7 @@ implemented on top of pgconn. The Conn.PgConn() method can be used to access thi
PgBouncer
By default pgx automatically uses prepared statements. Prepared statements are incompaptible with PgBouncer. This can be
By default pgx automatically uses prepared statements. Prepared statements are incompatible with PgBouncer. This can be
disabled by setting a different QueryExecMode in ConnConfig.DefaultQueryExecMode.
*/
package pgx

Some files were not shown because too many files have changed in this diff Show More