pig/pig-farm-controller
1
0
Fork 0
Code Issues 11 Pull Requests Actions Packages Projects Releases Wiki Activity

用户登录和接口鉴权

Browse Source
This commit is contained in:
huang
2025-09-07 21:13:15 +08:00
parent c4522b974b
commit 565cf3fa6a
380 changed files with 18330 additions and 16854 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
vendor/github.com/bytedance/sonic/.gitignore generated vendored
View File

@@ -49,7 +49,4 @@ ast/bench.sh
!testdata/*.json.gz
fuzz/testdata
*__debug_bin*
*pprof
*coverage.txt
tools/venv/*
*__debug_bin

3
vendor/github.com/bytedance/sonic/.gitmodules generated vendored
View File

@@ -4,6 +4,3 @@
[submodule "tools/simde"]
path = tools/simde
url = https://github.com/simd-everywhere/simde.git
[submodule "fuzz/go-fuzz-corpus"]
path = fuzz/go-fuzz-corpus
url = https://github.com/dvyukov/go-fuzz-corpus.git

56
vendor/github.com/bytedance/sonic/README.md generated vendored
View File

@@ -6,10 +6,9 @@ A blazingly fast JSON serializing & deserializing library, accelerated by JI
## Requirement
- Go: 1.18~1.25
- Notice: Go1.24.0 is not supported due to the [issue](https://github.com/golang/go/issues/71672), please use higher go version or add build tag `--ldflags="-checklinkname=0"`
- OS: Linux / MacOS / Windows
- CPU: AMD64 / (ARM64, need go1.20 above)
- Go 1.16~1.22
- Linux / MacOS / Windows(need go1.17 above)
- Amd64 ARCH
## Features
@@ -212,7 +211,7 @@ ret, err := Encode(v, EscapeHTML) // ret == `{"\u0026\u0026":{"X":"\u003c\u003e"
### Compact Format
Sonic encodes primitive objects (struct/map...) as compact-format JSON by default, except marshaling `json.RawMessage` or `json.Marshaler`: sonic ensures validating their output JSON but **DO NOT** compacting them for performance concerns. We provide the option `encoder.CompactMarshaler` to add compacting process.
Sonic encodes primitive objects (struct/map...) as compact-format JSON by default, except marshaling `json.RawMessage` or `json.Marshaler`: sonic ensures validating their output JSON but **DONOT** compacting them for performance concerns. We provide the option `encoder.CompactMarshaler` to add compacting process.
### Print Error
@@ -283,22 +282,6 @@ sub := root.Get("key3").Index(2).Int64() // == 3
**Tip**: since `Index()` uses offset to locate data, which is much faster than scanning like `Get()`, we suggest you use it as much as possible. And sonic also provides another API `IndexOrGet()` to underlying use offset as well as ensure the key is matched.
#### SearchOption
`Searcher` provides some options for user to meet different needs:
```go
opts := ast.SearchOption{ CopyReturn: true ... }
val, err := sonic.GetWithOptions(JSON, opts, "key")
```
- CopyReturn
Indicate the searcher to copy the result JSON string instead of refer from the input. This can help to reduce memory usage if you cache the results
- ConcurentRead
Since `ast.Node` use `Lazy-Load` design, it doesn't support Concurrently-Read by default. If you want to read it concurrently, please specify it.
- ValidateJSON
Indicate the searcher to validate the entire JSON. This option is enabled by default, which slow down the search speed a little.
#### Set/Unset
Modify the json content by Set()/Unset()
@@ -385,12 +368,16 @@ See [ast/visitor.go](https://github.com/bytedance/sonic/blob/main/ast/visitor.go
## Compatibility
For developers who want to use sonic to meet different scenarios, we provide some integrated configs as `sonic.API`
Sonic **DOES NOT** ensure to support all environments, due to the difficulty of developing high-performance codes. For developers who use sonic to build their applications in different environments, we have the following suggestions:
- `ConfigDefault`: the sonic's default config (`EscapeHTML=false`,`SortKeys=false`...) to run sonic fast meanwhile ensure security.
- `ConfigStd`: the std-compatible config (`EscapeHTML=true`,`SortKeys=true`...)
- `ConfigFastest`: the fastest config (`NoQuoteTextMarshaler=true`) to run on sonic as fast as possible.
Sonic **DOES NOT** ensure to support all environments, due to the difficulty of developing high-performance codes. On non-sonic-supporting environment, the implementation will fall back to `encoding/json`. Thus below configs will all equal to `ConfigStd`.
- Developing on **Mac M1**: Make sure you have Rosetta 2 installed on your machine, and set `GOARCH=amd64` when building your application. Rosetta 2 can automatically translate x86 binaries to arm64 binaries and run x86 applications on Mac M1.
- Developing on **Linux arm64**: You can install qemu and use the `qemu-x86_64 -cpu max` command to convert x86 binaries to amr64 binaries for applications built with sonic. The qemu can achieve a similar transfer effect to Rosetta 2 on Mac M1.
For developers who want to use sonic on Linux arm64 without qemu, or those who want to handle JSON strictly consistent with `encoding/json`, we provide some compatible APIs as `sonic.API`
- `ConfigDefault`: the sonic's default config (`EscapeHTML=false`,`SortKeys=false`...) to run on sonic-supporting environment. It will fall back to `encoding/json` with the corresponding config, and some options like `SortKeys=false` will be invalid.
- `ConfigStd`: the std-compatible config (`EscapeHTML=true`,`SortKeys=true`...) to run on sonic-supporting environment. It will fall back to `encoding/json`.
- `ConfigFastest`: the fastest config (`NoQuoteTextMarshaler=true`) to run on sonic-supporting environment. It will fall back to `encoding/json` with the corresponding config, and some options will be invalid.
## Tips
@@ -479,23 +466,6 @@ For better performance, in previous case the `ast.Visitor` will be the better ch
But `ast.Visitor` is not a very handy API. You might need to write a lot of code to implement your visitor and carefully maintain the tree hierarchy during decoding. Please read the comments in [ast/visitor.go](https://github.com/bytedance/sonic/blob/main/ast/visitor.go) carefully if you decide to use this API.
### Buffer Size
Sonic use memory pool in many places like `encoder.Encode`, `ast.Node.MarshalJSON` to improve performance, which may produce more memory usage (in-use) when server's load is high. See [issue 614](https://github.com/bytedance/sonic/issues/614). Therefore, we introduce some options to let user control the behavior of memory pool. See [option](https://pkg.go.dev/github.com/bytedance/sonic@v1.11.9/option#pkg-variables) package.
### Faster JSON Skip
For security, sonic use [FSM](native/skip_one.c) algorithm to validate JSON when decoding raw JSON or encoding `json.Marshaler`, which is much slower (1~10x) than [SIMD-searching-pair](native/skip_one_fast.c) algorithm. If user has many redundant JSON value and DO NOT NEED to strictly validate JSON correctness, you can enable below options:
- `Config.NoValidateSkipJSON`: for faster skipping JSON when decoding, such as unknown fields, json.Unmarshaler(json.RawMessage), mismatched values, and redundant array elements
- `Config.NoValidateJSONMarshaler`: avoid validating JSON when encoding `json.Marshaler`
- `SearchOption.ValidateJSON`: indicates if validate located JSON value when `Get`
## JSON-Path Support (GJSON)
[tidwall/gjson](https://github.com/tidwall/gjson) has provided a comprehensive and popular JSON-Path API, and
a lot of older codes heavily relies on it. Therefore, we provides a wrapper library, which combines gjson's API with sonic's SIMD algorithm to boost up the performance. See [cloudwego/gjson](https://github.com/cloudwego/gjson).
## Community
Sonic is a subproject of [CloudWeGo](https://www.cloudwego.io/). We are committed to building a cloud native ecosystem.

51
vendor/github.com/bytedance/sonic/README_ZH_CN.md generated vendored
View File

@@ -6,10 +6,9 @@
## 依赖
- Go: 1.18~1.25
- 注意：Go1.24.0 由于 [issue](https://github.com/golang/go/issues/71672) 不可用，请升级到更高 Go 版本，或添加编译选项 `--ldflags="-checklinkname=0"`
- OS: Linux / MacOS / Windows
- CPU: AMD64 / (ARM64, 需要 Go1.20 以上）
- Go 1.16~1.22
- Linux / MacOS / Windows（需要 Go1.17 以上）
- Amd64 架构
## 接口
@@ -261,7 +260,7 @@ fmt.Printf("%+v", data) // {A:0 B:1}
### `Ast.Node`
Sonic/ast.Node 是完全独立的 JSON 抽象语法树库。它实现了序列化和反序列化，并提供了获取和修改JSON数据的鲁棒的 API。
Sonic/ast.Node 是完全独立的 JSON 抽象语法树库。它实现了序列化和反序列化，并提供了获取和修改通用数据的鲁棒的 API。
#### 查找/索引
@@ -283,22 +282,6 @@ sub := root.Get("key3").Index(2).Int64() // == 3
**注意**：由于 `Index()` 使用偏移量来定位数据，比使用扫描的 `Get()` 要快的多，建议尽可能的使用 `Index` 。 Sonic 也提供了另一个 API， `IndexOrGet()` ，以偏移量为基础并且也确保键的匹配。
#### 查找选项
`ast.Searcher`提供了一些选项，以满足用户的不同需求:
```go
opts := ast.SearchOption{CopyReturn: true…}
val, err := sonic.GetWithOptions(JSON, opts, "key")
```
- CopyReturn
指示搜索器复制结果JSON字符串，而不是从输入引用。如果用户缓存结果，这有助于减少内存使用
- ConcurentRead
因为`ast.Node`使用`Lazy-Load`设计，默认不支持并发读取。如果您想同时读取，请指定它。
- ValidateJSON
指示搜索器来验证整个JSON。默认情况下启用该选项, 但是对于查找速度有一定影响。
#### 修改
使用 `Set()` / `Unset()` 修改 json 的内容
@@ -385,12 +368,16 @@ type Visitor interface {
## 兼容性
对于想要使用sonic来满足不同场景的开发人员，我们提供了一些集成配置:
由于开发高性能代码的困难性， Sonic **不**保证对所有环境的支持。对于在不同环境中使用 Sonic 构建应用程序的开发者，我们有以下建议：
- `ConfigDefault`: sonic的默认配置 (`EscapeHTML=false`， `SortKeys=false`…) 保证性能同时兼顾安全性。
- `ConfigStd`: 与 `encoding/json` 保证完全兼容的配置
- `ConfigFastest`: 最快的配置(`NoQuoteTextMarshaler=true...`) 保证性能最优但是会缺少一些安全性检查（validate UTF8 等）
Sonic **不**确保支持所有环境，由于开发高性能代码的困难。在不支持sonic的环境中，实现将回落到 `encoding/json`。因此上述配置将全部等于`ConfigStd`。
- 在 **Mac M1** 上开发：确保在您的计算机上安装了 Rosetta 2，并在构建时设置 `GOARCH=amd64` 。 Rosetta 2 可以自动将 x86 二进制文件转换为 arm64 二进制文件，并在 Mac M1 上运行 x86 应用程序。
- 在 **Linux arm64** 上开发：您可以安装 qemu 并使用 `qemu-x86_64 -cpu max` 命令来将 x86 二进制文件转换为 arm64 二进制文件。qemu可以实现与Mac M1上的Rosetta 2类似的转换效果。
对于希望在不使用 qemu 下使用 sonic 的开发者，或者希望处理 JSON 时与 `encoding/JSON` 严格保持一致的开发者，我们在 `sonic.API` 中提供了一些兼容性 API
- `ConfigDefault`: 在支持 sonic 的环境下 sonic 的默认配置（`EscapeHTML=false`，`SortKeys=false`等）。行为与具有相应配置的 `encoding/json` 一致，一些选项，如 `SortKeys=false` 将无效。
- `ConfigStd`: 在支持 sonic 的环境下与标准库兼容的配置（`EscapeHTML=true`，`SortKeys=true`等）。行为与 `encoding/json` 一致。
- `ConfigFastest`: 在支持 sonic 的环境下运行最快的配置（`NoQuoteTextMarshaler=true`）。行为与具有相应配置的 `encoding/json` 一致，某些选项将无效。
## 注意事项
@@ -477,18 +464,6 @@ go someFunc(user)
但是，`ast.Visitor` 并不是一个很易用的 API。你可能需要写大量的代码去实现自己的 `ast.Visitor`，并且需要在解析过程中仔细维护树的层级。如果你决定要使用这个 API，请先仔细阅读 [ast/visitor.go](https://github.com/bytedance/sonic/blob/main/ast/visitor.go) 中的注释。
### 缓冲区大小
Sonic在许多地方使用内存池，如`encoder.Encode`, `ast.Node.MarshalJSON`等来提高性能，这可能会在服务器负载高时产生更多的内存使用(in-use)。参见[issue 614](https://github.com/bytedance/sonic/issues/614)。因此，我们引入了一些选项来让用户配置内存池的行为。参见[option](https://pkg.go.dev/github.com/bytedance/sonic@v1.11.9/option#pkg-variables)包。
### 更快的 JSON Skip
为了安全起见，在跳过原始JSON 时，sonic decoder 默认使用[FSM](native/skip_one.c)算法扫描来跳过同时校验 JSON。它相比[SIMD-searching-pair](native/skip_one_fast.c)算法跳过要慢得多(1~10倍)。如果用户有很多冗余的JSON值，并且不需要严格验证JSON的正确性，你可以启用以下选项:
- `Config.NoValidateSkipJSON`: 用于在解码时更快地跳过JSON，例如未知字段，`json.RawMessage`，不匹配的值和冗余的数组元素等
- `Config.NoValidateJSONMarshaler`: 编码JSON时避免验证JSON。封送拆收器
- `SearchOption.ValidateJSON`: 指示当`Get`时是否验证定位的JSON值
## 社区
Sonic 是 [CloudWeGo](https://www.cloudwego.io/) 下的一个子项目。我们致力于构建云原生生态系统。

41
vendor/github.com/bytedance/sonic/api.go generated vendored
View File

@@ -23,16 +23,6 @@ import (
`github.com/bytedance/sonic/internal/rt`
)
const (
// UseStdJSON indicates you are using fallback implementation (encoding/json)
UseStdJSON = iota
// UseSonicJSON indicates you are using real sonic implementation
UseSonicJSON
)
// APIKind is the kind of API, 0 is std json, 1 is sonic.
const APIKind = apiKind
// Config is a combination of sonic/encoder.Options and sonic/decoder.Options
type Config struct {
// EscapeHTML indicates encoder to escape all HTML characters
@@ -77,26 +67,16 @@ type Config struct {
// CopyString indicates decoder to decode string values by copying instead of referring.
CopyString bool
// ValidateString indicates decoder and encoder to validate string values: decoder will return errors
// ValidateString indicates decoder and encoder to valid string values: decoder will return errors
// when unescaped control chars(\u0000-\u001f) in the string value of JSON.
ValidateString bool
// NoValidateJSONMarshaler indicates that the encoder should not validate the output string
// after encoding the JSONMarshaler to JSON.
NoValidateJSONMarshaler bool
// NoValidateJSONSkip indicates the decoder should not validate the JSON value when skipping it,
// such as unknown-fields, mismatched-type, redundant elements..
NoValidateJSONSkip bool
// NoEncoderNewline indicates that the encoder should not add a newline after every message
NoEncoderNewline bool
// Encode Infinity or Nan float into `null`, instead of returning an error.
EncodeNullForInfOrNan bool
// CaseSensitive indicates that the decoder should not ignore the case of object keys.
CaseSensitive bool
}
var (
@@ -114,15 +94,15 @@ var (
// ConfigFastest is the fastest config of APIs, aiming at speed.
ConfigFastest = Config{
NoQuoteTextMarshaler: true,
NoValidateJSONMarshaler: true,
NoValidateJSONSkip: true,
}.Froze()
)
// API is a binding of specific config.
// This interface is inspired by github.com/json-iterator/go,
// and has same behaviors under equivalent config.
// and has same behaviors under equavilent config.
type API interface {
// MarshalToString returns the JSON encoding string of v
MarshalToString(v interface{}) (string, error)
@@ -177,13 +157,6 @@ func Marshal(val interface{}) ([]byte, error) {
return ConfigDefault.Marshal(val)
}
// MarshalIndent is like Marshal but applies Indent to format the output.
// Each JSON element in the output will begin on a new line beginning with prefix
// followed by one or more copies of indent according to the indentation nesting.
func MarshalIndent(v interface{}, prefix, indent string) ([]byte, error) {
return ConfigDefault.MarshalIndent(v, prefix, indent)
}
// MarshalString returns the JSON encoding string of v.
func MarshalString(val interface{}) (string, error) {
return ConfigDefault.MarshalToString(val)
@@ -216,14 +189,6 @@ func Get(src []byte, path ...interface{}) (ast.Node, error) {
return GetCopyFromString(rt.Mem2Str(src), path...)
}
//GetWithOptions searches and locates the given path from src json,
// with specific options of ast.Searcher
func GetWithOptions(src []byte, opts ast.SearchOptions, path ...interface{}) (ast.Node, error) {
s := ast.NewSearcher(rt.Mem2Str(src))
s.SearchOptions = opts
return s.GetByPath(path...)
}
// GetFromString is same with Get except src is string.
//
// WARNING: The returned JSON is **Referenced** from the input.

17
vendor/github.com/bytedance/sonic/ast/api_compat.go generated vendored
View File

@@ -1,4 +1,4 @@
// +build !amd64,!arm64 go1.26 !go1.17 arm64,!go1.20
// +build !amd64,!arm64 go1.23 !go1.16 arm64,!go1.20
/*
* Copyright 2022 ByteDance Inc.
@@ -23,17 +23,28 @@ import (
`unicode/utf8`
`github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/internal/compat`
`github.com/bytedance/sonic/internal/rt`
)
func init() {
compat.Warn("sonic/ast")
println("WARNING:(ast) sonic only supports Go1.16~1.22, but your environment is not suitable")
}
func quote(buf *[]byte, val string) {
quoteString(buf, val)
}
// unquote unescapes a internal JSON string (it doesn't count quotas at the begining and end)
func unquote(src string) (string, types.ParsingError) {
sp := rt.IndexChar(src, -1)
out, ok := unquoteBytes(rt.BytesFrom(sp, len(src)+2, len(src)+2))
if !ok {
return "", types.ERR_INVALID_ESCAPE
}
return rt.Mem2Str(out), 0
}
func (self *Parser) decodeValue() (val types.JsonState) {
e, v := decodeValue(self.s, self.p, self.dbuf == nil)
if e < 0 {

97
vendor/github.com/bytedance/sonic/ast/decode.go generated vendored
View File

@@ -17,35 +17,36 @@
package ast
import (
"encoding/base64"
"runtime"
"strconv"
"unsafe"
`encoding/base64`
`runtime`
`strconv`
`unsafe`
"github.com/bytedance/sonic/internal/native/types"
"github.com/bytedance/sonic/internal/rt"
"github.com/bytedance/sonic/internal/utils"
"github.com/bytedance/sonic/unquote"
`github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/internal/rt`
)
var bytesNull = []byte("null")
const _blankCharsMask = (1 << ' ') | (1 << '\t') | (1 << '\r') | (1 << '\n')
const (
strNull = "null"
bytesNull = "null"
bytesTrue = "true"
bytesFalse = "false"
bytesObject = "{}"
bytesArray = "[]"
)
func isSpace(c byte) bool {
return (int(1<<c) & _blankCharsMask) != 0
}
//go:nocheckptr
func skipBlank(src string, pos int) int {
se := uintptr(rt.IndexChar(src, len(src)))
sp := uintptr(rt.IndexChar(src, pos))
for sp < se {
if !utils.IsSpace(*(*byte)(unsafe.Pointer(sp))) {
if !isSpace(*(*byte)(unsafe.Pointer(sp))) {
break
}
sp += 1
@@ -62,7 +63,7 @@ func decodeNull(src string, pos int) (ret int) {
if ret > len(src) {
return -int(types.ERR_EOF)
}
if src[pos:ret] == strNull {
if src[pos:ret] == bytesNull {
return ret
} else {
return -int(types.ERR_INVALID_CHAR)
@@ -102,13 +103,13 @@ func decodeString(src string, pos int) (ret int, v string) {
return ret, v
}
result, err := unquote.String(src[pos:ret])
if err != 0 {
vv, ok := unquoteBytes(rt.Str2Mem(src[pos:ret]))
if !ok {
return -int(types.ERR_INVALID_CHAR), ""
}
runtime.KeepAlive(src)
return ret, result
return ret, rt.Mem2Str(vv)
}
func decodeBinary(src string, pos int) (ret int, v []byte) {
@@ -286,7 +287,67 @@ func decodeValue(src string, pos int, skipnum bool) (ret int, v types.JsonState)
//go:nocheckptr
func skipNumber(src string, pos int) (ret int) {
return utils.SkipNumber(src, pos)
sp := uintptr(rt.IndexChar(src, pos))
se := uintptr(rt.IndexChar(src, len(src)))
if uintptr(sp) >= se {
return -int(types.ERR_EOF)
}
if c := *(*byte)(unsafe.Pointer(sp)); c == '-' {
sp += 1
}
ss := sp
var pointer bool
var exponent bool
var lastIsDigit bool
var nextNeedDigit = true
for ; sp < se; sp += uintptr(1) {
c := *(*byte)(unsafe.Pointer(sp))
if isDigit(c) {
lastIsDigit = true
nextNeedDigit = false
continue
} else if nextNeedDigit {
return -int(types.ERR_INVALID_CHAR)
} else if c == '.' {
if !lastIsDigit || pointer || exponent || sp == ss {
return -int(types.ERR_INVALID_CHAR)
}
pointer = true
lastIsDigit = false
nextNeedDigit = true
continue
} else if c == 'e' || c == 'E' {
if !lastIsDigit || exponent {
return -int(types.ERR_INVALID_CHAR)
}
if sp == se-1 {
return -int(types.ERR_EOF)
}
exponent = true
lastIsDigit = false
nextNeedDigit = false
continue
} else if c == '-' || c == '+' {
if prev := *(*byte)(unsafe.Pointer(sp - 1)); prev != 'e' && prev != 'E' {
return -int(types.ERR_INVALID_CHAR)
}
lastIsDigit = false
nextNeedDigit = true
continue
} else {
break
}
}
if nextNeedDigit {
return -int(types.ERR_EOF)
}
runtime.KeepAlive(src)
return int(uintptr(sp) - uintptr((*rt.GoString)(unsafe.Pointer(&src)).Ptr))
}
//go:nocheckptr
@@ -544,7 +605,7 @@ func _DecodeString(src string, pos int, needEsc bool, validStr bool) (v string,
return str, p.p, true
}
/* unquote the string */
out, err := unquote.String(str)
out, err := unquote(str)
/* check for errors */
if err != 0 {
return "", -int(err), true

61
vendor/github.com/bytedance/sonic/ast/encode.go generated vendored
View File

@@ -17,12 +17,12 @@
package ast
import (
"sync"
"unicode/utf8"
`sync`
`unicode/utf8`
)
"github.com/bytedance/gopkg/lang/dirtmake"
"github.com/bytedance/sonic/internal/rt"
"github.com/bytedance/sonic/option"
const (
_MaxBuffer = 1024 // 1KB buffer size
)
func quoteString(e *[]byte, s string) {
@@ -30,7 +30,7 @@ func quoteString(e *[]byte, s string) {
start := 0
for i := 0; i < len(s); {
if b := s[i]; b < utf8.RuneSelf {
if rt.SafeSet[b] {
if safeSet[b] {
i++
continue
}
@@ -54,8 +54,8 @@ func quoteString(e *[]byte, s string) {
// user-controlled strings are rendered into JSON
// and served to some browsers.
*e = append(*e, `u00`...)
*e = append(*e, rt.Hex[b>>4])
*e = append(*e, rt.Hex[b&0xF])
*e = append(*e, hex[b>>4])
*e = append(*e, hex[b&0xF])
}
i++
start = i
@@ -76,7 +76,7 @@ func quoteString(e *[]byte, s string) {
*e = append(*e, s[start:i]...)
}
*e = append(*e, `\u202`...)
*e = append(*e, rt.Hex[c&0xF])
*e = append(*e, hex[c&0xF])
i += size
start = i
continue
@@ -92,29 +92,16 @@ func quoteString(e *[]byte, s string) {
var bytesPool = sync.Pool{}
func (self *Node) MarshalJSON() ([]byte, error) {
if self == nil {
return bytesNull, nil
}
// fast path for raw node
if self.isRaw() {
return rt.Str2Mem(self.toString()), nil
}
buf := newBuffer()
err := self.encode(buf)
if err != nil {
freeBuffer(buf)
return nil, err
}
var ret []byte
if !rt.CanSizeResue(cap(*buf)) {
ret = *buf
} else {
ret = dirtmake.Bytes(len(*buf), len(*buf))
copy(ret, *buf)
freeBuffer(buf)
}
ret := make([]byte, len(*buf))
copy(ret, *buf)
freeBuffer(buf)
return ret, err
}
@@ -122,24 +109,21 @@ func newBuffer() *[]byte {
if ret := bytesPool.Get(); ret != nil {
return ret.(*[]byte)
} else {
buf := make([]byte, 0, option.DefaultAstBufferSize)
buf := make([]byte, 0, _MaxBuffer)
return &buf
}
}
func freeBuffer(buf *[]byte) {
if !rt.CanSizeResue(cap(*buf)) {
return
}
*buf = (*buf)[:0]
bytesPool.Put(buf)
}
func (self *Node) encode(buf *[]byte) error {
if self.isRaw() {
if self.IsRaw() {
return self.encodeRaw(buf)
}
switch int(self.itype()) {
switch self.Type() {
case V_NONE : return ErrNotExist
case V_ERROR : return self.Check()
case V_NULL : return self.encodeNull(buf)
@@ -155,21 +139,16 @@ func (self *Node) encode(buf *[]byte) error {
}
func (self *Node) encodeRaw(buf *[]byte) error {
lock := self.rlock()
if !self.isRaw() {
self.runlock()
return self.encode(buf)
}
raw := self.toString()
if lock {
self.runlock()
raw, err := self.Raw()
if err != nil {
return err
}
*buf = append(*buf, raw...)
return nil
}
func (self *Node) encodeNull(buf *[]byte) error {
*buf = append(*buf, strNull...)
*buf = append(*buf, bytesNull...)
return nil
}

6
vendor/github.com/bytedance/sonic/ast/error.go generated vendored
View File

@@ -17,10 +17,6 @@ func newError(err types.ParsingError, msg string) *Node {
}
}
func newErrorPair(err SyntaxError) *Pair {
return &Pair{0, "", *newSyntaxError(err)}
}
// Error returns error message if the node is invalid
func (self Node) Error() string {
if self.t != V_ERROR {
@@ -83,7 +79,7 @@ func (self SyntaxError) description() string {
/* check for empty source */
if self.Src == "" {
return fmt.Sprintf("no sources available, the input json is empty: %#v", self)
return fmt.Sprintf("no sources available: %#v", self)
}
/* prevent slicing before the beginning */

25
vendor/github.com/bytedance/sonic/ast/iterator.go generated vendored
View File

@@ -17,29 +17,19 @@
package ast
import (
"fmt"
`fmt`
"github.com/bytedance/sonic/internal/caching"
"github.com/bytedance/sonic/internal/native/types"
`github.com/bytedance/sonic/internal/native/types`
)
type Pair struct {
hash uint64
Key string
Value Node
}
func NewPair(key string, val Node) Pair {
return Pair{
hash: caching.StrHash(key),
Key: key,
Value: val,
}
}
// Values returns iterator for array's children traversal
func (self *Node) Values() (ListIterator, error) {
if err := self.should(types.V_ARRAY); err != nil {
if err := self.should(types.V_ARRAY, "an array"); err != nil {
return ListIterator{}, err
}
return self.values(), nil
@@ -51,7 +41,7 @@ func (self *Node) values() ListIterator {
// Properties returns iterator for object's children traversal
func (self *Node) Properties() (ObjectIterator, error) {
if err := self.should(types.V_OBJECT); err != nil {
if err := self.should(types.V_OBJECT, "an object"); err != nil {
return ObjectIterator{}, err
}
return self.properties(), nil
@@ -173,14 +163,11 @@ type Scanner func(path Sequence, node *Node) bool
// ForEach scans one V_OBJECT node's children from JSON head to tail,
// and pass the Sequence and Node of corresponding JSON value.
//
// Especially, if the node is not V_ARRAY or V_OBJECT,
// Especailly, if the node is not V_ARRAY or V_OBJECT,
// the node itself will be returned and Sequence.Index == -1.
//
// NOTICE: An unset node WON'T trigger sc, but its index still counts into Path.Index
// NOTICE: A unsetted node WON'T trigger sc, but its index still counts into Path.Index
func (self *Node) ForEach(sc Scanner) error {
if err := self.checkRaw(); err != nil {
return err
}
switch self.itype() {
case types.V_ARRAY:
iter, err := self.Values()

354
vendor/github.com/bytedance/sonic/ast/node.go generated vendored
View File

@@ -17,15 +17,13 @@
package ast
import (
"encoding/json"
"fmt"
"strconv"
"sync"
"sync/atomic"
"unsafe"
"github.com/bytedance/sonic/internal/native/types"
"github.com/bytedance/sonic/internal/rt"
`encoding/json`
`fmt`
`strconv`
`unsafe`
`github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/internal/rt`
)
const (
@@ -38,7 +36,7 @@ const (
_V_ARRAY_LAZY = _V_LAZY | types.V_ARRAY
_V_OBJECT_LAZY = _V_LAZY | types.V_OBJECT
_MASK_LAZY = _V_LAZY - 1
_MASK_RAW = _V_RAW - 1
_MASK_RAW = _V_RAW - 1
)
const (
@@ -58,20 +56,19 @@ type Node struct {
t types.ValueType
l uint
p unsafe.Pointer
m *sync.RWMutex
}
// UnmarshalJSON is just an adapter to json.Unmarshaler.
// If you want better performance, use Searcher.GetByPath() directly
func (self *Node) UnmarshalJSON(data []byte) (err error) {
*self = newRawNode(rt.Mem2Str(data), switchRawType(data[0]), false)
return nil
*self = NewRaw(string(data))
return self.Check()
}
/** Node Type Accessor **/
// Type returns json type represented by the node
// It will be one of bellows:
// It will be one of belows:
// V_NONE = 0 (empty node, key not exists)
// V_ERROR = 1 (error node)
// V_NULL = 2 (json value `null`, key exists)
@@ -82,39 +79,17 @@ func (self *Node) UnmarshalJSON(data []byte) (err error) {
// V_STRING = 7 (json value string)
// V_NUMBER = 33 (json value number )
// V_ANY = 34 (golang interface{})
//
// Deprecated: not concurrent safe. Use TypeSafe instead
func (self Node) Type() int {
return int(self.t & _MASK_LAZY & _MASK_RAW)
}
// Type concurrently-safe returns json type represented by the node
// It will be one of bellows:
// V_NONE = 0 (empty node, key not exists)
// V_ERROR = 1 (error node)
// V_NULL = 2 (json value `null`, key exists)
// V_TRUE = 3 (json value `true`)
// V_FALSE = 4 (json value `false`)
// V_ARRAY = 5 (json value array)
// V_OBJECT = 6 (json value object)
// V_STRING = 7 (json value string)
// V_NUMBER = 33 (json value number )
// V_ANY = 34 (golang interface{})
func (self *Node) TypeSafe() int {
return int(self.loadt() & _MASK_LAZY & _MASK_RAW)
}
func (self *Node) itype() types.ValueType {
func (self Node) itype() types.ValueType {
return self.t & _MASK_LAZY & _MASK_RAW
}
// Exists returns false only if the self is nil or empty node V_NONE
func (self *Node) Exists() bool {
if self == nil {
return false
}
t := self.loadt()
return t != V_ERROR && t != _V_NONE
return self.Valid() && self.t != _V_NONE
}
// Valid reports if self is NOT V_ERROR or nil
@@ -122,7 +97,7 @@ func (self *Node) Valid() bool {
if self == nil {
return false
}
return self.loadt() != V_ERROR
return self.t != V_ERROR
}
// Check checks if the node itself is valid, and return:
@@ -131,31 +106,24 @@ func (self *Node) Valid() bool {
func (self *Node) Check() error {
if self == nil {
return ErrNotExist
} else if self.loadt() != V_ERROR {
} else if self.t != V_ERROR {
return nil
} else {
return self
}
}
// isRaw returns true if node's underlying value is raw json
//
// Deprecated: not concurrent safe
func (self Node) IsRaw() bool {
return self.t & _V_RAW != 0
}
// IsRaw returns true if node's underlying value is raw json
func (self *Node) isRaw() bool {
return self.loadt() & _V_RAW != 0
func (self Node) IsRaw() bool {
return self.t&_V_RAW != 0
}
func (self *Node) isLazy() bool {
return self != nil && self.t & _V_LAZY != 0
return self != nil && self.t&_V_LAZY != 0
}
func (self *Node) isAny() bool {
return self != nil && self.loadt() == _V_ANY
return self != nil && self.t == _V_ANY
}
/** Simple Value Methods **/
@@ -165,26 +133,18 @@ func (self *Node) Raw() (string, error) {
if self == nil {
return "", ErrNotExist
}
lock := self.rlock()
if !self.isRaw() {
if lock {
self.runlock()
}
if !self.IsRaw() {
buf, err := self.MarshalJSON()
return rt.Mem2Str(buf), err
}
ret := self.toString()
if lock {
self.runlock()
}
return ret, nil
return self.toString(), nil
}
func (self *Node) checkRaw() error {
if err := self.Check(); err != nil {
return err
}
if self.isRaw() {
if self.IsRaw() {
self.parseRaw(false)
}
return self.Check()
@@ -440,7 +400,7 @@ func (self *Node) String() (string, error) {
}
}
// StrictString returns string value (unescaped), including V_STRING, V_ANY of string.
// StrictString returns string value (unescaped), includeing V_STRING, V_ANY of string.
// In other cases, it will return empty string.
func (self *Node) StrictString() (string, error) {
if err := self.checkRaw(); err != nil {
@@ -509,24 +469,7 @@ func (self *Node) Float64() (float64, error) {
}
}
func (self *Node) StrictBool() (bool, error) {
if err := self.checkRaw(); err!= nil {
return false, err
}
switch self.t {
case types.V_TRUE : return true, nil
case types.V_FALSE : return false, nil
case _V_ANY :
any := self.packAny()
switch v := any.(type) {
case bool : return v, nil
default : return false, ErrUnsupportType
}
default : return false, ErrUnsupportType
}
}
// Float64 exports underlying float64 value, including V_NUMBER, V_ANY
// Float64 exports underlying float64 value, includeing V_NUMBER, V_ANY
func (self *Node) StrictFloat64() (float64, error) {
if err := self.checkRaw(); err != nil {
return 0.0, err
@@ -544,7 +487,7 @@ func (self *Node) StrictFloat64() (float64, error) {
}
}
/** Sequential Value Methods **/
/** Sequencial Value Methods **/
// Len returns children count of a array|object|string node
// WARN: For partially loaded node, it also works but only counts the parsed children
@@ -561,7 +504,7 @@ func (self *Node) Len() (int, error) {
}
}
func (self *Node) len() int {
func (self Node) len() int {
return int(self.l)
}
@@ -584,7 +527,7 @@ func (self *Node) Cap() (int, error) {
//
// If self is V_NONE or V_NULL, it becomes V_OBJECT and sets the node at the key.
func (self *Node) Set(key string, node Node) (bool, error) {
if err := self.checkRaw(); err != nil {
if err := self.Check(); err != nil {
return false, err
}
if err := node.Check(); err != nil {
@@ -592,7 +535,7 @@ func (self *Node) Set(key string, node Node) (bool, error) {
}
if self.t == _V_NONE || self.t == types.V_NULL {
*self = NewObject([]Pair{NewPair(key, node)})
*self = NewObject([]Pair{{key, node}})
return false, nil
} else if self.itype() != types.V_OBJECT {
return false, ErrUnsupportType
@@ -606,7 +549,7 @@ func (self *Node) Set(key string, node Node) (bool, error) {
*self = newObject(new(linkedPairs))
}
s := (*linkedPairs)(self.p)
s.Push(NewPair(key, node))
s.Push(Pair{key, node})
self.l++
return false, nil
@@ -625,10 +568,10 @@ func (self *Node) SetAny(key string, val interface{}) (bool, error) {
// Unset REMOVE (soft) the node of given key under object parent, and reports if the key has existed.
func (self *Node) Unset(key string) (bool, error) {
if err := self.should(types.V_OBJECT); err != nil {
if err := self.should(types.V_OBJECT, "an object"); err != nil {
return false, err
}
// NOTICE: must get accurate length before deduct
// NOTICE: must get acurate length before deduct
if err := self.skipAllKey(); err != nil {
return false, err
}
@@ -646,7 +589,7 @@ func (self *Node) Unset(key string) (bool, error) {
//
// The index must be within self's children.
func (self *Node) SetByIndex(index int, node Node) (bool, error) {
if err := self.checkRaw(); err != nil {
if err := self.Check(); err != nil {
return false, err
}
if err := node.Check(); err != nil {
@@ -674,7 +617,7 @@ func (self *Node) SetAnyByIndex(index int, val interface{}) (bool, error) {
return self.SetByIndex(index, NewAny(val))
}
// UnsetByIndex REMOVE (softly) the node of given index.
// UnsetByIndex REOMVE (softly) the node of given index.
//
// WARN: this will change address of elements, which is a dangerous action.
// Use Unset() for object or Pop() for array instead.
@@ -726,7 +669,7 @@ func (self *Node) UnsetByIndex(index int) (bool, error) {
//
// If self is V_NONE or V_NULL, it becomes V_ARRAY and sets the node at index 0.
func (self *Node) Add(node Node) error {
if err := self.checkRaw(); err != nil {
if err := self.Check(); err != nil {
return err
}
@@ -734,7 +677,7 @@ func (self *Node) Add(node Node) error {
*self = NewArray([]Node{node})
return nil
}
if err := self.should(types.V_ARRAY); err != nil {
if err := self.should(types.V_ARRAY, "an array"); err != nil {
return err
}
@@ -793,11 +736,11 @@ func (self *Node) Pop() error {
}
// Move moves the child at src index to dst index,
// meanwhile slides siblings from src+1 to dst.
// meanwhile slides sliblings from src+1 to dst.
//
// WARN: this will change address of elements, which is a dangerous action.
func (self *Node) Move(dst, src int) error {
if err := self.should(types.V_ARRAY); err != nil {
if err := self.should(types.V_ARRAY, "an array"); err != nil {
return err
}
@@ -833,7 +776,7 @@ func (self *Node) Move(dst, src int) error {
return nil
}
// AddAny wraps val with V_ANY node, and Add() the node.
// SetAny wraps val with V_ANY node, and Add() the node.
func (self *Node) AddAny(val interface{}) error {
return self.Add(NewAny(val))
}
@@ -869,7 +812,7 @@ func (self *Node) GetByPath(path ...interface{}) *Node {
// Get loads given key of an object node on demands
func (self *Node) Get(key string) *Node {
if err := self.should(types.V_OBJECT); err != nil {
if err := self.should(types.V_OBJECT, "an object"); err != nil {
return unwrapError(err)
}
n, _ := self.skipKey(key)
@@ -902,14 +845,14 @@ func (self *Node) Index(idx int) *Node {
// IndexPair indexies pair at given idx,
// node type MUST be either V_OBJECT
func (self *Node) IndexPair(idx int) *Pair {
if err := self.should(types.V_OBJECT); err != nil {
if err := self.should(types.V_OBJECT, "an object"); err != nil {
return nil
}
return self.skipIndexPair(idx)
}
func (self *Node) indexOrGet(idx int, key string) (*Node, int) {
if err := self.should(types.V_OBJECT); err != nil {
if err := self.should(types.V_OBJECT, "an object"); err != nil {
return unwrapError(err), idx
}
@@ -946,16 +889,16 @@ func (self *Node) Map() (map[string]interface{}, error) {
return nil, ErrUnsupportType
}
}
if err := self.should(types.V_OBJECT); err != nil {
if err := self.should(types.V_OBJECT, "an object"); err != nil {
return nil, err
}
if err := self.loadAllKey(false); err != nil {
if err := self.loadAllKey(); err != nil {
return nil, err
}
return self.toGenericObject()
}
// MapUseNumber loads all keys of an object node, with numeric nodes cast to json.Number
// MapUseNumber loads all keys of an object node, with numeric nodes casted to json.Number
func (self *Node) MapUseNumber() (map[string]interface{}, error) {
if self.isAny() {
any := self.packAny()
@@ -965,16 +908,16 @@ func (self *Node) MapUseNumber() (map[string]interface{}, error) {
return nil, ErrUnsupportType
}
}
if err := self.should(types.V_OBJECT); err != nil {
if err := self.should(types.V_OBJECT, "an object"); err != nil {
return nil, err
}
if err := self.loadAllKey(false); err != nil {
if err := self.loadAllKey(); err != nil {
return nil, err
}
return self.toGenericObjectUseNumber()
}
// MapUseNode scans both parsed and non-parsed children nodes,
// MapUseNode scans both parsed and non-parsed chidren nodes,
// and map them by their keys
func (self *Node) MapUseNode() (map[string]Node, error) {
if self.isAny() {
@@ -985,7 +928,7 @@ func (self *Node) MapUseNode() (map[string]Node, error) {
return nil, ErrUnsupportType
}
}
if err := self.should(types.V_OBJECT); err != nil {
if err := self.should(types.V_OBJECT, "an object"); err != nil {
return nil, err
}
if err := self.skipAllKey(); err != nil {
@@ -1091,16 +1034,16 @@ func (self *Node) Array() ([]interface{}, error) {
return nil, ErrUnsupportType
}
}
if err := self.should(types.V_ARRAY); err != nil {
if err := self.should(types.V_ARRAY, "an array"); err != nil {
return nil, err
}
if err := self.loadAllIndex(false); err != nil {
if err := self.loadAllIndex(); err != nil {
return nil, err
}
return self.toGenericArray()
}
// ArrayUseNumber loads all indexes of an array node, with numeric nodes cast to json.Number
// ArrayUseNumber loads all indexes of an array node, with numeric nodes casted to json.Number
func (self *Node) ArrayUseNumber() ([]interface{}, error) {
if self.isAny() {
any := self.packAny()
@@ -1110,16 +1053,16 @@ func (self *Node) ArrayUseNumber() ([]interface{}, error) {
return nil, ErrUnsupportType
}
}
if err := self.should(types.V_ARRAY); err != nil {
if err := self.should(types.V_ARRAY, "an array"); err != nil {
return nil, err
}
if err := self.loadAllIndex(false); err != nil {
if err := self.loadAllIndex(); err != nil {
return nil, err
}
return self.toGenericArrayUseNumber()
}
// ArrayUseNode copies both parsed and non-parsed children nodes,
// ArrayUseNode copys both parsed and non-parsed chidren nodes,
// and indexes them by original order
func (self *Node) ArrayUseNode() ([]Node, error) {
if self.isAny() {
@@ -1130,7 +1073,7 @@ func (self *Node) ArrayUseNode() ([]Node, error) {
return nil, ErrUnsupportType
}
}
if err := self.should(types.V_ARRAY); err != nil {
if err := self.should(types.V_ARRAY, "an array"); err != nil {
return nil, err
}
if err := self.skipAllIndex(); err != nil {
@@ -1164,9 +1107,9 @@ func (self *Node) unsafeArray() (*linkedNodes, error) {
return (*linkedNodes)(self.p), nil
}
// Interface loads all children under all paths from this node,
// Interface loads all children under all pathes from this node,
// and converts itself as generic type.
// WARN: all numeric nodes are cast to float64
// WARN: all numberic nodes are casted to float64
func (self *Node) Interface() (interface{}, error) {
if err := self.checkRaw(); err != nil {
return nil, err
@@ -1186,12 +1129,12 @@ func (self *Node) Interface() (interface{}, error) {
}
return v, nil
case _V_ARRAY_LAZY :
if err := self.loadAllIndex(false); err != nil {
if err := self.loadAllIndex(); err != nil {
return nil, err
}
return self.toGenericArray()
case _V_OBJECT_LAZY :
if err := self.loadAllKey(false); err != nil {
if err := self.loadAllKey(); err != nil {
return nil, err
}
return self.toGenericObject()
@@ -1210,7 +1153,7 @@ func (self *Node) packAny() interface{} {
}
// InterfaceUseNumber works same with Interface()
// except numeric nodes are cast to json.Number
// except numberic nodes are casted to json.Number
func (self *Node) InterfaceUseNumber() (interface{}, error) {
if err := self.checkRaw(); err != nil {
return nil, err
@@ -1225,12 +1168,12 @@ func (self *Node) InterfaceUseNumber() (interface{}, error) {
case types.V_STRING : return self.toString(), nil
case _V_NUMBER : return self.toNumber(), nil
case _V_ARRAY_LAZY :
if err := self.loadAllIndex(false); err != nil {
if err := self.loadAllIndex(); err != nil {
return nil, err
}
return self.toGenericArrayUseNumber()
case _V_OBJECT_LAZY :
if err := self.loadAllKey(false); err != nil {
if err := self.loadAllKey(); err != nil {
return nil, err
}
return self.toGenericObjectUseNumber()
@@ -1262,30 +1205,70 @@ func (self *Node) InterfaceUseNode() (interface{}, error) {
}
}
// LoadAll loads the node's children
// and ensure all its children can be READ concurrently (include its children's children)
// LoadAll loads all the node's children and children's children as parsed.
// After calling it, the node can be safely used on concurrency
func (self *Node) LoadAll() error {
return self.Load()
if self.IsRaw() {
self.parseRaw(true)
return self.Check()
}
switch self.itype() {
case types.V_ARRAY:
e := self.len()
if err := self.loadAllIndex(); err != nil {
return err
}
for i := 0; i < e; i++ {
n := self.nodeAt(i)
if n.IsRaw() {
n.parseRaw(true)
}
if err := n.Check(); err != nil {
return err
}
}
return nil
case types.V_OBJECT:
e := self.len()
if err := self.loadAllKey(); err != nil {
return err
}
for i := 0; i < e; i++ {
n := self.pairAt(i)
if n.Value.IsRaw() {
n.Value.parseRaw(true)
}
if err := n.Value.Check(); err != nil {
return err
}
}
return nil
default:
return self.Check()
}
}
// Load loads the node's children as parsed.
// and ensure all its children can be READ concurrently (include its children's children)
// After calling it, only the node itself can be used on concurrency (not include its children)
func (self *Node) Load() error {
if err := self.checkRaw(); err != nil {
return err
}
switch self.t {
case _V_ARRAY_LAZY: self.loadAllIndex(true)
case _V_OBJECT_LAZY: self.loadAllKey(true)
case V_ERROR: return self
case V_NONE: return nil
case _V_ARRAY_LAZY:
return self.skipAllIndex()
case _V_OBJECT_LAZY:
return self.skipAllKey()
default:
return self.Check()
}
if self.m == nil {
self.m = new(sync.RWMutex)
}
return self.checkRaw()
}
/**---------------------------------- Internal Helper Methods ----------------------------------**/
func (self *Node) should(t types.ValueType) error {
func (self *Node) should(t types.ValueType, s string) error {
if err := self.checkRaw(); err != nil {
return err
}
@@ -1456,17 +1439,13 @@ func (self *Node) skipIndexPair(index int) *Pair {
return nil
}
func (self *Node) loadAllIndex(loadOnce bool) error {
func (self *Node) loadAllIndex() error {
if !self.isLazy() {
return nil
}
var err types.ParsingError
parser, stack := self.getParserAndArrayStack()
if !loadOnce {
parser.noLazy = true
} else {
parser.loadOnce = true
}
parser.noLazy = true
*self, err = parser.decodeArray(&stack.v)
if err != 0 {
return parser.ExportError(err)
@@ -1474,19 +1453,14 @@ func (self *Node) loadAllIndex(loadOnce bool) error {
return nil
}
func (self *Node) loadAllKey(loadOnce bool) error {
func (self *Node) loadAllKey() error {
if !self.isLazy() {
return nil
}
var err types.ParsingError
parser, stack := self.getParserAndObjectStack()
if !loadOnce {
parser.noLazy = true
*self, err = parser.decodeObject(&stack.v)
} else {
parser.loadOnce = true
*self, err = parser.decodeObject(&stack.v)
}
parser.noLazy = true
*self, err = parser.decodeObject(&stack.v)
if err != 0 {
return parser.ExportError(err)
}
@@ -1655,23 +1629,7 @@ func NewRaw(json string) Node {
if it == _V_NONE {
return Node{}
}
return newRawNode(parser.s[start:parser.p], it, false)
}
// NewRawConcurrentRead creates a node of raw json, which can be READ
// (GetByPath/Get/Index/GetOrIndex/Int64/Bool/Float64/String/Number/Interface/Array/Map/Raw/MarshalJSON) concurrently.
// If the input json is invalid, NewRaw returns a error Node.
func NewRawConcurrentRead(json string) Node {
parser := NewParserObj(json)
start, err := parser.skip()
if err != 0 {
return *newError(err, err.Message())
}
it := switchRawType(parser.s[start])
if it == _V_NONE {
return Node{}
}
return newRawNode(parser.s[start:parser.p], it, true)
return newRawNode(parser.s[start:parser.p], it)
}
// NewAny creates a node of type V_ANY if any's type isn't Node or *Node,
@@ -1695,7 +1653,7 @@ func NewBytes(src []byte) Node {
if len(src) == 0 {
panic("empty src bytes")
}
out := rt.EncodeBase64ToString(src)
out := encodeBase64(src)
return NewString(out)
}
@@ -1731,15 +1689,15 @@ func NewNumber(v string) Node {
}
}
func (node *Node) toNumber() json.Number {
func (node Node) toNumber() json.Number {
return json.Number(rt.StrFrom(node.p, int64(node.l)))
}
func (self *Node) toString() string {
func (self Node) toString() string {
return rt.StrFrom(self.p, int64(self.l))
}
func (node *Node) toFloat64() (float64, error) {
func (node Node) toFloat64() (float64, error) {
ret, err := node.toNumber().Float64()
if err != nil {
return 0, err
@@ -1747,7 +1705,7 @@ func (node *Node) toFloat64() (float64, error) {
return ret, nil
}
func (node *Node) toInt64() (int64, error) {
func (node Node) toInt64() (int64, error) {
ret,err := node.toNumber().Int64()
if err != nil {
return 0, err
@@ -1783,8 +1741,6 @@ func NewArray(v []Node) Node {
return newArray(s)
}
const _Threshold_Index = 16
func newArray(v *linkedNodes) Node {
return Node{
t: types.V_ARRAY,
@@ -1808,9 +1764,6 @@ func NewObject(v []Pair) Node {
}
func newObject(v *linkedPairs) Node {
if v.size > _Threshold_Index {
v.BuildIndex()
}
return Node{
t: types.V_OBJECT,
l: uint(v.Len()),
@@ -1819,42 +1772,53 @@ func newObject(v *linkedPairs) Node {
}
func (self *Node) setObject(v *linkedPairs) {
if v.size > _Threshold_Index {
v.BuildIndex()
}
self.t = types.V_OBJECT
self.l = uint(v.Len())
self.p = unsafe.Pointer(v)
}
func (self *Node) parseRaw(full bool) {
lock := self.lock()
defer self.unlock()
if !self.isRaw() {
return
func newRawNode(str string, typ types.ValueType) Node {
return Node{
t: _V_RAW | typ,
p: rt.StrPtr(str),
l: uint(len(str)),
}
}
func (self *Node) parseRaw(full bool) {
raw := self.toString()
parser := NewParserObj(raw)
var e types.ParsingError
if full {
parser.noLazy = true
*self, e = parser.Parse()
} else if lock {
var n Node
parser.noLazy = true
parser.loadOnce = true
n, e = parser.Parse()
self.assign(n)
} else {
*self, e = parser.Parse()
parser.skipValue = false
}
var e types.ParsingError
*self, e = parser.Parse()
if e != 0 {
*self = *newSyntaxError(parser.syntaxError(e))
}
}
func (self *Node) assign(n Node) {
self.l = n.l
self.p = n.p
atomic.StoreInt64(&self.t, n.t)
var typeJumpTable = [256]types.ValueType{
'"' : types.V_STRING,
'-' : _V_NUMBER,
'0' : _V_NUMBER,
'1' : _V_NUMBER,
'2' : _V_NUMBER,
'3' : _V_NUMBER,
'4' : _V_NUMBER,
'5' : _V_NUMBER,
'6' : _V_NUMBER,
'7' : _V_NUMBER,
'8' : _V_NUMBER,
'9' : _V_NUMBER,
'[' : types.V_ARRAY,
'f' : types.V_FALSE,
'n' : types.V_NULL,
't' : types.V_TRUE,
'{' : types.V_OBJECT,
}
func switchRawType(c byte) types.ValueType {
return typeJumpTable[c]
}

164
vendor/github.com/bytedance/sonic/ast/parser.go generated vendored
View File

@@ -17,18 +17,14 @@
package ast
import (
"fmt"
"sync"
"sync/atomic"
`fmt`
"github.com/bytedance/sonic/internal/native/types"
"github.com/bytedance/sonic/internal/rt"
"github.com/bytedance/sonic/internal/utils"
"github.com/bytedance/sonic/unquote"
`github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/internal/rt`
)
const (
_DEFAULT_NODE_CAP int = 16
_DEFAULT_NODE_CAP int = 8
_APPEND_GROW_SHIFT = 1
)
@@ -49,7 +45,6 @@ type Parser struct {
p int
s string
noLazy bool
loadOnce bool
skipValue bool
dbuf *byte
}
@@ -65,7 +60,7 @@ func (self *Parser) delim() types.ParsingError {
return types.ERR_EOF
}
/* check for the delimiter */
/* check for the delimtier */
if self.s[p] != ':' {
return types.ERR_INVALID_CHAR
}
@@ -84,7 +79,7 @@ func (self *Parser) object() types.ParsingError {
return types.ERR_EOF
}
/* check for the delimiter */
/* check for the delimtier */
if self.s[p] != '{' {
return types.ERR_INVALID_CHAR
}
@@ -103,7 +98,7 @@ func (self *Parser) array() types.ParsingError {
return types.ERR_EOF
}
/* check for the delimiter */
/* check for the delimtier */
if self.s[p] != '[' {
return types.ERR_INVALID_CHAR
}
@@ -115,15 +110,11 @@ func (self *Parser) array() types.ParsingError {
func (self *Parser) lspace(sp int) int {
ns := len(self.s)
for ; sp<ns && utils.IsSpace(self.s[sp]); sp+=1 {}
for ; sp<ns && isSpace(self.s[sp]); sp+=1 {}
return sp
}
func (self *Parser) backward() {
for ; self.p >= 0 && utils.IsSpace(self.s[self.p]); self.p-=1 {}
}
func (self *Parser) decodeArray(ret *linkedNodes) (Node, types.ParsingError) {
sp := self.p
ns := len(self.s)
@@ -157,7 +148,7 @@ func (self *Parser) decodeArray(ret *linkedNodes) (Node, types.ParsingError) {
if t == _V_NONE {
return Node{}, types.ERR_INVALID_CHAR
}
val = newRawNode(self.s[start:self.p], t, false)
val = newRawNode(self.s[start:self.p], t)
}else{
/* decode the value */
if val, err = self.Parse(); err != 0 {
@@ -219,7 +210,7 @@ func (self *Parser) decodeObject(ret *linkedPairs) (Node, types.ParsingError) {
/* check for escape sequence */
if njs.Ep != -1 {
if key, err = unquote.String(key); err != 0 {
if key, err = unquote(key); err != 0 {
return Node{}, err
}
}
@@ -243,7 +234,7 @@ func (self *Parser) decodeObject(ret *linkedPairs) (Node, types.ParsingError) {
if t == _V_NONE {
return Node{}, types.ERR_INVALID_CHAR
}
val = newRawNode(self.s[start:self.p], t, false)
val = newRawNode(self.s[start:self.p], t)
} else {
/* decode the value */
if val, err = self.Parse(); err != 0 {
@@ -253,7 +244,7 @@ func (self *Parser) decodeObject(ret *linkedPairs) (Node, types.ParsingError) {
/* add the value to result */
// FIXME: ret's address may change here, thus previous referred node in ret may be invalid !!
ret.Push(NewPair(key, val))
ret.Push(Pair{Key: key, Value: val})
self.p = self.lspace(self.p)
/* check for EOF */
@@ -284,7 +275,7 @@ func (self *Parser) decodeString(iv int64, ep int) (Node, types.ParsingError) {
}
/* unquote the string */
out, err := unquote.String(s)
out, err := unquote(s)
/* check for errors */
if err != 0 {
@@ -300,10 +291,6 @@ func (self *Parser) Pos() int {
return self.p
}
// Parse returns a ast.Node representing the parser's JSON.
// NOTICE: the specific parsing lazy dependens parser's option
// It only parse first layer and first child for Object or Array be default
func (self *Parser) Parse() (Node, types.ParsingError) {
switch val := self.decodeValue(); val.Vt {
case types.V_EOF : return Node{}, types.ERR_EOF
@@ -312,48 +299,22 @@ func (self *Parser) Parse() (Node, types.ParsingError) {
case types.V_FALSE : return falseNode, 0
case types.V_STRING : return self.decodeString(val.Iv, val.Ep)
case types.V_ARRAY:
s := self.p - 1;
if p := skipBlank(self.s, self.p); p >= self.p && self.s[p] == ']' {
self.p = p + 1
return Node{t: types.V_ARRAY}, 0
}
if self.noLazy {
if self.loadOnce {
self.noLazy = false
}
return self.decodeArray(new(linkedNodes))
}
// NOTICE: loadOnce always keep raw json for object or array
if self.loadOnce {
self.p = s
s, e := self.skipFast()
if e != 0 {
return Node{}, e
}
return newRawNode(self.s[s:self.p], types.V_ARRAY, true), 0
}
return newLazyArray(self), 0
case types.V_OBJECT:
s := self.p - 1;
if p := skipBlank(self.s, self.p); p >= self.p && self.s[p] == '}' {
self.p = p + 1
return Node{t: types.V_OBJECT}, 0
}
// NOTICE: loadOnce always keep raw json for object or array
if self.noLazy {
if self.loadOnce {
self.noLazy = false
}
return self.decodeObject(new(linkedPairs))
}
if self.loadOnce {
self.p = s
s, e := self.skipFast()
if e != 0 {
return Node{}, e
}
return newRawNode(self.s[s:self.p], types.V_OBJECT, true), 0
}
return newLazyObject(self), 0
case types.V_DOUBLE : return NewNumber(self.s[val.Ep:self.p]), 0
case types.V_INTEGER : return NewNumber(self.s[val.Ep:self.p]), 0
@@ -394,7 +355,7 @@ func (self *Parser) searchKey(match string) types.ParsingError {
/* check for escape sequence */
if njs.Ep != -1 {
if key, err = unquote.String(key); err != 0 {
if key, err = unquote(key); err != 0 {
return err
}
}
@@ -510,7 +471,7 @@ func (self *Node) skipNextNode() *Node {
if t == _V_NONE {
return newSyntaxError(parser.syntaxError(types.ERR_INVALID_CHAR))
}
val = newRawNode(parser.s[start:parser.p], t, false)
val = newRawNode(parser.s[start:parser.p], t)
}
/* add the value to result */
@@ -549,7 +510,7 @@ func (self *Node) skipNextPair() (*Pair) {
/* check for EOF */
if parser.p = parser.lspace(sp); parser.p >= ns {
return newErrorPair(parser.syntaxError(types.ERR_EOF))
return &Pair{"", *newSyntaxError(parser.syntaxError(types.ERR_EOF))}
}
/* check for empty object */
@@ -566,7 +527,7 @@ func (self *Node) skipNextPair() (*Pair) {
/* decode the key */
if njs = parser.decodeValue(); njs.Vt != types.V_STRING {
return newErrorPair(parser.syntaxError(types.ERR_INVALID_CHAR))
return &Pair{"", *newSyntaxError(parser.syntaxError(types.ERR_INVALID_CHAR))}
}
/* extract the key */
@@ -575,35 +536,35 @@ func (self *Node) skipNextPair() (*Pair) {
/* check for escape sequence */
if njs.Ep != -1 {
if key, err = unquote.String(key); err != 0 {
return newErrorPair(parser.syntaxError(err))
if key, err = unquote(key); err != 0 {
return &Pair{key, *newSyntaxError(parser.syntaxError(err))}
}
}
/* expect a ':' delimiter */
if err = parser.delim(); err != 0 {
return newErrorPair(parser.syntaxError(err))
return &Pair{key, *newSyntaxError(parser.syntaxError(err))}
}
/* skip the value */
if start, err := parser.skipFast(); err != 0 {
return newErrorPair(parser.syntaxError(err))
return &Pair{key, *newSyntaxError(parser.syntaxError(err))}
} else {
t := switchRawType(parser.s[start])
if t == _V_NONE {
return newErrorPair(parser.syntaxError(types.ERR_INVALID_CHAR))
return &Pair{key, *newSyntaxError(parser.syntaxError(types.ERR_INVALID_CHAR))}
}
val = newRawNode(parser.s[start:parser.p], t, false)
val = newRawNode(parser.s[start:parser.p], t)
}
/* add the value to result */
ret.Push(NewPair(key, val))
ret.Push(Pair{Key: key, Value: val})
self.l++
parser.p = parser.lspace(parser.p)
/* check for EOF */
if parser.p >= ns {
return newErrorPair(parser.syntaxError(types.ERR_EOF))
return &Pair{key, *newSyntaxError(parser.syntaxError(types.ERR_EOF))}
}
/* check for the next character */
@@ -616,7 +577,7 @@ func (self *Node) skipNextPair() (*Pair) {
self.setObject(ret)
return ret.At(ret.Len()-1)
default:
return newErrorPair(parser.syntaxError(types.ERR_INVALID_CHAR))
return &Pair{key, *newSyntaxError(parser.syntaxError(types.ERR_INVALID_CHAR))}
}
}
@@ -640,7 +601,7 @@ func Loads(src string) (int, interface{}, error) {
}
}
// LoadsUseNumber parse all json into interface{}, with numeric nodes cast to json.Number
// LoadsUseNumber parse all json into interface{}, with numeric nodes casted to json.Number
func LoadsUseNumber(src string) (int, interface{}, error) {
ps := &Parser{s: src}
np, err := ps.Parse()
@@ -694,75 +655,6 @@ func (self *Parser) ExportError(err types.ParsingError) error {
}
func backward(src string, i int) int {
for ; i>=0 && utils.IsSpace(src[i]); i-- {}
for ; i>=0 && isSpace(src[i]); i-- {}
return i
}
func newRawNode(str string, typ types.ValueType, lock bool) Node {
ret := Node{
t: typ | _V_RAW,
p: rt.StrPtr(str),
l: uint(len(str)),
}
if lock {
ret.m = new(sync.RWMutex)
}
return ret
}
var typeJumpTable = [256]types.ValueType{
'"' : types.V_STRING,
'-' : _V_NUMBER,
'0' : _V_NUMBER,
'1' : _V_NUMBER,
'2' : _V_NUMBER,
'3' : _V_NUMBER,
'4' : _V_NUMBER,
'5' : _V_NUMBER,
'6' : _V_NUMBER,
'7' : _V_NUMBER,
'8' : _V_NUMBER,
'9' : _V_NUMBER,
'[' : types.V_ARRAY,
'f' : types.V_FALSE,
'n' : types.V_NULL,
't' : types.V_TRUE,
'{' : types.V_OBJECT,
}
func switchRawType(c byte) types.ValueType {
return typeJumpTable[c]
}
func (self *Node) loadt() types.ValueType {
return (types.ValueType)(atomic.LoadInt64(&self.t))
}
func (self *Node) lock() bool {
if m := self.m; m != nil {
m.Lock()
return true
}
return false
}
func (self *Node) unlock() {
if m := self.m; m != nil {
m.Unlock()
}
}
func (self *Node) rlock() bool {
if m := self.m; m != nil {
m.RLock()
return true
}
return false
}
func (self *Node) runlock() {
if m := self.m; m != nil {
m.RUnlock()
}
}

31
vendor/github.com/bytedance/sonic/ast/search.go generated vendored
View File

@@ -21,23 +21,8 @@ import (
`github.com/bytedance/sonic/internal/native/types`
)
// SearchOptions controls Searcher's behavior
type SearchOptions struct {
// ValidateJSON indicates the searcher to validate the entire JSON
ValidateJSON bool
// CopyReturn indicates the searcher to copy the result JSON instead of refer from the input
// This can help to reduce memory usage if you cache the results
CopyReturn bool
// ConcurrentRead indicates the searcher to return a concurrently-READ-safe node,
// including: GetByPath/Get/Index/GetOrIndex/Int64/Bool/Float64/String/Number/Interface/Array/Map/Raw/MarshalJSON
ConcurrentRead bool
}
type Searcher struct {
parser Parser
SearchOptions
}
func NewSearcher(str string) *Searcher {
@@ -46,16 +31,12 @@ func NewSearcher(str string) *Searcher {
s: str,
noLazy: false,
},
SearchOptions: SearchOptions{
ValidateJSON: true,
},
}
}
// GetByPathCopy search in depth from top json and returns a **Copied** json node at the path location
func (self *Searcher) GetByPathCopy(path ...interface{}) (Node, error) {
self.CopyReturn = true
return self.getByPath(path...)
return self.getByPath(true, true, path...)
}
// GetByPathNoCopy search in depth from top json and returns a **Referenced** json node at the path location
@@ -63,15 +44,15 @@ func (self *Searcher) GetByPathCopy(path ...interface{}) (Node, error) {
// WARN: this search directly refer partial json from top json, which has faster speed,
// may consumes more memory.
func (self *Searcher) GetByPath(path ...interface{}) (Node, error) {
return self.getByPath(path...)
return self.getByPath(false, true, path...)
}
func (self *Searcher) getByPath(path ...interface{}) (Node, error) {
func (self *Searcher) getByPath(copystring bool, validate bool, path ...interface{}) (Node, error) {
var err types.ParsingError
var start int
self.parser.p = 0
start, err = self.parser.getByPath(self.ValidateJSON, path...)
start, err = self.parser.getByPath(validate, path...)
if err != 0 {
// for compatibility with old version
if err == types.ERR_NOT_FOUND {
@@ -90,12 +71,12 @@ func (self *Searcher) getByPath(path ...interface{}) (Node, error) {
// copy string to reducing memory usage
var raw string
if self.CopyReturn {
if copystring {
raw = rt.Mem2Str([]byte(self.parser.s[start:self.parser.p]))
} else {
raw = self.parser.s[start:self.parser.p]
}
return newRawNode(raw, t, self.ConcurrentRead), nil
return newRawNode(raw, t), nil
}
// GetByPath searches a path and returns relaction and types of target

16
vendor/github.com/bytedance/sonic/compat.go generated vendored
View File

@@ -1,4 +1,4 @@
// +build !amd64,!arm64 go1.26 !go1.17 arm64,!go1.20
// +build !amd64 !go1.16 go1.23
/*
* Copyright 2021 ByteDance Inc.
@@ -27,8 +27,6 @@ import (
`github.com/bytedance/sonic/option`
)
const apiKind = UseStdJSON
type frozenConfig struct {
Config
}
@@ -87,17 +85,7 @@ func (cfg frozenConfig) UnmarshalFromString(buf string, val interface{}) error {
if cfg.DisallowUnknownFields {
dec.DisallowUnknownFields()
}
err := dec.Decode(val)
if err != nil {
return err
}
// check the trailing chars
offset := dec.InputOffset()
if t, err := dec.Token(); !(t == nil && err == io.EOF) {
return &json.SyntaxError{ Offset: offset}
}
return nil
return dec.Decode(val)
}
// Unmarshal is implemented by sonic

43
vendor/github.com/bytedance/sonic/decoder/decoder_compat.go generated vendored
View File

@@ -1,5 +1,4 @@
//go:build (!amd64 && !arm64) || go1.26 || !go1.17 || (arm64 && !go1.20)
// +build !amd64,!arm64 go1.26 !go1.17 arm64,!go1.20
// +build !amd64 !go1.16 go1.23
/*
* Copyright 2023 ByteDance Inc.
@@ -20,33 +19,29 @@
package decoder
import (
"bytes"
"encoding/json"
"io"
"reflect"
"unsafe"
`bytes`
`encoding/json`
`io`
`reflect`
`unsafe`
"github.com/bytedance/sonic/internal/decoder/consts"
"github.com/bytedance/sonic/internal/native/types"
"github.com/bytedance/sonic/option"
"github.com/bytedance/sonic/internal/compat"
`github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/option`
)
func init() {
compat.Warn("sonic/decoder")
println("WARNING: sonic only supports Go1.16~1.22 && CPU amd64, but your environment is not suitable")
}
const (
_F_use_int64 = consts.F_use_int64
_F_disable_urc = consts.F_disable_unknown
_F_disable_unknown = consts.F_disable_unknown
_F_copy_string = consts.F_copy_string
_F_use_int64 = 0
_F_disable_urc = 2
_F_disable_unknown = 3
_F_copy_string = 4
_F_use_number = consts.F_use_number
_F_validate_string = consts.F_validate_string
_F_allow_control = consts.F_allow_control
_F_no_validate_json = consts.F_no_validate_json
_F_case_sensitive = consts.F_case_sensitive
_F_use_number = types.B_USE_NUMBER
_F_validate_string = types.B_VALIDATE_STRING
_F_allow_control = types.B_ALLOW_CONTROL
)
type Options uint64
@@ -58,8 +53,6 @@ const (
OptionDisableUnknown Options = 1 << _F_disable_unknown
OptionCopyString Options = 1 << _F_copy_string
OptionValidateString Options = 1 << _F_validate_string
OptionNoValidateJSON Options = 1 << _F_no_validate_json
OptionCaseSensitive Options = 1 << _F_case_sensitive
)
func (self *Decoder) SetOptions(opts Options) {
@@ -197,5 +190,5 @@ func (s SyntaxError) Error() string {
return (*json.SyntaxError)(unsafe.Pointer(&s)).Error()
}
// MismatchTypeError represents mismatching between json and object
type MismatchTypeError json.UnmarshalTypeError
// MismatchTypeError represents dismatching between json and object
type MismatchTypeError json.UnmarshalTypeError

7
vendor/github.com/bytedance/sonic/encoder/encoder_compat.go generated vendored
View File

@@ -1,4 +1,4 @@
// +build !amd64,!arm64 go1.26 !go1.17 arm64,!go1.20
// +build !amd64 !go1.16 go1.23
/*
* Copyright 2023 ByteDance Inc.
@@ -19,17 +19,16 @@
package encoder
import (
`io`
`io`
`bytes`
`encoding/json`
`reflect`
`github.com/bytedance/sonic/option`
`github.com/bytedance/sonic/internal/compat`
)
func init() {
compat.Warn("sonic/encoder")
println("WARNING:(encoder) sonic only supports Go1.16~1.22 && CPU amd64, but your environment is not suitable")
}
// EnableFallback indicates if encoder use fallback

8
vendor/github.com/bytedance/sonic/internal/caching/hashing.go generated vendored
View File

@@ -23,12 +23,16 @@ import (
)
var (
V_strhash = rt.UnpackEface(rt.Strhash)
V_strhash = rt.UnpackEface(strhash)
S_strhash = *(*uintptr)(V_strhash.Value)
)
//go:noescape
//go:linkname strhash runtime.strhash
func strhash(_ unsafe.Pointer, _ uintptr) uintptr
func StrHash(s string) uint64 {
if v := rt.Strhash(unsafe.Pointer(&s), 0); v == 0 {
if v := strhash(unsafe.Pointer(&s), 0); v == 0 {
return 1
} else {
return uint64(v)

4
vendor/github.com/bytedance/sonic/internal/cpu/features.go generated vendored
View File

@@ -24,6 +24,7 @@ import (
)
var (
HasAVX = cpuid.CPU.Has(cpuid.AVX)
HasAVX2 = cpuid.CPU.Has(cpuid.AVX2)
HasSSE = cpuid.CPU.Has(cpuid.SSE)
)
@@ -32,8 +33,7 @@ func init() {
switch v := os.Getenv("SONIC_MODE"); v {
case "" : break
case "auto" : break
case "noavx" : HasAVX2 = false
// will also disable avx, act as `noavx`, we remain it to make sure forward compatibility
case "noavx" : HasAVX = false; fallthrough
case "noavx2" : HasAVX2 = false
default : panic(fmt.Sprintf("invalid mode: '%s', should be one of 'auto', 'noavx', 'noavx2'", v))
}

1384
vendor/github.com/bytedance/sonic/internal/encoder/compiler.go generated vendored
View File

File diff suppressed because it is too large Load Diff

170
vendor/github.com/bytedance/sonic/internal/encoder/encoder.go generated vendored
View File

@@ -17,63 +17,72 @@
package encoder
import (
"bytes"
"encoding/json"
"reflect"
"runtime"
"unsafe"
`bytes`
`encoding/json`
`reflect`
`runtime`
`unsafe`
"github.com/bytedance/sonic/utf8"
"github.com/bytedance/sonic/internal/encoder/alg"
"github.com/bytedance/sonic/internal/encoder/vars"
"github.com/bytedance/sonic/internal/rt"
"github.com/bytedance/sonic/option"
"github.com/bytedance/gopkg/lang/dirtmake"
`github.com/bytedance/sonic/internal/native`
`github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/internal/rt`
`github.com/bytedance/sonic/utf8`
`github.com/bytedance/sonic/option`
)
// Options is a set of encoding options.
type Options uint64
const (
bitSortMapKeys = iota
bitEscapeHTML
bitCompactMarshaler
bitNoQuoteTextMarshaler
bitNoNullSliceOrMap
bitValidateString
bitNoValidateJSONMarshaler
bitNoEncoderNewline
// used for recursive compile
bitPointerValue = 63
)
const (
// SortMapKeys indicates that the keys of a map needs to be sorted
// before serializing into JSON.
// WARNING: This hurts performance A LOT, USE WITH CARE.
SortMapKeys Options = 1 << alg.BitSortMapKeys
SortMapKeys Options = 1 << bitSortMapKeys
// EscapeHTML indicates encoder to escape all HTML characters
// after serializing into JSON (see https://pkg.go.dev/encoding/json#HTMLEscape).
// WARNING: This hurts performance A LOT, USE WITH CARE.
EscapeHTML Options = 1 << alg.BitEscapeHTML
EscapeHTML Options = 1 << bitEscapeHTML
// CompactMarshaler indicates that the output JSON from json.Marshaler
// is always compact and needs no validation
CompactMarshaler Options = 1 << alg.BitCompactMarshaler
CompactMarshaler Options = 1 << bitCompactMarshaler
// NoQuoteTextMarshaler indicates that the output text from encoding.TextMarshaler
// is always escaped string and needs no quoting
NoQuoteTextMarshaler Options = 1 << alg.BitNoQuoteTextMarshaler
NoQuoteTextMarshaler Options = 1 << bitNoQuoteTextMarshaler
// NoNullSliceOrMap indicates all empty Array or Object are encoded as '[]' or '{}',
// instead of 'null'.
// NOTE: The priority of this option is lower than json tag `omitempty`.
NoNullSliceOrMap Options = 1 << alg.BitNoNullSliceOrMap
// instead of 'null'
NoNullSliceOrMap Options = 1 << bitNoNullSliceOrMap
// ValidateString indicates that encoder should validate the input string
// before encoding it into JSON.
ValidateString Options = 1 << alg.BitValidateString
ValidateString Options = 1 << bitValidateString
// NoValidateJSONMarshaler indicates that the encoder should not validate the output string
// after encoding the JSONMarshaler to JSON.
NoValidateJSONMarshaler Options = 1 << alg.BitNoValidateJSONMarshaler
NoValidateJSONMarshaler Options = 1 << bitNoValidateJSONMarshaler
// NoEncoderNewline indicates that the encoder should not add a newline after every message
NoEncoderNewline Options = 1 << alg.BitNoEncoderNewline
NoEncoderNewline Options = 1 << bitNoEncoderNewline
// CompatibleWithStd is used to be compatible with std encoder.
CompatibleWithStd Options = SortMapKeys | EscapeHTML | CompactMarshaler
// Encode Infinity or Nan float into `null`, instead of returning an error.
EncodeNullForInfOrNan Options = 1 << alg.BitEncodeNullForInfOrNan
)
// Encoder represents a specific set of encoder configurations.
@@ -162,45 +171,53 @@ func (enc *Encoder) SetIndent(prefix, indent string) {
// Quote returns the JSON-quoted version of s.
func Quote(s string) string {
buf := make([]byte, 0, len(s)+2)
buf = alg.Quote(buf, s, false)
return rt.Mem2Str(buf)
var n int
var p []byte
/* check for empty string */
if s == "" {
return `""`
}
/* allocate space for result */
n = len(s) + 2
p = make([]byte, 0, n)
/* call the encoder */
_ = encodeString(&p, s)
return rt.Mem2Str(p)
}
// Encode returns the JSON encoding of val, encoded with opts.
func Encode(val interface{}, opts Options) ([]byte, error) {
var ret []byte
buf := vars.NewBytes()
err := encodeIntoCheckRace(buf, val, opts)
buf := newBytes()
err := encodeInto(&buf, val, opts)
/* check for errors */
if err != nil {
vars.FreeBytes(buf)
freeBytes(buf)
return nil, err
}
/* htmlescape or correct UTF-8 if opts enable */
old := buf
*buf = encodeFinish(*old, opts)
pbuf := ((*rt.GoSlice)(unsafe.Pointer(buf))).Ptr
pold := ((*rt.GoSlice)(unsafe.Pointer(old))).Ptr
buf = encodeFinish(old, opts)
pbuf := ((*rt.GoSlice)(unsafe.Pointer(&buf))).Ptr
pold := ((*rt.GoSlice)(unsafe.Pointer(&old))).Ptr
/* return when allocated a new buffer */
if pbuf != pold {
vars.FreeBytes(old)
return *buf, nil
freeBytes(old)
return buf, nil
}
/* make a copy of the result */
if rt.CanSizeResue(cap(*buf)) {
ret = dirtmake.Bytes(len(*buf), len(*buf))
copy(ret, *buf)
vars.FreeBytes(buf)
} else {
ret = *buf
}
ret = make([]byte, len(buf))
copy(ret, buf)
freeBytes(buf)
/* return the buffer into pool */
return ret, nil
}
@@ -208,7 +225,7 @@ func Encode(val interface{}, opts Options) ([]byte, error) {
// EncodeInto is like Encode but uses a user-supplied buffer instead of allocating
// a new one.
func EncodeInto(buf *[]byte, val interface{}, opts Options) error {
err := encodeIntoCheckRace(buf, val, opts)
err := encodeInto(buf, val, opts)
if err != nil {
return err
}
@@ -217,15 +234,15 @@ func EncodeInto(buf *[]byte, val interface{}, opts Options) error {
}
func encodeInto(buf *[]byte, val interface{}, opts Options) error {
stk := vars.NewStack()
stk := newStack()
efv := rt.UnpackEface(val)
err := encodeTypedPointer(buf, efv.Type, &efv.Value, stk, uint64(opts))
/* return the stack into pool */
if err != nil {
vars.ResetStack(stk)
resetStack(stk)
}
vars.FreeStack(stk)
freeStack(stk)
/* avoid GC ahead */
runtime.KeepAlive(buf)
@@ -237,12 +254,13 @@ func encodeFinish(buf []byte, opts Options) []byte {
if opts & EscapeHTML != 0 {
buf = HTMLEscape(nil, buf)
}
if (opts & ValidateString != 0) && !utf8.Validate(buf) {
if opts & ValidateString != 0 && !utf8.Validate(buf) {
buf = utf8.CorrectWith(nil, buf, `\ufffd`)
}
return buf
}
var typeByte = rt.UnpackType(reflect.TypeOf(byte(0)))
// HTMLEscape appends to dst the JSON-encoded src with <, >, &, U+2028 and U+2029
// characters inside string literals changed to \u003c, \u003e, \u0026, \u2028, \u2029
@@ -251,7 +269,7 @@ func encodeFinish(buf []byte, opts Options) []byte {
// escaping within <script> tags, so an alternative JSON encoding must
// be used.
func HTMLEscape(dst []byte, src []byte) []byte {
return alg.HtmlEscape(dst, src)
return htmlEscape(dst, src)
}
// EncodeIndented is like Encode but applies Indent to format the output.
@@ -259,40 +277,37 @@ func HTMLEscape(dst []byte, src []byte) []byte {
// followed by one or more copies of indent according to the indentation nesting.
func EncodeIndented(val interface{}, prefix string, indent string, opts Options) ([]byte, error) {
var err error
var out []byte
var buf *bytes.Buffer
/* encode into the buffer */
out := vars.NewBytes()
err = EncodeInto(out, val, opts)
out = newBytes()
err = EncodeInto(&out, val, opts)
/* check for errors */
if err != nil {
vars.FreeBytes(out)
freeBytes(out)
return nil, err
}
/* indent the JSON */
buf = vars.NewBuffer()
err = json.Indent(buf, *out, prefix, indent)
vars.FreeBytes(out)
buf = newBuffer()
err = json.Indent(buf, out, prefix, indent)
/* check for errors */
if err != nil {
vars.FreeBuffer(buf)
freeBytes(out)
freeBuffer(buf)
return nil, err
}
/* copy to the result buffer */
var ret []byte
if rt.CanSizeResue(cap(buf.Bytes())) {
ret = make([]byte, buf.Len())
copy(ret, buf.Bytes())
/* return the buffers into pool */
vars.FreeBuffer(buf)
} else {
ret = buf.Bytes()
}
ret := make([]byte, buf.Len())
copy(ret, buf.Bytes())
/* return the buffers into pool */
freeBytes(out)
freeBuffer(buf)
return ret, nil
}
@@ -315,5 +330,26 @@ func Pretouch(vt reflect.Type, opts ...option.CompileOption) error {
//
// Note: it does not check for the invalid UTF-8 characters.
func Valid(data []byte) (ok bool, start int) {
return alg.Valid(data)
n := len(data)
if n == 0 {
return false, -1
}
s := rt.Mem2Str(data)
p := 0
m := types.NewStateMachine()
ret := native.ValidateOne(&s, &p, m, types.F_VALIDATE_STRING)
types.FreeStateMachine(m)
if ret < 0 {
return false, p-1
}
/* check for trailing spaces */
for ;p < n; p++ {
if (types.SPACE_MASK & (1 << data[p])) == 0 {
return false, p
}
}
return true, ret
}

28
vendor/github.com/bytedance/sonic/internal/encoder/stream.go generated vendored
View File

@@ -17,10 +17,8 @@
package encoder
import (
"encoding/json"
"io"
"github.com/bytedance/sonic/internal/encoder/vars"
`encoding/json`
`io`
)
// StreamEncoder uses io.Writer as input.
@@ -38,20 +36,21 @@ func NewStreamEncoder(w io.Writer) *StreamEncoder {
// Encode encodes interface{} as JSON to io.Writer
func (enc *StreamEncoder) Encode(val interface{}) (err error) {
out := vars.NewBytes()
buf := newBytes()
out := buf
/* encode into the buffer */
err = EncodeInto(out, val, enc.Opts)
err = EncodeInto(&out, val, enc.Opts)
if err != nil {
goto free_bytes
}
if enc.indent != "" || enc.prefix != "" {
/* indent the JSON */
buf := vars.NewBuffer()
err = json.Indent(buf, *out, enc.prefix, enc.indent)
buf := newBuffer()
err = json.Indent(buf, out, enc.prefix, enc.indent)
if err != nil {
vars.FreeBuffer(buf)
freeBuffer(buf)
goto free_bytes
}
@@ -63,17 +62,16 @@ func (enc *StreamEncoder) Encode(val interface{}) (err error) {
/* copy into io.Writer */
_, err = io.Copy(enc.w, buf)
if err != nil {
vars.FreeBuffer(buf)
freeBuffer(buf)
goto free_bytes
}
} else {
/* copy into io.Writer */
var n int
buf := *out
for len(buf) > 0 {
n, err = enc.w.Write(buf)
buf = buf[n:]
for len(out) > 0 {
n, err = enc.w.Write(out)
out = out[n:]
if err != nil {
goto free_bytes
}
@@ -86,6 +84,6 @@ func (enc *StreamEncoder) Encode(val interface{}) (err error) {
}
free_bytes:
vars.FreeBytes(out)
freeBytes(buf)
return err
}

13
vendor/github.com/bytedance/sonic/internal/jit/arch_amd64.go generated vendored
View File

@@ -17,10 +17,8 @@
package jit
import (
"unsafe"
"github.com/twitchyliquid64/golang-asm/asm/arch"
"github.com/twitchyliquid64/golang-asm/obj"
`github.com/twitchyliquid64/golang-asm/asm/arch`
`github.com/twitchyliquid64/golang-asm/obj`
)
var (
@@ -35,13 +33,6 @@ func As(op string) obj.As {
}
}
func ImmPtr(imm unsafe.Pointer) obj.Addr {
return obj.Addr {
Type : obj.TYPE_CONST,
Offset : int64(uintptr(imm)),
}
}
func Imm(imm int64) obj.Addr {
return obj.Addr {
Type : obj.TYPE_CONST,

7
vendor/github.com/bytedance/sonic/internal/jit/backend.go generated vendored
View File

@@ -21,7 +21,6 @@ import (
`sync`
_ `unsafe`
`github.com/bytedance/sonic/internal/rt`
`github.com/twitchyliquid64/golang-asm/asm/arch`
`github.com/twitchyliquid64/golang-asm/obj`
`github.com/twitchyliquid64/golang-asm/objabi`
@@ -39,6 +38,10 @@ var (
_progPool sync.Pool
)
//go:nosplit
//go:linkname throw runtime.throw
func throw(_ string)
func newProg() *obj.Prog {
if val := _progPool.Get(); val == nil {
return new(obj.Prog)
@@ -68,7 +71,7 @@ func newLinkContext(arch *obj.LinkArch) (ret *obj.Link) {
}
func diagLinkContext(str string, args ...interface{}) {
rt.Throw(fmt.Sprintf(str, args...))
throw(fmt.Sprintf(str, args...))
}
func (self *Backend) New() (ret *obj.Prog) {

2
vendor/github.com/bytedance/sonic/internal/jit/runtime.go generated vendored
View File

@@ -37,7 +37,7 @@ func Type(t reflect.Type) obj.Addr {
}
func Itab(i *rt.GoType, t reflect.Type) obj.Addr {
return Imm(int64(uintptr(unsafe.Pointer(rt.GetItab(rt.IfaceType(i), rt.UnpackType(t), false)))))
return Imm(int64(uintptr(unsafe.Pointer(rt.Getitab(rt.IfaceType(i), rt.UnpackType(t), false)))))
}
func Gitab(i *rt.GoItab) obj.Addr {

75
vendor/github.com/bytedance/sonic/internal/native/dispatch_amd64.go generated vendored
View File

@@ -20,6 +20,7 @@ import (
`unsafe`
`github.com/bytedance/sonic/internal/cpu`
`github.com/bytedance/sonic/internal/native/avx`
`github.com/bytedance/sonic/internal/native/avx2`
`github.com/bytedance/sonic/internal/native/sse`
`github.com/bytedance/sonic/internal/native/types`
@@ -86,10 +87,6 @@ var (
__ValidateUTF8 func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer) (ret int)
__ValidateUTF8Fast func(s unsafe.Pointer) (ret int)
__ParseWithPadding func(parser unsafe.Pointer) (ret int)
__LookupSmallKey func(key unsafe.Pointer, table unsafe.Pointer, lowerOff int) (index int)
)
//go:nosplit
@@ -162,22 +159,12 @@ func ValidateUTF8Fast(s *string) (ret int) {
return __ValidateUTF8Fast(rt.NoEscape(unsafe.Pointer(s)))
}
//go:nosplit
func ParseWithPadding(parser unsafe.Pointer) (ret int) {
return __ParseWithPadding(rt.NoEscape(unsafe.Pointer(parser)))
}
//go:nosplit
func LookupSmallKey(key *string, table *[]byte, lowerOff int) (index int) {
return __LookupSmallKey(rt.NoEscape(unsafe.Pointer(key)), rt.NoEscape(unsafe.Pointer(table)), lowerOff)
}
func useSSE() {
sse.Use()
S_f64toa = sse.S_f64toa
__F64toa = sse.F_f64toa
S_f32toa = sse.S_f32toa
__F32toa = sse.F_f32toa
__F64toa = sse.F_f64toa
S_i64toa = sse.S_i64toa
__I64toa = sse.F_i64toa
S_u64toa = sse.S_u64toa
@@ -205,8 +192,42 @@ func useSSE() {
__ValidateOne = sse.F_validate_one
__ValidateUTF8= sse.F_validate_utf8
__ValidateUTF8Fast = sse.F_validate_utf8_fast
__ParseWithPadding = sse.F_parse_with_padding
__LookupSmallKey = sse.F_lookup_small_key
}
func useAVX() {
avx.Use()
S_f64toa = avx.S_f64toa
__F64toa = avx.F_f64toa
S_f32toa = avx.S_f32toa
__F64toa = avx.F_f64toa
S_i64toa = avx.S_i64toa
__I64toa = avx.F_i64toa
S_u64toa = avx.S_u64toa
__U64toa = avx.F_u64toa
S_lspace = avx.S_lspace
S_quote = avx.S_quote
__Quote = avx.F_quote
S_unquote = avx.S_unquote
__Unquote = avx.F_unquote
S_value = avx.S_value
__Value = avx.F_value
S_vstring = avx.S_vstring
S_vnumber = avx.S_vnumber
S_vsigned = avx.S_vsigned
S_vunsigned = avx.S_vunsigned
S_skip_one = avx.S_skip_one
__SkipOne = avx.F_skip_one
__SkipOneFast = avx.F_skip_one_fast
S_skip_array = avx.S_skip_array
S_skip_object = avx.S_skip_object
S_skip_number = avx.S_skip_number
S_get_by_path = avx.S_get_by_path
__GetByPath = avx.F_get_by_path
__HTMLEscape = avx.F_html_escape
__ValidateOne = avx.F_validate_one
__ValidateUTF8= avx.F_validate_utf8
__ValidateUTF8Fast = avx.F_validate_utf8_fast
}
func useAVX2() {
@@ -214,7 +235,7 @@ func useAVX2() {
S_f64toa = avx2.S_f64toa
__F64toa = avx2.F_f64toa
S_f32toa = avx2.S_f32toa
__F32toa = avx2.F_f32toa
__F64toa = avx2.F_f64toa
S_i64toa = avx2.S_i64toa
__I64toa = avx2.F_i64toa
S_u64toa = avx2.S_u64toa
@@ -242,17 +263,17 @@ func useAVX2() {
__ValidateOne = avx2.F_validate_one
__ValidateUTF8= avx2.F_validate_utf8
__ValidateUTF8Fast = avx2.F_validate_utf8_fast
__ParseWithPadding = avx2.F_parse_with_padding
__LookupSmallKey = avx2.F_lookup_small_key
}
func init() {
if cpu.HasAVX2 {
useAVX2()
} else if cpu.HasSSE {
useSSE()
} else {
panic("Unsupported CPU, lacks of AVX2 or SSE CPUID Flag. maybe it's too old to run Sonic.")
}
if cpu.HasAVX2 {
useAVX2()
} else if cpu.HasAVX {
useAVX()
} else if cpu.HasSSE {
useSSE()
} else {
panic("Unsupported CPU, maybe it's too old to run Sonic.")
}
}

5
vendor/github.com/bytedance/sonic/internal/native/types/types.go generated vendored
View File

@@ -22,7 +22,7 @@ import (
`unsafe`
)
type ValueType = int64
type ValueType int
type ParsingError uint
type SearchingError uint
@@ -57,9 +57,6 @@ const (
B_USE_NUMBER = 1
B_VALIDATE_STRING = 5
B_ALLOW_CONTROL = 31
// for native.SkipOne() flags
B_NO_VALIDATE_JSON= 6
)
const (

40
vendor/github.com/bytedance/sonic/internal/resolver/resolver.go generated vendored
View File

@@ -17,11 +17,10 @@
package resolver
import (
"fmt"
"reflect"
"strings"
"sync"
_ "unsafe"
`fmt`
`reflect`
`strings`
`sync`
)
type FieldOpts int
@@ -30,7 +29,6 @@ type OffsetType int
const (
F_omitempty FieldOpts = 1 << iota
F_stringize
F_omitzero
)
const (
@@ -49,7 +47,6 @@ type FieldMeta struct {
Path []Offset
Opts FieldOpts
Type reflect.Type
IsZero func(reflect.Value) bool
}
func (self *FieldMeta) String() string {
@@ -120,26 +117,20 @@ func resolveFields(vt reflect.Type) []FieldMeta {
/* convert each field */
for _, fv := range tfv.list {
/* add to result */
ret = append(ret, FieldMeta{})
fm := &ret[len(ret)-1]
item := vt
path := []Offset(nil)
opts := FieldOpts(0)
/* check for "string" */
if fv.quoted {
fm.Opts |= F_stringize
opts |= F_stringize
}
/* check for "omitempty" */
if fv.omitEmpty {
fm.Opts |= F_omitempty
opts |= F_omitempty
}
/* handle the "omitzero" */
handleOmitZero(fv, fm)
/* dump the field path */
for _, i := range fv.index {
kind := F_offset
@@ -170,9 +161,13 @@ func resolveFields(vt reflect.Type) []FieldMeta {
path[idx].Kind = F_offset
}
fm.Type = fvt
fm.Path = path
fm.Name = fv.name
/* add to result */
ret = append(ret, FieldMeta {
Type: fvt,
Opts: opts,
Path: path,
Name: fv.name,
})
}
/* optimize the offsets */
@@ -217,10 +212,3 @@ func ResolveStruct(vt reflect.Type) []FieldMeta {
fieldCache[vt] = fm
return fm
}
func handleOmitZero(fv StdField, fm *FieldMeta) {
if fv.omitZero {
fm.Opts |= F_omitzero
fm.IsZero = fv.isZero
}
}

41
vendor/github.com/bytedance/sonic/internal/rt/asm_amd64.s generated vendored
View File

@@ -1,5 +1,4 @@
// +build !noasm,amd64 !appengine,amd64
// Code generated by asm2asm, DO NOT EDIT·
#include "go_asm.h"
#include "funcdata.h"
@@ -18,3 +17,43 @@ _entry:
_stack_grow:
CALL runtime·morestack_noctxt<>(SB)
JMP _entry
TEXT ·StopProf(SB), NOSPLIT, $0-0
NO_LOCAL_POINTERS
CMPB github·com∕bytedance∕sonic∕internal∕rt·StopProfiling(SB), $0
JEQ _ret_1
MOVL $1, AX
LEAQ github·com∕bytedance∕sonic∕internal∕rt·yieldCount(SB), CX
LOCK
XADDL AX, (CX)
MOVL runtime·prof+4(SB), AX
TESTL AX, AX
JEQ _ret_1
MOVL AX, github·com∕bytedance∕sonic∕internal∕rt·oldHz(SB)
MOVL $0, runtime·prof+4(SB)
_ret_1:
RET
TEXT ·StartProf(SB), NOSPLIT, $0-0
NO_LOCAL_POINTERS
CMPB github·com∕bytedance∕sonic∕internal∕rt·StopProfiling(SB), $0
JEQ _ret_2
MOVL $-1, AX
LEAQ github·com∕bytedance∕sonic∕internal∕rt·yieldCount(SB), CX
LOCK
XADDL AX, (CX)
CMPL github·com∕bytedance∕sonic∕internal∕rt·yieldCount(SB), $0
JNE _ret_2
CMPL runtime·prof+4(SB), $0
JNE _ret_2
CMPL github·com∕bytedance∕sonic∕internal∕rt·oldHz(SB), $0
JNE _branch_1
MOVL $100, github·com∕bytedance∕sonic∕internal∕rt·oldHz(SB)
_branch_1:
MOVL github·com∕bytedance∕sonic∕internal∕rt·oldHz(SB), AX
MOVL AX, runtime·prof+4(SB)
_ret_2:
RET

36
vendor/github.com/bytedance/sonic/internal/rt/fastmem.go generated vendored
View File

@@ -17,10 +17,8 @@
package rt
import (
"reflect"
"unsafe"
"github.com/bytedance/sonic/option"
`unsafe`
`reflect`
)
//go:nosplit
@@ -92,21 +90,6 @@ func GuardSlice(buf *[]byte, n int) {
}
}
func GuardSlice2(buf []byte, n int) []byte {
c := cap(buf)
l := len(buf)
if c-l < n {
c = c>>1 + n + l
if c < 32 {
c = 32
}
tmp := make([]byte, l, c)
copy(tmp, buf)
buf = tmp
}
return buf
}
//go:nosplit
func Ptr2SlicePtr(s unsafe.Pointer, l int, c int) unsafe.Pointer {
slice := &GoSlice{
@@ -139,17 +122,4 @@ func StrFrom(p unsafe.Pointer, n int64) (s string) {
func NoEscape(p unsafe.Pointer) unsafe.Pointer {
x := uintptr(p)
return unsafe.Pointer(x ^ 0)
}
//go:nosplit
func MoreStack(size uintptr)
//go:nosplit
func Add(ptr unsafe.Pointer, off uintptr) unsafe.Pointer {
return unsafe.Pointer(uintptr(ptr) + off)
}
// CanSizeResue
func CanSizeResue(cap int) bool {
return cap <= int(option.LimitBufferSize)
}
}

218
vendor/github.com/bytedance/sonic/internal/rt/fastvalue.go generated vendored
View File

@@ -17,179 +17,209 @@
package rt
import (
"reflect"
"unsafe"
`reflect`
`unsafe`
)
var (
reflectRtypeItab = findReflectRtypeItab()
reflectRtypeItab = findReflectRtypeItab()
)
// GoType.KindFlags const
const (
F_direct = 1 << 5
F_kind_mask = (1 << 5) - 1
F_direct = 1 << 5
F_kind_mask = (1 << 5) - 1
)
// GoType.Flags const
const (
tflagUncommon uint8 = 1 << 0
tflagExtraStar uint8 = 1 << 1
tflagNamed uint8 = 1 << 2
tflagRegularMemory uint8 = 1 << 3
tflagUncommon uint8 = 1 << 0
tflagExtraStar uint8 = 1 << 1
tflagNamed uint8 = 1 << 2
tflagRegularMemory uint8 = 1 << 3
)
type GoType struct {
Size uintptr
PtrData uintptr
Hash uint32
Flags uint8
Align uint8
FieldAlign uint8
KindFlags uint8
Traits unsafe.Pointer
GCData *byte
Str int32
PtrToSelf int32
Size uintptr
PtrData uintptr
Hash uint32
Flags uint8
Align uint8
FieldAlign uint8
KindFlags uint8
Traits unsafe.Pointer
GCData *byte
Str int32
PtrToSelf int32
}
func (self *GoType) IsNamed() bool {
return (self.Flags & tflagNamed) != 0
return (self.Flags & tflagNamed) != 0
}
func (self *GoType) Kind() reflect.Kind {
return reflect.Kind(self.KindFlags & F_kind_mask)
return reflect.Kind(self.KindFlags & F_kind_mask)
}
func (self *GoType) Pack() (t reflect.Type) {
(*GoIface)(unsafe.Pointer(&t)).Itab = reflectRtypeItab
(*GoIface)(unsafe.Pointer(&t)).Value = unsafe.Pointer(self)
return
(*GoIface)(unsafe.Pointer(&t)).Itab = reflectRtypeItab
(*GoIface)(unsafe.Pointer(&t)).Value = unsafe.Pointer(self)
return
}
func (self *GoType) String() string {
return self.Pack().String()
return self.Pack().String()
}
func (self *GoType) Indirect() bool {
return self.KindFlags&F_direct == 0
return self.KindFlags & F_direct == 0
}
type GoMap struct {
Count int
Flags uint8
B uint8
Overflow uint16
Hash0 uint32
Buckets unsafe.Pointer
OldBuckets unsafe.Pointer
Evacuate uintptr
Extra unsafe.Pointer
}
type GoMapIterator struct {
K unsafe.Pointer
V unsafe.Pointer
T *GoMapType
H *GoMap
Buckets unsafe.Pointer
Bptr *unsafe.Pointer
Overflow *[]unsafe.Pointer
OldOverflow *[]unsafe.Pointer
StartBucket uintptr
Offset uint8
Wrapped bool
B uint8
I uint8
Bucket uintptr
CheckBucket uintptr
}
type GoItab struct {
it unsafe.Pointer
Vt *GoType
hv uint32
_ [4]byte
fn [1]uintptr
it unsafe.Pointer
Vt *GoType
hv uint32
_ [4]byte
fn [1]uintptr
}
type GoIface struct {
Itab *GoItab
Value unsafe.Pointer
Itab *GoItab
Value unsafe.Pointer
}
type GoEface struct {
Type *GoType
Value unsafe.Pointer
Type *GoType
Value unsafe.Pointer
}
func (self GoEface) Pack() (v interface{}) {
*(*GoEface)(unsafe.Pointer(&v)) = self
return
*(*GoEface)(unsafe.Pointer(&v)) = self
return
}
type GoPtrType struct {
GoType
Elem *GoType
GoType
Elem *GoType
}
type GoMapType struct {
GoType
Key *GoType
Elem *GoType
Bucket *GoType
Hasher func(unsafe.Pointer, uintptr) uintptr
KeySize uint8
ElemSize uint8
BucketSize uint16
Flags uint32
GoType
Key *GoType
Elem *GoType
Bucket *GoType
Hasher func(unsafe.Pointer, uintptr) uintptr
KeySize uint8
ElemSize uint8
BucketSize uint16
Flags uint32
}
func (self *GoMapType) IndirectElem() bool {
return self.Flags&2 != 0
return self.Flags & 2 != 0
}
type GoStructType struct {
GoType
Pkg *byte
Fields []GoStructField
GoType
Pkg *byte
Fields []GoStructField
}
type GoStructField struct {
Name *byte
Type *GoType
OffEmbed uintptr
Name *byte
Type *GoType
OffEmbed uintptr
}
type GoInterfaceType struct {
GoType
PkgPath *byte
Methods []GoInterfaceMethod
GoType
PkgPath *byte
Methods []GoInterfaceMethod
}
type GoInterfaceMethod struct {
Name int32
Type int32
Name int32
Type int32
}
type GoSlice struct {
Ptr unsafe.Pointer
Len int
Cap int
Ptr unsafe.Pointer
Len int
Cap int
}
type GoString struct {
Ptr unsafe.Pointer
Len int
Ptr unsafe.Pointer
Len int
}
func PtrElem(t *GoType) *GoType {
return (*GoPtrType)(unsafe.Pointer(t)).Elem
return (*GoPtrType)(unsafe.Pointer(t)).Elem
}
func MapType(t *GoType) *GoMapType {
return (*GoMapType)(unsafe.Pointer(t))
return (*GoMapType)(unsafe.Pointer(t))
}
func IfaceType(t *GoType) *GoInterfaceType {
return (*GoInterfaceType)(unsafe.Pointer(t))
return (*GoInterfaceType)(unsafe.Pointer(t))
}
func UnpackType(t reflect.Type) *GoType {
return (*GoType)((*GoIface)(unsafe.Pointer(&t)).Value)
return (*GoType)((*GoIface)(unsafe.Pointer(&t)).Value)
}
func UnpackEface(v interface{}) GoEface {
return *(*GoEface)(unsafe.Pointer(&v))
return *(*GoEface)(unsafe.Pointer(&v))
}
func UnpackIface(v interface{}) GoIface {
return *(*GoIface)(unsafe.Pointer(&v))
return *(*GoIface)(unsafe.Pointer(&v))
}
func findReflectRtypeItab() *GoItab {
v := reflect.TypeOf(struct{}{})
return (*GoIface)(unsafe.Pointer(&v)).Itab
v := reflect.TypeOf(struct{}{})
return (*GoIface)(unsafe.Pointer(&v)).Itab
}
func AssertI2I2(t *GoType, i GoIface) (r GoIface) {
inter := IfaceType(t)
inter := IfaceType(t)
tab := i.Itab
if tab == nil {
return
}
if (*GoInterfaceType)(tab.it) != inter {
tab = GetItab(inter, tab.Vt, true)
tab = Getitab(inter, tab.Vt, true)
if tab == nil {
return
}
@@ -199,33 +229,15 @@ func AssertI2I2(t *GoType, i GoIface) (r GoIface) {
return
}
func (t *GoType) IsInt64() bool {
return t.Kind() == reflect.Int64 || (t.Kind() == reflect.Int && t.Size == 8)
}
func (t *GoType) IsInt32() bool {
return t.Kind() == reflect.Int32 || (t.Kind() == reflect.Int && t.Size == 4)
}
//go:nosplit
func (t *GoType) IsUint64() bool {
isUint := t.Kind() == reflect.Uint || t.Kind() == reflect.Uintptr
return t.Kind() == reflect.Uint64 || (isUint && t.Size == 8)
}
//go:nosplit
func (t *GoType) IsUint32() bool {
isUint := t.Kind() == reflect.Uint || t.Kind() == reflect.Uintptr
return t.Kind() == reflect.Uint32 || (isUint && t.Size == 4)
}
//go:nosplit
func PtrAdd(ptr unsafe.Pointer, offset uintptr) unsafe.Pointer {
return unsafe.Pointer(uintptr(ptr) + offset)
}
//go:noescape
//go:linkname GetItab runtime.getitab
func GetItab(inter *GoInterfaceType, typ *GoType, canfail bool) *GoItab
//go:linkname Getitab runtime.getitab
func Getitab(inter *GoInterfaceType, typ *GoType, canfail bool) *GoItab
func GetFuncPC(fn interface{}) uintptr {
ft := UnpackEface(fn)
if ft.Type.Kind() != reflect.Func {
panic("not a function")
}
return *(*uintptr)(ft.Value)
}

55
vendor/github.com/bytedance/sonic/internal/rt/gcwb.go generated vendored
View File

@@ -1,5 +1,3 @@
// +build go1.21,!go1.26
/*
* Copyright 2021 ByteDance Inc.
*
@@ -19,18 +17,13 @@
package rt
import (
`os`
`sync/atomic`
`unsafe`
`golang.org/x/arch/x86/x86asm`
)
//go:linkname GcWriteBarrier2 runtime.gcWriteBarrier2
func GcWriteBarrier2()
//go:linkname RuntimeWriteBarrier runtime.writeBarrier
var RuntimeWriteBarrier uintptr
const (
_MaxInstr = 15
)
@@ -83,3 +76,49 @@ func GcwbAddr() uintptr {
}
}
// StopProfiling is used to stop traceback introduced by SIGPROF while native code is running.
// WARN: this option is only a workaround for traceback issue (https://github.com/bytedance/sonic/issues/310),
// and will be dropped when the issue is fixed.
var StopProfiling = os.Getenv("SONIC_STOP_PROFILING") != ""
// WARN: must be aligned with runtime.Prof
// type Prof struct {
// signalLock uint32
// hz int32
// }
var (
// // go:linkname runtimeProf runtime.prof
// runtimeProf Prof
// count of native-C calls
yieldCount uint32
// previous value of runtimeProf.hz
oldHz int32
)
//go:nosplit
func MoreStack(size uintptr)
func StopProf()
// func StopProf() {
// atomic.AddUint32(&yieldCount, 1)
// if runtimeProf.hz != 0 {
// oldHz = runtimeProf.hz
// runtimeProf.hz = 0
// }
// }
func StartProf()
// func StartProf() {
// atomic.AddUint32(&yieldCount, ^uint32(0))
// if yieldCount == 0 && runtimeProf.hz == 0 {
// if oldHz == 0 {
// oldHz = 100
// }
// runtimeProf.hz = oldHz
// }
// }

32
vendor/github.com/bytedance/sonic/loader/stubs.go generated vendored
View File

@@ -17,6 +17,8 @@
package loader
import (
"sync/atomic"
"unsafe"
_ `unsafe`
)
@@ -24,5 +26,35 @@ import (
//goland:noinspection GoUnusedGlobalVariable
var lastmoduledatap *moduledata
func registerModule(mod *moduledata) {
registerModuleLockFree(&lastmoduledatap, mod)
}
//go:linkname moduledataverify1 runtime.moduledataverify1
func moduledataverify1(_ *moduledata)
func registerModuleLockFree(tail **moduledata, mod *moduledata) {
for {
oldTail := loadModule(tail)
if casModule(tail, oldTail, mod) {
storeModule(&oldTail.next, mod)
break
}
}
}
func loadModule(p **moduledata) *moduledata {
return (*moduledata)(atomic.LoadPointer((*unsafe.Pointer)(unsafe.Pointer(p))))
}
func storeModule(p **moduledata, value *moduledata) {
atomic.StorePointer((*unsafe.Pointer)(unsafe.Pointer(p)), unsafe.Pointer(value))
}
func casModule(p **moduledata, oldValue *moduledata, newValue *moduledata) bool {
return atomic.CompareAndSwapPointer(
(*unsafe.Pointer)(unsafe.Pointer(p)),
unsafe.Pointer(oldValue),
unsafe.Pointer(newValue),
)
}

14
vendor/github.com/bytedance/sonic/option/option.go generated vendored
View File

@@ -18,17 +18,10 @@ package option
var (
// DefaultDecoderBufferSize is the initial buffer size of StreamDecoder
DefaultDecoderBufferSize uint = 4 * 1024
DefaultDecoderBufferSize uint = 128 * 1024
// DefaultEncoderBufferSize is the initial buffer size of Encoder
DefaultEncoderBufferSize uint = 4 * 1024
// DefaultAstBufferSize is the initial buffer size of ast.Node.MarshalJSON()
DefaultAstBufferSize uint = 4 * 1024
// LimitBufferSize indicates the max pool buffer size, in case of OOM.
// See issue https://github.com/bytedance/sonic/issues/614
LimitBufferSize uint = 1024 * 1024
DefaultEncoderBufferSize uint = 128 * 1024
)
// CompileOptions includes all options for encoder or decoder compiler.
@@ -68,7 +61,7 @@ type CompileOption func(o *CompileOptions)
//
// For deep nested struct (depth exceeds MaxInlineDepth),
// try to set more loops to completely compile,
// thus reduce JIT instability in the first hit.
// thus reduce JIT unstability in the first hit.
func WithCompileRecursiveDepth(loop int) CompileOption {
return func(o *CompileOptions) {
if loop < 0 {
@@ -90,3 +83,4 @@ func WithCompileMaxInlineDepth(depth int) CompileOption {
o.MaxInlineDepth = depth
}
}

15
vendor/github.com/bytedance/sonic/sonic.go generated vendored
View File

@@ -1,5 +1,4 @@
//go:build (amd64 && go1.17 && !go1.26) || (arm64 && go1.20 && !go1.26)
// +build amd64,go1.17,!go1.26 arm64,go1.20,!go1.26
// +build amd64,go1.16,!go1.23
/*
* Copyright 2021 ByteDance Inc.
@@ -17,6 +16,7 @@
* limitations under the License.
*/
//go:generate make
package sonic
import (
@@ -29,8 +29,6 @@ import (
`github.com/bytedance/sonic/internal/rt`
)
const apiKind = UseSonicJSON
type frozenConfig struct {
Config
encoderOpts encoder.Options
@@ -66,14 +64,8 @@ func (cfg Config) Froze() API {
if cfg.NoEncoderNewline {
api.encoderOpts |= encoder.NoEncoderNewline
}
if cfg.EncodeNullForInfOrNan {
api.encoderOpts |= encoder.EncodeNullForInfOrNan
}
// configure decoder options:
if cfg.NoValidateJSONSkip {
api.decoderOpts |= decoder.OptionNoValidateJSON
}
if cfg.UseInt64 {
api.decoderOpts |= decoder.OptionUseInt64
}
@@ -89,9 +81,6 @@ func (cfg Config) Froze() API {
if cfg.ValidateString {
api.decoderOpts |= decoder.OptionValidateString
}
if cfg.CaseSensitive {
api.decoderOpts |= decoder.OptionCaseSensitive
}
return api
}

8
vendor/github.com/bytedance/sonic/unquote/unquote.go generated vendored
View File

@@ -1,7 +1,3 @@
//go:build (amd64 && go1.17 && !go1.26) || (arm64 && go1.20 && !go1.26)
// +build amd64,go1.17,!go1.26 arm64,go1.20,!go1.26
/*
* Copyright 2021 ByteDance Inc.
*
@@ -29,7 +25,7 @@ import (
`github.com/bytedance/sonic/internal/rt`
)
// String unescapes an escaped string (not including `"` at beginning and end)
// String unescapes a escaped string (not including `"` at begining and end)
// It validates invalid UTF8 and replace with `\ufffd`
func String(s string) (ret string, err types.ParsingError) {
mm := make([]byte, 0, len(s))
@@ -47,7 +43,7 @@ func IntoBytes(s string, m *[]byte) types.ParsingError {
}
}
// String unescapes an escaped string (not including `"` at beginning and end)
// String unescapes a escaped string (not including `"` at begining and end)
// - replace enables replacing invalid utf8 escaped char with `\uffd`
func _String(s string, replace bool) (ret string, err error) {
mm := make([]byte, 0, len(s))

19
vendor/github.com/bytedance/sonic/utf8/utf8.go generated vendored
View File

@@ -1,6 +1,3 @@
//go:build (amd64 && go1.17 && !go1.26) || (arm64 && go1.20 && !go1.26)
// +build amd64,go1.17,!go1.26 arm64,go1.20,!go1.26
/*
* Copyright 2022 ByteDance Inc.
*
@@ -20,8 +17,6 @@
package utf8
import (
`runtime`
`github.com/bytedance/sonic/internal/rt`
`github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/internal/native`
@@ -32,7 +27,7 @@ func CorrectWith(dst []byte, src []byte, repl string) []byte {
sstr := rt.Mem2Str(src)
sidx := 0
/* state machine records the invalid positions */
/* state machine records the invalid postions */
m := types.NewStateMachine()
m.Sp = 0 // invalid utf8 numbers
@@ -65,20 +60,12 @@ func CorrectWith(dst []byte, src []byte, repl string) []byte {
return dst
}
// Validate is a simd-accelerated drop-in replacement for the standard library's utf8.Valid.
// Validate is a simd-accelereated drop-in replacement for the standard library's utf8.Valid.
func Validate(src []byte) bool {
if src == nil {
return true
}
return ValidateString(rt.Mem2Str(src))
}
// ValidateString as Validate, but for string.
func ValidateString(src string) bool {
if src == "" {
return true
}
ret := native.ValidateUTF8Fast(&src) == 0
runtime.KeepAlive(src)
return ret
return native.ValidateUTF8Fast(&src) == 0
}