pig/pig-farm-controller
1
0
Fork 0
Code Issues 11 Pull Requests Actions Packages Projects Releases Wiki Activity

处理AI胡乱生成的乱摊子

Browse Source
This commit is contained in:
huang
2025-09-07 20:36:02 +08:00
parent ba513e0827
commit c4522b974b
403 changed files with 22915 additions and 44424 deletions
Download Patch File Download Diff File Expand all files Collapse all files

45
go.mod
View File

@@ -1,46 +1,47 @@
module git.huangwc.com/pig/pig-farm-controller module git.huangwc.com/pig/pig-farm-controller
go 1.23 go 1.24.0
require ( require (
github.com/gin-gonic/gin v1.9.1 github.com/gin-gonic/gin v1.10.1
gopkg.in/yaml.v2 v2.4.0 gopkg.in/yaml.v2 v2.4.0
gorm.io/driver/postgres v1.5.7 gorm.io/driver/postgres v1.5.7
gorm.io/gorm v1.25.7 gorm.io/gorm v1.25.7
) )
require ( require (
github.com/bytedance/sonic v1.10.0 // indirect github.com/bytedance/gopkg v0.1.3 // indirect
github.com/chenzhuoyu/base64x v0.0.0-20230717121745-296ad89f973d // indirect github.com/bytedance/sonic v1.14.1 // indirect
github.com/chenzhuoyu/iasm v0.9.0 // indirect github.com/bytedance/sonic/loader v0.3.0 // indirect
github.com/gabriel-vasile/mimetype v1.4.2 // indirect github.com/cloudwego/base64x v0.1.6 // indirect
github.com/gin-contrib/sse v0.1.0 // indirect github.com/gabriel-vasile/mimetype v1.4.10 // indirect
github.com/gin-contrib/sse v1.1.0 // indirect
github.com/go-playground/locales v0.14.1 // indirect github.com/go-playground/locales v0.14.1 // indirect
github.com/go-playground/universal-translator v0.18.1 // indirect github.com/go-playground/universal-translator v0.18.1 // indirect
github.com/go-playground/validator/v10 v10.14.0 // indirect github.com/go-playground/validator/v10 v10.27.0 // indirect
github.com/goccy/go-json v0.10.2 // indirect github.com/goccy/go-json v0.10.5 // indirect
github.com/google/go-cmp v0.6.0 // indirect
github.com/jackc/pgpassfile v1.0.0 // indirect github.com/jackc/pgpassfile v1.0.0 // indirect
github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a // indirect github.com/jackc/pgservicefile v0.0.0-20221227161230-091c0ba34f0a // indirect
github.com/jackc/pgx/v5 v5.4.3 // indirect github.com/jackc/pgx/v5 v5.4.3 // indirect
github.com/jinzhu/inflection v1.0.0 // indirect github.com/jinzhu/inflection v1.0.0 // indirect
github.com/jinzhu/now v1.1.5 // indirect github.com/jinzhu/now v1.1.5 // indirect
github.com/json-iterator/go v1.1.12 // indirect github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/cpuid/v2 v2.2.5 // indirect github.com/klauspost/cpuid/v2 v2.3.0 // indirect
github.com/kr/text v0.2.0 // indirect github.com/kr/pretty v0.3.1 // indirect
github.com/leodido/go-urn v1.2.4 // indirect github.com/leodido/go-urn v1.4.0 // indirect
github.com/mattn/go-isatty v0.0.19 // indirect github.com/mattn/go-isatty v0.0.20 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/pelletier/go-toml/v2 v2.0.9 // indirect github.com/pelletier/go-toml/v2 v2.2.4 // indirect
github.com/rogpeppe/go-internal v1.14.1 // indirect github.com/rogpeppe/go-internal v1.14.1 // indirect
github.com/stretchr/testify v1.11.1 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/ugorji/go/codec v1.2.11 // indirect github.com/ugorji/go/codec v1.3.0 // indirect
golang.org/x/arch v0.4.0 // indirect golang.org/x/arch v0.21.0 // indirect
golang.org/x/crypto v0.28.0 // indirect golang.org/x/crypto v0.41.0 // indirect
golang.org/x/net v0.30.0 // indirect golang.org/x/net v0.43.0 // indirect
golang.org/x/sys v0.26.0 // indirect golang.org/x/sys v0.36.0 // indirect
golang.org/x/text v0.19.0 // indirect golang.org/x/text v0.28.0 // indirect
google.golang.org/protobuf v1.31.0 // indirect google.golang.org/protobuf v1.36.8 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect gopkg.in/yaml.v3 v3.0.1 // indirect
) )

107
go.sum
View File

@@ -1,37 +1,33 @@
github.com/bytedance/sonic v1.5.0/go.mod h1:ED5hyg4y6t3/9Ku1R6dU/4KyJ48DZ4jPhfY1O2AihPM= github.com/bytedance/gopkg v0.1.3 h1:TPBSwH8RsouGCBcMBktLt1AymVo2TVsBVCY4b6TnZ/M=
github.com/bytedance/sonic v1.10.0-rc/go.mod h1:ElCzW+ufi8qKqNW0FY314xriJhyJhuoJ3gFZdAHF7NM= github.com/bytedance/gopkg v0.1.3/go.mod h1:576VvJ+eJgyCzdjS+c4+77QF3p7ubbtiKARP3TxducM=
github.com/bytedance/sonic v1.10.0 h1:qtNZduETEIWJVIyDl01BeNxur2rW9OwTQ/yBqFRkKEk= github.com/bytedance/sonic v1.14.1 h1:FBMC0zVz5XUmE4z9wF4Jey0An5FueFvOsTKKKtwIl7w=
github.com/bytedance/sonic v1.10.0/go.mod h1:iZcSUejdk5aukTND/Eu/ivjQuEL0Cu9/rf50Hi0u/g4= github.com/bytedance/sonic v1.14.1/go.mod h1:gi6uhQLMbTdeP0muCnrjHLeCUPyb70ujhnNlhOylAFc=
github.com/chenzhuoyu/base64x v0.0.0-20211019084208-fb5309c8db06/go.mod h1:DH46F32mSOjUmXrMHnKwZdA8wcEefY7UVqBKYGjpdQY= github.com/bytedance/sonic/loader v0.3.0 h1:dskwH8edlzNMctoruo8FPTJDF3vLtDT0sXZwvZJyqeA=
github.com/chenzhuoyu/base64x v0.0.0-20221115062448-fe3a3abad311/go.mod h1:b583jCggY9gE99b6G5LEC39OIiVsWj+R97kbl5odCEk= github.com/bytedance/sonic/loader v0.3.0/go.mod h1:N8A3vUdtUebEY2/VQC0MyhYeKUFosQU6FxH2JmUe6VI=
github.com/chenzhuoyu/base64x v0.0.0-20230717121745-296ad89f973d h1:77cEq6EriyTZ0g/qfRdp61a3Uu/AWrgIq2s0ClJV1g0= github.com/cloudwego/base64x v0.1.6 h1:t11wG9AECkCDk5fMSoxmufanudBtJ+/HemLstXDLI2M=
github.com/chenzhuoyu/base64x v0.0.0-20230717121745-296ad89f973d/go.mod h1:8EPpVsBuRksnlj1mLy4AWzRNQYxauNi62uWcE3to6eA= github.com/cloudwego/base64x v0.1.6/go.mod h1:OFcloc187FXDaYHvrNIjxSe8ncn0OOM8gEHfghB2IPU=
github.com/chenzhuoyu/iasm v0.9.0 h1:9fhXjVzq5hUy2gkhhgHl95zG2cEAhw9OSGs8toWWAwo=
github.com/chenzhuoyu/iasm v0.9.0/go.mod h1:Xjy2NpN3h7aUqeqM+woSuuvxmIe6+DDsiNLIrkAmYog=
github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E= github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c= github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/gabriel-vasile/mimetype v1.4.2 h1:w5qFW6JKBz9Y393Y4q372O9A7cUSequkh1Q7OhCmWKU= github.com/gabriel-vasile/mimetype v1.4.10 h1:zyueNbySn/z8mJZHLt6IPw0KoZsiQNszIpU+bX4+ZK0=
github.com/gabriel-vasile/mimetype v1.4.2/go.mod h1:zApsH/mKG4w07erKIaJPFiX0Tsq9BFQgN3qGY5GnNgA= github.com/gabriel-vasile/mimetype v1.4.10/go.mod h1:d+9Oxyo1wTzWdyVUPMmXFvp4F9tea18J8ufA774AB3s=
github.com/gin-contrib/sse v0.1.0 h1:Y/yl/+YNO8GZSjAhjMsSuLt29uWRFHdHYUb5lYOV9qE= github.com/gin-contrib/sse v1.1.0 h1:n0w2GMuUpWDVp7qSpvze6fAu9iRxJY4Hmj6AmBOU05w=
github.com/gin-contrib/sse v0.1.0/go.mod h1:RHrZQHXnP2xjPF+u1gW/2HnVO7nvIa9PG3Gm+fLHvGI= github.com/gin-contrib/sse v1.1.0/go.mod h1:hxRZ5gVpWMT7Z0B0gSNYqqsSCNIJMjzvm6fqCz9vjwM=
github.com/gin-gonic/gin v1.9.1 h1:4idEAncQnU5cB7BeOkPtxjfCSye0AAm1R0RVIqJ+Jmg= github.com/gin-gonic/gin v1.10.1 h1:T0ujvqyCSqRopADpgPgiTT63DUQVSfojyME59Ei63pQ=
github.com/gin-gonic/gin v1.9.1/go.mod h1:hPrL7YrpYKXt5YId3A/Tnip5kqbEAP+KLuI3SUcPTeU= github.com/gin-gonic/gin v1.10.1/go.mod h1:4PMNQiOhvDRa013RKVbsiNwoyezlm2rm0uX/T7kzp5Y=
github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s= github.com/go-playground/assert/v2 v2.2.0 h1:JvknZsQTYeFEAhQwI4qEt9cyV5ONwRHC+lYKSsYSR8s=
github.com/go-playground/assert/v2 v2.2.0/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4= github.com/go-playground/assert/v2 v2.2.0/go.mod h1:VDjEfimB/XKnb+ZQfWdccd7VUvScMdVu0Titje2rxJ4=
github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA= github.com/go-playground/locales v0.14.1 h1:EWaQ/wswjilfKLTECiXz7Rh+3BjFhfDFKv/oXslEjJA=
github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY= github.com/go-playground/locales v0.14.1/go.mod h1:hxrqLVvrK65+Rwrd5Fc6F2O76J/NuW9t0sjnWqG1slY=
github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY= github.com/go-playground/universal-translator v0.18.1 h1:Bcnm0ZwsGyWbCzImXv+pAJnYK9S473LQFuzCbDbfSFY=
github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY= github.com/go-playground/universal-translator v0.18.1/go.mod h1:xekY+UJKNuX9WP91TpwSH2VMlDf28Uj24BCp08ZFTUY=
github.com/go-playground/validator/v10 v10.14.0 h1:vgvQWe3XCz3gIeFDm/HnTIbj6UGmg/+t63MyGU2n5js= github.com/go-playground/validator/v10 v10.27.0 h1:w8+XrWVMhGkxOaaowyKH35gFydVHOvC0/uWoy2Fzwn4=
github.com/go-playground/validator/v10 v10.14.0/go.mod h1:9iXMNT7sEkjXb0I+enO7QXmzG6QCsPWY4zveKFVRSyU= github.com/go-playground/validator/v10 v10.27.0/go.mod h1:I5QpIEbmr8On7W0TktmJAumgzX4CA1XNl4ZmDuVHKKo=
github.com/goccy/go-json v0.10.2 h1:CrxCmQqYDkv1z7lO7Wbh2HN93uovUHgrECaO5ZrCXAU= github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4=
github.com/goccy/go-json v0.10.2/go.mod h1:6MelG93GURQebXPDq3khkgXZkazVtN9CRI+MGFi0w8I= github.com/goccy/go-json v0.10.5/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaSAoJOfIk= github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
github.com/google/go-cmp v0.5.5/go.mod h1:v8dTdLbMG2kIc/vJvl+f65V22dbkXbowE6jgT/gNBxE= github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
github.com/google/go-cmp v0.6.0 h1:ofyhxvXcZhMsU5ulbFiLKl/XBFqE1GSq7atu8tAmTRI=
github.com/google/go-cmp v0.6.0/go.mod h1:17dUlkBOakJ0+DkrSSNjCkIjxS6bF9zb3elmeNGIjoY=
github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg= github.com/google/gofuzz v1.0.0/go.mod h1:dBl0BpW6vV/+mYPU4Po3pmUjxk6FQPldtuIdl/M65Eg=
github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM= github.com/jackc/pgpassfile v1.0.0 h1:/6Hmqy13Ss2zCq62VdNG8tM1wchn8zjSGOBJ6icpsIM=
github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg= github.com/jackc/pgpassfile v1.0.0/go.mod h1:CEx0iS5ambNFdcRtxPj5JhEz+xB6uRky5eyVu/W2HEg=
@@ -45,27 +41,27 @@ github.com/jinzhu/now v1.1.5 h1:/o9tlHleP7gOFmsnYNz3RGnqzefHA47wQpKrrdTIwXQ=
github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8= github.com/jinzhu/now v1.1.5/go.mod h1:d3SSVoowX0Lcu0IBviAWJpolVfI5UJVZZ7cO71lE/z8=
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM= github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo= github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/klauspost/cpuid/v2 v2.0.9/go.mod h1:FInQzS24/EEf25PyTYn52gqo7WaD8xa0213Md/qVLRg= github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
github.com/klauspost/cpuid/v2 v2.2.5 h1:0E5MSMDEoAulmXNFquVs//DdoomxaoTY1kUhbc/qbZg= github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/klauspost/cpuid/v2 v2.2.5/go.mod h1:Lcz8mBdAVJIBVzewtcLocK12l3Y+JytZYpaMropDUws= github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/knz/go-libedit v1.10.1/go.mod h1:MZTVkCWyz0oBc7JOWP3wNAzd002ZbM/5hgShxwh4x8M= github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/pretty v0.3.0 h1:WgNl7dwNpEZ6jJ9k1snq4pZsg7DOEN8hP9Xw0Tsjwk0=
github.com/kr/pretty v0.3.0/go.mod h1:640gp4NfQd8pI5XOwp5fnNeVWj67G7CFk/SaSQn7NBk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY= github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE= github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/leodido/go-urn v1.2.4 h1:XlAE/cm/ms7TE/VMVoduSpNBoyc2dOxHs5MZSwAN63Q= github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
github.com/leodido/go-urn v1.2.4/go.mod h1:7ZrI8mTSeBSHl/UaRyKQW1qZeMgak41ANeCNaVckg+4= github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
github.com/mattn/go-isatty v0.0.19 h1:JITubQf0MOLdlGRuRq+jtsDlekdYPia9ZFsB8h/APPA= github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.19/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y= github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/concurrent v0.0.0-20180228061459-e0a39a4cb421/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w8PVh93nsPXa1VrQ6jlwL5oN8l14QlcNfg=
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q= github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M= github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk= github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/pelletier/go-toml/v2 v2.0.9 h1:uH2qQXheeefCCkuBBSLi7jCiSmj3VRh2+Goq2N7Xxu0= github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
github.com/pelletier/go-toml/v2 v2.0.9/go.mod h1:tJU2Z3ZkXwnxa4DPO899bsyIoywizdUvyaeZurnPPDc= github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/pkg/diff v0.0.0-20210226163009-20ebb0f2a09e/go.mod h1:pJLUxLENpZxwdsKMEsNbx1VGcRFpLqf3715MtcvvzbA=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ= github.com/rogpeppe/go-internal v1.14.1 h1:UQB4HGPB6osV0SQTLymcB4TgvyWu6ZyliaW0tI/otEQ=
github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc= github.com/rogpeppe/go-internal v1.14.1/go.mod h1:MaRKkUm5W0goXpeCfT7UZI6fk/L7L7so1lCWt35ZSgc=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
@@ -76,30 +72,25 @@ github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU= github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
github.com/stretchr/testify v1.8.2/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4= github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
github.com/stretchr/testify v1.8.4 h1:CcVxjf3Q8PM0mHUKJCdn+eZZtm5yQwehR5yeSVQQcUk= github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI= github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08= github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
github.com/ugorji/go/codec v1.2.11 h1:BMaWp1Bb6fHwEtbplGBGJ498wD+LKlNSl25MjdZY4dU= github.com/ugorji/go/codec v1.3.0 h1:Qd2W2sQawAfG8XSvzwhBeoGq71zXOC/Q1E9y/wUcsUA=
github.com/ugorji/go/codec v1.2.11/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg= github.com/ugorji/go/codec v1.3.0/go.mod h1:pRBVtBSKl77K30Bv8R2P+cLSGaTtex6fsA2Wjqmfxj4=
golang.org/x/arch v0.0.0-20210923205945-b76863e36670/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8= golang.org/x/arch v0.21.0 h1:iTC9o7+wP6cPWpDWkivCvQFGAHDQ59SrSxsLPcnkArw=
golang.org/x/arch v0.4.0 h1:A8WCeEWhLwPBKNbFi5Wv5UTCBx5zzubnXDlMOFAzFMc= golang.org/x/arch v0.21.0/go.mod h1:dNHoOeKiyja7GTvF9NJS1l3Z2yntpQNzgrjh1cU103A=
golang.org/x/arch v0.4.0/go.mod h1:5om86z9Hs0C8fWVUuoMHwpExlXzs5Tkyp9hOrfG7pp8= golang.org/x/crypto v0.41.0 h1:WKYxWedPGCTVVl5+WHSSrOBT0O8lx32+zxmHxijgXp4=
golang.org/x/crypto v0.28.0 h1:GBDwsMXVQi34v5CCYUm2jkJvu4cbtru2U4TN2PSyQnw= golang.org/x/crypto v0.41.0/go.mod h1:pO5AFd7FA68rFak7rOAGVuygIISepHftHnr8dr6+sUc=
golang.org/x/crypto v0.28.0/go.mod h1:rmgy+3RHxRZMyY0jjAJShp2zgEdOqj2AO7U0pYmeQ7U= golang.org/x/net v0.43.0 h1:lat02VYK2j4aLzMzecihNvTlJNQUq316m2Mr9rnM6YE=
golang.org/x/net v0.30.0 h1:AcW1SDZMkb8IpzCdQUaIq2sP4sZ4zw+55h6ynffypl4= golang.org/x/net v0.43.0/go.mod h1:vhO1fvI4dGsIjh73sWfUVjj3N7CA9WkKJNQm2svM6Jg=
golang.org/x/net v0.30.0/go.mod h1:2wGyMJ5iFasEhkwi13ChkO/t1ECNC4X4eBKkVFyYFlU=
golang.org/x/sys v0.5.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg= golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.26.0 h1:KHjCJyddX0LoSTb3J+vWpupP9p0oznkqVk/IfjymZbo= golang.org/x/sys v0.36.0 h1:KVRy2GtZBrk1cBYA7MKu5bEZFxQk4NIDV6RLVcC8o0k=
golang.org/x/sys v0.26.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA= golang.org/x/sys v0.36.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
golang.org/x/text v0.19.0 h1:kTxAhCbGbxhK0IwgSKiMO5awPoDQ0RpfiVYBfK860YM= golang.org/x/text v0.28.0 h1:rhazDwis8INMIwQ4tpjLDzUhx6RlXqZNPEM0huQojng=
golang.org/x/text v0.19.0/go.mod h1:BuEKDfySbSR4drPmRPG/7iBdf8hvFMuRexcpahXilzY= golang.org/x/text v0.28.0/go.mod h1:U8nCwOR8jO/marOQ0QbDiOngZVEBB7MAiitBuMjXiNU=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0= google.golang.org/protobuf v1.36.8 h1:xHScyCOEuuwZEc6UtSOvPbAT4zRh0xcNRYekJwfqyMc=
google.golang.org/protobuf v1.26.0-rc.1/go.mod h1:jlhhOSvTdKEhbULTjvd4ARK9grFBp09yW+WbY/TyQbw= google.golang.org/protobuf v1.36.8/go.mod h1:fuxRtAxBytpl4zzqUh6/eyUujkJdNiuEkXntxiD/uRU=
google.golang.org/protobuf v1.31.0 h1:g0LDEJHgrBl9N9r17Ru3sqWhkIx2NB67okBHPwC7hs8=
google.golang.org/protobuf v1.31.0/go.mod h1:HV8QOd/L58Z+nl8r43ehVNZIU/HEI6OcFqwMG9pJV4I=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q= gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
@@ -112,5 +103,3 @@ gorm.io/driver/postgres v1.5.7 h1:8ptbNJTDbEmhdr62uReG5BGkdQyeasu/FZHxI0IMGnM=
gorm.io/driver/postgres v1.5.7/go.mod h1:3e019WlBaYI5o5LIdNV+LyxCMNtLOQETBXL2h4chKpA= gorm.io/driver/postgres v1.5.7/go.mod h1:3e019WlBaYI5o5LIdNV+LyxCMNtLOQETBXL2h4chKpA=
gorm.io/gorm v1.25.7 h1:VsD6acwRjz2zFxGO50gPO6AkNs7KKnvfzUjHQhZDz/A= gorm.io/gorm v1.25.7 h1:VsD6acwRjz2zFxGO50gPO6AkNs7KKnvfzUjHQhZDz/A=
gorm.io/gorm v1.25.7/go.mod h1:hbnx/Oo0ChWMn1BIhpy1oYozzpM15i4YPuHDmfYtwg8= gorm.io/gorm v1.25.7/go.mod h1:hbnx/Oo0ChWMn1BIhpy1oYozzpM15i4YPuHDmfYtwg8=
nullprogram.com/x/optparse v1.0.0/go.mod h1:KdyPE+Igbe0jQUrVfMqDMeJQIJZEuyV7pjYmp6pbG50=
rsc.io/pdf v0.1.1/go.mod h1:n8OzWcQ6Sp37PL01nO98y4iUCRdTGarVfzxY20ICaU4=

5
vendor/github.com/bytedance/sonic/.gitignore generated vendored
View File

@@ -49,4 +49,7 @@ ast/bench.sh
!testdata/*.json.gz !testdata/*.json.gz
fuzz/testdata fuzz/testdata
*__debug_bin *__debug_bin*
*pprof
*coverage.txt
tools/venv/*

10
vendor/github.com/bytedance/sonic/.gitmodules generated vendored
View File

@@ -1,3 +1,9 @@
[submodule "tools/asm2asm"] [submodule "cloudwego"]
path = tools/asm2asm path = tools/asm2asm
url = https://github.com/chenzhuoyu/asm2asm url = https://github.com/cloudwego/asm2asm.git
[submodule "tools/simde"]
path = tools/simde
url = https://github.com/simd-everywhere/simde.git
[submodule "fuzz/go-fuzz-corpus"]
path = fuzz/go-fuzz-corpus
url = https://github.com/dvyukov/go-fuzz-corpus.git

111
vendor/github.com/bytedance/sonic/Makefile generated vendored
View File

@@ -1,111 +0,0 @@
#
# Copyright 2021 ByteDance Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
ARCH := avx avx2 sse
TMP_DIR := output
OUT_DIR := internal/native
SRC_FILE := native/native.c
CPU_avx := amd64
CPU_avx2 := amd64
CPU_sse := amd64
TMPL_avx := fastint_amd64_test fastfloat_amd64_test native_amd64_test recover_amd64_test
TMPL_avx2 := fastint_amd64_test fastfloat_amd64_test native_amd64_test recover_amd64_test
TMPL_sse := fastint_amd64_test fastfloat_amd64_test native_amd64_test recover_amd64_test
CFLAGS_avx := -msse -mno-sse4 -mavx -mpclmul -mno-avx2 -mstack-alignment=0 -DUSE_AVX=1 -DUSE_AVX2=0
CFLAGS_avx2 := -msse -mno-sse4 -mavx -mpclmul -mavx2 -mstack-alignment=0 -DUSE_AVX=1 -DUSE_AVX2=1
CFLAGS_sse := -msse -mno-sse4 -mno-avx -mno-avx2 -mpclmul
CC_amd64 := clang
ASM2ASM_amd64 := tools/asm2asm/asm2asm.py
CFLAGS := -mno-red-zone
CFLAGS += -target x86_64-apple-macos11
CFLAGS += -fno-asynchronous-unwind-tables
CFLAGS += -fno-builtin
CFLAGS += -fno-exceptions
CFLAGS += -fno-rtti
CFLAGS += -fno-stack-protector
CFLAGS += -nostdlib
CFLAGS += -O3
CFLAGS += -Wall -Werror
NATIVE_SRC := $(wildcard native/*.h)
NATIVE_SRC += $(wildcard native/*.c)
.PHONY: all clean ${ARCH}
define build_tmpl
$(eval @arch := $(1))
$(eval @tmpl := $(2))
$(eval @dest := $(3))
${@dest}: ${@tmpl}
mkdir -p $(dir ${@dest})
echo '// Code generated by Makefile, DO NOT EDIT.' > ${@dest}
echo >> ${@dest}
sed -e 's/{{PACKAGE}}/${@arch}/g' ${@tmpl} >> ${@dest}
endef
define build_arch
$(eval @cpu := $(value CPU_$(1)))
$(eval @deps := $(foreach tmpl,$(value TMPL_$(1)),${OUT_DIR}/$(1)/${tmpl}.go))
$(eval @asmin := ${TMP_DIR}/$(1)/native.s)
$(eval @asmout := ${OUT_DIR}/$(1)/native_text_${@cpu}.go)
$(eval @stubin := ${OUT_DIR}/native_${@cpu}.tmpl)
$(eval @stubout := ${OUT_DIR}/$(1)/native_${@cpu}.go)
$(1): ${@asmout} ${@deps}
${@asmout}: ${@stubout} ${NATIVE_SRC}
mkdir -p ${TMP_DIR}/$(1)
$${CC_${@cpu}} $${CFLAGS} $${CFLAGS_$(1)} -S -o ${TMP_DIR}/$(1)/native.s ${SRC_FILE}
python3 $${ASM2ASM_${@cpu}} -r ${@stubout} ${TMP_DIR}/$(1)/native.s
$(eval $(call \
build_tmpl, \
$(1), \
${@stubin}, \
${@stubout} \
))
$(foreach \
tmpl, \
$(value TMPL_$(1)), \
$(eval $(call \
build_tmpl, \
$(1), \
${OUT_DIR}/${tmpl}.tmpl, \
${OUT_DIR}/$(1)/${tmpl}.go \
)) \
)
endef
all: ${ARCH}
clean:
for arch in ${ARCH}; do \
rm -vfr ${TMP_DIR}/$${arch}; \
rm -vfr ${OUT_DIR}/$${arch}; \
done
$(foreach \
arch, \
${ARCH}, \
$(eval $(call build_arch,${arch})) \
)

120
vendor/github.com/bytedance/sonic/README.md generated vendored
View File

@@ -5,18 +5,28 @@ English | [中文](README_ZH_CN.md)
A blazingly fast JSON serializing & deserializing library, accelerated by JIT (just-in-time compiling) and SIMD (single-instruction-multiple-data). A blazingly fast JSON serializing & deserializing library, accelerated by JIT (just-in-time compiling) and SIMD (single-instruction-multiple-data).
## Requirement ## Requirement
- Go 1.16~1.21
- Linux / MacOS / Windows(need go1.17 above) - Go: 1.18~1.25
- Amd64 ARCH - Notice: Go1.24.0 is not supported due to the [issue](https://github.com/golang/go/issues/71672), please use higher go version or add build tag `--ldflags="-checklinkname=0"`
- OS: Linux / MacOS / Windows
- CPU: AMD64 / (ARM64, need go1.20 above)
## Features ## Features
- Runtime object binding without code generation - Runtime object binding without code generation
- Complete APIs for JSON value manipulation - Complete APIs for JSON value manipulation
- Fast, fast, fast! - Fast, fast, fast!
## APIs
see [go.dev](https://pkg.go.dev/github.com/bytedance/sonic)
## Benchmarks ## Benchmarks
For **all sizes** of json and **all scenarios** of usage, **Sonic performs best**. For **all sizes** of json and **all scenarios** of usage, **Sonic performs best**.
- [Medium](https://github.com/bytedance/sonic/blob/main/decoder/testdata_test.go#L19) (13KB, 300+ key, 6 layers) - [Medium](https://github.com/bytedance/sonic/blob/main/decoder/testdata_test.go#L19) (13KB, 300+ key, 6 layers)
```powershell ```powershell
goversion: 1.17.1 goversion: 1.17.1
goos: darwin goos: darwin
@@ -81,14 +91,16 @@ BenchmarkLoadNode_Parallel/LoadAll()-16 5493 ns/op 2370.6
BenchmarkLoadNode/Interface()-16 17722 ns/op 734.85 MB/s 13323 B/op 88 allocs/op BenchmarkLoadNode/Interface()-16 17722 ns/op 734.85 MB/s 13323 B/op 88 allocs/op
BenchmarkLoadNode_Parallel/Interface()-16 10330 ns/op 1260.70 MB/s 15178 B/op 88 allocs/op BenchmarkLoadNode_Parallel/Interface()-16 10330 ns/op 1260.70 MB/s 15178 B/op 88 allocs/op
``` ```
- [Small](https://github.com/bytedance/sonic/blob/main/testdata/small.go) (400B, 11 keys, 3 layers) - [Small](https://github.com/bytedance/sonic/blob/main/testdata/small.go) (400B, 11 keys, 3 layers)
![small benchmarks](./docs/imgs/bench-small.png) ![small benchmarks](./docs/imgs/bench-small.png)
- [Large](https://github.com/bytedance/sonic/blob/main/testdata/twitter.json) (635KB, 10000+ key, 6 layers) - [Large](https://github.com/bytedance/sonic/blob/main/testdata/twitter.json) (635KB, 10000+ key, 6 layers)
![large benchmarks](./docs/imgs/bench-large.png) ![large benchmarks](./docs/imgs/bench-large.png)
See [bench.sh](https://github.com/bytedance/sonic/blob/main/bench.sh) for benchmark codes. See [bench.sh](https://github.com/bytedance/sonic/blob/main/scripts/bench.sh) for benchmark codes.
## How it works ## How it works
See [INTRODUCTION.md](./docs/INTRODUCTION.md). See [INTRODUCTION.md](./docs/INTRODUCTION.md).
## Usage ## Usage
@@ -96,6 +108,7 @@ See [INTRODUCTION.md](./docs/INTRODUCTION.md).
### Marshal/Unmarshal ### Marshal/Unmarshal
Default behaviors are mostly consistent with `encoding/json`, except HTML escaping form (see [Escape HTML](https://github.com/bytedance/sonic/blob/main/README.md#escape-html)) and `SortKeys` feature (optional support see [Sort Keys](https://github.com/bytedance/sonic/blob/main/README.md#sort-keys)) that is **NOT** in conformity to [RFC8259](https://datatracker.ietf.org/doc/html/rfc8259). Default behaviors are mostly consistent with `encoding/json`, except HTML escaping form (see [Escape HTML](https://github.com/bytedance/sonic/blob/main/README.md#escape-html)) and `SortKeys` feature (optional support see [Sort Keys](https://github.com/bytedance/sonic/blob/main/README.md#sort-keys)) that is **NOT** in conformity to [RFC8259](https://datatracker.ietf.org/doc/html/rfc8259).
```go ```go
import "github.com/bytedance/sonic" import "github.com/bytedance/sonic"
@@ -107,8 +120,11 @@ err := sonic.Unmarshal(output, &data)
``` ```
### Streaming IO ### Streaming IO
Sonic supports decoding json from `io.Reader` or encoding objects into `io.`Writer`, aims at handling multiple values as well as reducing memory consumption.
Sonic supports decoding json from `io.Reader` or encoding objects into `io.Writer`, aims at handling multiple values as well as reducing memory consumption.
- encoder - encoder
```go ```go
var o1 = map[string]interface{}{ var o1 = map[string]interface{}{
"a": "b", "a": "b",
@@ -123,7 +139,9 @@ fmt.Println(w.String())
// {"a":"b"} // {"a":"b"}
// 1 // 1
``` ```
- decoder - decoder
```go ```go
var o = map[string]interface{}{} var o = map[string]interface{}{}
var r = strings.NewReader(`{"a":"b"}{"1":"2"}`) var r = strings.NewReader(`{"a":"b"}{"1":"2"}`)
@@ -136,6 +154,7 @@ fmt.Printf("%+v", o)
``` ```
### Use Number/Use Int64 ### Use Number/Use Int64
```go ```go
import "github.com/bytedance/sonic/decoder" import "github.com/bytedance/sonic/decoder"
@@ -164,7 +183,9 @@ fm := root.Interface().(float64) // jn == jm
``` ```
### Sort Keys ### Sort Keys
On account of the performance loss from sorting (roughly 10%), sonic doesn't enable this feature by default. If your component depends on it to work (like [zstd](https://github.com/facebook/zstd)), Use it like this: On account of the performance loss from sorting (roughly 10%), sonic doesn't enable this feature by default. If your component depends on it to work (like [zstd](https://github.com/facebook/zstd)), Use it like this:
```go ```go
import "github.com/bytedance/sonic" import "github.com/bytedance/sonic"
import "github.com/bytedance/sonic/encoder" import "github.com/bytedance/sonic/encoder"
@@ -177,19 +198,26 @@ v, err := encoder.Encode(m, encoder.SortMapKeys)
var root := sonic.Get(JSON) var root := sonic.Get(JSON)
err := root.SortKeys() err := root.SortKeys()
``` ```
### Escape HTML ### Escape HTML
On account of the performance loss (roughly 15%), sonic doesn't enable this feature by default. You can use `encoder.EscapeHTML` option to open this feature (align with `encoding/json.HTMLEscape`). On account of the performance loss (roughly 15%), sonic doesn't enable this feature by default. You can use `encoder.EscapeHTML` option to open this feature (align with `encoding/json.HTMLEscape`).
```go ```go
import "github.com/bytedance/sonic" import "github.com/bytedance/sonic"
v := map[string]string{"&&":"<>"} v := map[string]string{"&&":"<>"}
ret, err := Encode(v, EscapeHTML) // ret == `{"\u0026\u0026":{"X":"\u003c\u003e"}}` ret, err := Encode(v, EscapeHTML) // ret == `{"\u0026\u0026":{"X":"\u003c\u003e"}}`
``` ```
### Compact Format ### Compact Format
Sonic encodes primitive objects (struct/map...) as compact-format JSON by default, except marshaling `json.RawMessage` or `json.Marshaler`: sonic ensures validating their output JSON but **DONOT** compacting them for performance concerns. We provide the option `encoder.CompactMarshaler` to add compacting process.
Sonic encodes primitive objects (struct/map...) as compact-format JSON by default, except marshaling `json.RawMessage` or `json.Marshaler`: sonic ensures validating their output JSON but **DO NOT** compacting them for performance concerns. We provide the option `encoder.CompactMarshaler` to add compacting process.
### Print Error ### Print Error
If there invalid syntax in input JSON, sonic will return `decoder.SyntaxError`, which supports pretty-printing of error position If there invalid syntax in input JSON, sonic will return `decoder.SyntaxError`, which supports pretty-printing of error position
```go ```go
import "github.com/bytedance/sonic" import "github.com/bytedance/sonic"
import "github.com/bytedance/sonic/decoder" import "github.com/bytedance/sonic/decoder"
@@ -215,7 +243,9 @@ if err != nil {
``` ```
#### Mismatched Types [Sonic v1.6.0] #### Mismatched Types [Sonic v1.6.0]
If there a **mismatch-typed** value for a given key, sonic will report `decoder.MismatchTypeError` (if there are many, report the last one), but still skip wrong the value and keep decoding next JSON. If there a **mismatch-typed** value for a given key, sonic will report `decoder.MismatchTypeError` (if there are many, report the last one), but still skip wrong the value and keep decoding next JSON.
```go ```go
import "github.com/bytedance/sonic" import "github.com/bytedance/sonic"
import "github.com/bytedance/sonic/decoder" import "github.com/bytedance/sonic/decoder"
@@ -228,10 +258,15 @@ err := UnmarshalString(`{"A":"1","B":1}`, &data)
println(err.Error()) // Mismatch type int with value string "at index 5: mismatched type with value\n\n\t{\"A\":\"1\",\"B\":1}\n\t.....^.........\n" println(err.Error()) // Mismatch type int with value string "at index 5: mismatched type with value\n\n\t{\"A\":\"1\",\"B\":1}\n\t.....^.........\n"
fmt.Printf("%+v", data) // {A:0 B:1} fmt.Printf("%+v", data) // {A:0 B:1}
``` ```
### Ast.Node ### Ast.Node
Sonic/ast.Node is a completely self-contained AST for JSON. It implements serialization and deserialization both and provides robust APIs for obtaining and modification of generic data. Sonic/ast.Node is a completely self-contained AST for JSON. It implements serialization and deserialization both and provides robust APIs for obtaining and modification of generic data.
#### Get/Index #### Get/Index
Search partial JSON by given paths, which must be non-negative integer or string, or nil Search partial JSON by given paths, which must be non-negative integer or string, or nil
```go ```go
import "github.com/bytedance/sonic" import "github.com/bytedance/sonic"
@@ -245,10 +280,29 @@ raw := root.Raw() // == string(input)
root, err := sonic.Get(input, "key1", 1, "key2") root, err := sonic.Get(input, "key1", 1, "key2")
sub := root.Get("key3").Index(2).Int64() // == 3 sub := root.Get("key3").Index(2).Int64() // == 3
``` ```
**Tip**: since `Index()` uses offset to locate data, which is much faster than scanning like `Get()`, we suggest you use it as much as possible. And sonic also provides another API `IndexOrGet()` to underlying use offset as well as ensure the key is matched. **Tip**: since `Index()` uses offset to locate data, which is much faster than scanning like `Get()`, we suggest you use it as much as possible. And sonic also provides another API `IndexOrGet()` to underlying use offset as well as ensure the key is matched.
#### SearchOption
`Searcher` provides some options for user to meet different needs:
```go
opts := ast.SearchOption{ CopyReturn: true ... }
val, err := sonic.GetWithOptions(JSON, opts, "key")
```
- CopyReturn
Indicate the searcher to copy the result JSON string instead of refer from the input. This can help to reduce memory usage if you cache the results
- ConcurentRead
Since `ast.Node` use `Lazy-Load` design, it doesn't support Concurrently-Read by default. If you want to read it concurrently, please specify it.
- ValidateJSON
Indicate the searcher to validate the entire JSON. This option is enabled by default, which slow down the search speed a little.
#### Set/Unset #### Set/Unset
Modify the json content by Set()/Unset() Modify the json content by Set()/Unset()
```go ```go
import "github.com/bytedance/sonic" import "github.com/bytedance/sonic"
@@ -265,7 +319,9 @@ println(root.Get("key4").Check()) // "value not exist"
``` ```
#### Serialize #### Serialize
To encode `ast.Node` as json, use `MarshalJson()` or `json.Marshal()` (MUST pass the node's pointer) To encode `ast.Node` as json, use `MarshalJson()` or `json.Marshal()` (MUST pass the node's pointer)
```go ```go
import ( import (
"encoding/json" "encoding/json"
@@ -279,6 +335,7 @@ println(string(buf) == string(exp)) // true
``` ```
#### APIs #### APIs
- validation: `Check()`, `Error()`, `Valid()`, `Exist()` - validation: `Check()`, `Error()`, `Valid()`, `Exist()`
- searching: `Index()`, `Get()`, `IndexPair()`, `IndexOrGet()`, `GetByPath()` - searching: `Index()`, `Get()`, `IndexPair()`, `IndexOrGet()`, `GetByPath()`
- go-type casting: `Int64()`, `Float64()`, `String()`, `Number()`, `Bool()`, `Map[UseNumber|UseNode]()`, `Array[UseNumber|UseNode]()`, `Interface[UseNumber|UseNode]()` - go-type casting: `Int64()`, `Float64()`, `String()`, `Number()`, `Bool()`, `Map[UseNumber|UseNode]()`, `Array[UseNumber|UseNode]()`, `Interface[UseNumber|UseNode]()`
@@ -287,7 +344,9 @@ println(string(buf) == string(exp)) // true
- modification: `Set()`, `SetByIndex()`, `Add()` - modification: `Set()`, `SetByIndex()`, `Add()`
### Ast.Visitor ### Ast.Visitor
Sonic provides an advanced API for fully parsing JSON into non-standard types (neither `struct` not `map[string]interface{}`) without using any intermediate representation (`ast.Node` or `interface{}`). For example, you might have the following types which are like `interface{}` but actually not `interface{}`: Sonic provides an advanced API for fully parsing JSON into non-standard types (neither `struct` not `map[string]interface{}`) without using any intermediate representation (`ast.Node` or `interface{}`). For example, you might have the following types which are like `interface{}` but actually not `interface{}`:
```go ```go
type UserNode interface {} type UserNode interface {}
@@ -302,7 +361,9 @@ type (
UserArray struct{ Value []UserNode } UserArray struct{ Value []UserNode }
) )
``` ```
Sonic provides the following API to return **the preorder traversal of a JSON AST**. The `ast.Visitor` is a SAX style interface which is used in some C++ JSON library. You should implement `ast.Visitor` by yourself and pass it to `ast.Preorder()` method. In your visitor you can make your custom types to represent JSON values. There may be an O(n) space container (such as stack) in your visitor to record the object / array hierarchy. Sonic provides the following API to return **the preorder traversal of a JSON AST**. The `ast.Visitor` is a SAX style interface which is used in some C++ JSON library. You should implement `ast.Visitor` by yourself and pass it to `ast.Preorder()` method. In your visitor you can make your custom types to represent JSON values. There may be an O(n) space container (such as stack) in your visitor to record the object / array hierarchy.
```go ```go
func Preorder(str string, visitor Visitor, opts *VisitorOptions) error func Preorder(str string, visitor Visitor, opts *VisitorOptions) error
@@ -323,20 +384,20 @@ type Visitor interface {
See [ast/visitor.go](https://github.com/bytedance/sonic/blob/main/ast/visitor.go) for detailed usage. We also implement a demo visitor for `UserNode` in [ast/visitor_test.go](https://github.com/bytedance/sonic/blob/main/ast/visitor_test.go). See [ast/visitor.go](https://github.com/bytedance/sonic/blob/main/ast/visitor.go) for detailed usage. We also implement a demo visitor for `UserNode` in [ast/visitor_test.go](https://github.com/bytedance/sonic/blob/main/ast/visitor_test.go).
## Compatibility ## Compatibility
Sonic **DOES NOT** ensure to support all environments, due to the difficulty of developing high-performance codes. For developers who use sonic to build their applications in different environments, we have the following suggestions:
- Developing on **Mac M1**: Make sure you have Rosetta 2 installed on your machine, and set `GOARCH=amd64` when building your application. Rosetta 2 can automatically translate x86 binaries to arm64 binaries and run x86 applications on Mac M1. For developers who want to use sonic to meet different scenarios, we provide some integrated configs as `sonic.API`
- Developing on **Linux arm64**: You can install qemu and use the `qemu-x86_64 -cpu max` command to convert x86 binaries to amr64 binaries for applications built with sonic. The qemu can achieve a similar transfer effect to Rosetta 2 on Mac M1.
For developers who want to use sonic on Linux arm64 without qemu, or those who want to handle JSON strictly consistent with `encoding/json`, we provide some compatible APIs as `sonic.API` - `ConfigDefault`: the sonic's default config (`EscapeHTML=false`,`SortKeys=false`...) to run sonic fast meanwhile ensure security.
- `ConfigDefault`: the sonic's default config (`EscapeHTML=false`,`SortKeys=false`...) to run on sonic-supporting environment. It will fall back to `encoding/json` with the corresponding config, and some options like `SortKeys=false` will be invalid. - `ConfigStd`: the std-compatible config (`EscapeHTML=true`,`SortKeys=true`...)
- `ConfigStd`: the std-compatible config (`EscapeHTML=true`,`SortKeys=true`...) to run on sonic-supporting environment. It will fall back to `encoding/json`. - `ConfigFastest`: the fastest config (`NoQuoteTextMarshaler=true`) to run on sonic as fast as possible.
- `ConfigFastest`: the fastest config (`NoQuoteTextMarshaler=true`) to run on sonic-supporting environment. It will fall back to `encoding/json` with the corresponding config, and some options will be invalid. Sonic **DOES NOT** ensure to support all environments, due to the difficulty of developing high-performance codes. On non-sonic-supporting environment, the implementation will fall back to `encoding/json`. Thus below configs will all equal to `ConfigStd`.
## Tips ## Tips
### Pretouch ### Pretouch
Since Sonic uses [golang-asm](https://github.com/twitchyliquid64/golang-asm) as a JIT assembler, which is NOT very suitable for runtime compiling, first-hit running of a huge schema may cause request-timeout or even process-OOM. For better stability, we advise **using `Pretouch()` for huge-schema or compact-memory applications** before `Marshal()/Unmarshal()`. Since Sonic uses [golang-asm](https://github.com/twitchyliquid64/golang-asm) as a JIT assembler, which is NOT very suitable for runtime compiling, first-hit running of a huge schema may cause request-timeout or even process-OOM. For better stability, we advise **using `Pretouch()` for huge-schema or compact-memory applications** before `Marshal()/Unmarshal()`.
```go ```go
import ( import (
"reflect" "reflect"
@@ -362,17 +423,23 @@ func init() {
``` ```
### Copy string ### Copy string
When decoding **string values without any escaped characters**, sonic references them from the origin JSON buffer instead of mallocing a new buffer to copy. This helps a lot for CPU performance but may leave the whole JSON buffer in memory as long as the decoded objects are being used. In practice, we found the extra memory introduced by referring JSON buffer is usually 20% ~ 80% of decoded objects. Once an application holds these objects for a long time (for example, cache the decoded objects for reusing), its in-use memory on the server may go up. We provide the option `decoder.CopyString()` for users to choose not to reference the JSON buffer, which may cause a decline in CPU performance to some degree.
When decoding **string values without any escaped characters**, sonic references them from the origin JSON buffer instead of mallocing a new buffer to copy. This helps a lot for CPU performance but may leave the whole JSON buffer in memory as long as the decoded objects are being used. In practice, we found the extra memory introduced by referring JSON buffer is usually 20% ~ 80% of decoded objects. Once an application holds these objects for a long time (for example, cache the decoded objects for reusing), its in-use memory on the server may go up. - `Config.CopyString`/`decoder.CopyString()`: We provide the option for `Decode()` / `Unmarshal()` users to choose not to reference the JSON buffer, which may cause a decline in CPU performance to some degree.
- `GetFromStringNoCopy()`: For memory safety, `sonic.Get()` / `sonic.GetFromString()` now copies return JSON. If users want to get json more quickly and not care about memory usage, you can use `GetFromStringNoCopy()` to return a JSON directly referenced from source.
### Pass string or []byte? ### Pass string or []byte?
For alignment to `encoding/json`, we provide API to pass `[]byte` as an argument, but the string-to-bytes copy is conducted at the same time considering safety, which may lose performance when the origin JSON is huge. Therefore, you can use `UnmarshalString()` and `GetFromString()` to pass a string, as long as your origin data is a string or **nocopy-cast** is safe for your []byte. We also provide API `MarshalString()` for convenient **nocopy-cast** of encoded JSON []byte, which is safe since sonic's output bytes is always duplicated and unique. For alignment to `encoding/json`, we provide API to pass `[]byte` as an argument, but the string-to-bytes copy is conducted at the same time considering safety, which may lose performance when the origin JSON is huge. Therefore, you can use `UnmarshalString()` and `GetFromString()` to pass a string, as long as your origin data is a string or **nocopy-cast** is safe for your []byte. We also provide API `MarshalString()` for convenient **nocopy-cast** of encoded JSON []byte, which is safe since sonic's output bytes is always duplicated and unique.
### Accelerate `encoding.TextMarshaler` ### Accelerate `encoding.TextMarshaler`
To ensure data security, sonic.Encoder quotes and escapes string values from `encoding.TextMarshaler` interfaces by default, which may degrade performance much if most of your data is in form of them. We provide `encoder.NoQuoteTextMarshaler` to skip these operations, which means you **MUST** ensure their output string escaped and quoted following [RFC8259](https://datatracker.ietf.org/doc/html/rfc8259). To ensure data security, sonic.Encoder quotes and escapes string values from `encoding.TextMarshaler` interfaces by default, which may degrade performance much if most of your data is in form of them. We provide `encoder.NoQuoteTextMarshaler` to skip these operations, which means you **MUST** ensure their output string escaped and quoted following [RFC8259](https://datatracker.ietf.org/doc/html/rfc8259).
### Better performance for generic data ### Better performance for generic data
In **fully-parsed** scenario, `Unmarshal()` performs better than `Get()`+`Node.Interface()`. But if you only have a part of the schema for specific json, you can combine `Get()` and `Unmarshal()` together: In **fully-parsed** scenario, `Unmarshal()` performs better than `Get()`+`Node.Interface()`. But if you only have a part of the schema for specific json, you can combine `Get()` and `Unmarshal()` together:
```go ```go
import "github.com/bytedance/sonic" import "github.com/bytedance/sonic"
@@ -380,7 +447,9 @@ node, err := sonic.GetFromString(_TwitterJson, "statuses", 3, "user")
var user User // your partial schema... var user User // your partial schema...
err = sonic.UnmarshalString(node.Raw(), &user) err = sonic.UnmarshalString(node.Raw(), &user)
``` ```
Even if you don't have any schema, use `ast.Node` as the container of generic values instead of `map` or `interface`: Even if you don't have any schema, use `ast.Node` as the container of generic values instead of `map` or `interface`:
```go ```go
import "github.com/bytedance/sonic" import "github.com/bytedance/sonic"
@@ -391,7 +460,9 @@ err = user.Check()
// err = user.LoadAll() // only call this when you want to use 'user' concurrently... // err = user.LoadAll() // only call this when you want to use 'user' concurrently...
go someFunc(user) go someFunc(user)
``` ```
Why? Because `ast.Node` stores its children using `array`: Why? Because `ast.Node` stores its children using `array`:
- `Array`'s performance is **much better** than `Map` when Inserting (Deserialize) and Scanning (Serialize) data; - `Array`'s performance is **much better** than `Map` when Inserting (Deserialize) and Scanning (Serialize) data;
- **Hashing** (`map[x]`) is not as efficient as **Indexing** (`array[x]`), which `ast.Node` can conduct on **both array and object**; - **Hashing** (`map[x]`) is not as efficient as **Indexing** (`array[x]`), which `ast.Node` can conduct on **both array and object**;
- Using `Interface()`/`Map()` means Sonic must parse all the underlying values, while `ast.Node` can parse them **on demand**. - Using `Interface()`/`Map()` means Sonic must parse all the underlying values, while `ast.Node` can parse them **on demand**.
@@ -399,6 +470,7 @@ Why? Because `ast.Node` stores its children using `array`:
**CAUTION:** `ast.Node` **DOESN'T** ensure concurrent security directly, due to its **lazy-load** design. However, you can call `Node.Load()`/`Node.LoadAll()` to achieve that, which may bring performance reduction while it still works faster than converting to `map` or `interface{}` **CAUTION:** `ast.Node` **DOESN'T** ensure concurrent security directly, due to its **lazy-load** design. However, you can call `Node.Load()`/`Node.LoadAll()` to achieve that, which may bring performance reduction while it still works faster than converting to `map` or `interface{}`
### Ast.Node or Ast.Visitor? ### Ast.Node or Ast.Visitor?
For generic data, `ast.Node` should be enough for your needs in most cases. For generic data, `ast.Node` should be enough for your needs in most cases.
However, `ast.Node` is designed for partially processing JSON string. It has some special designs such as lazy-load which might not be suitable for directly parsing the whole JSON string like `Unmarshal()`. Although `ast.Node` is better then `map` or `interface{}`, it's also a kind of intermediate representation after all if your final types are customized and you have to convert the above types to your custom types after parsing. However, `ast.Node` is designed for partially processing JSON string. It has some special designs such as lazy-load which might not be suitable for directly parsing the whole JSON string like `Unmarshal()`. Although `ast.Node` is better then `map` or `interface{}`, it's also a kind of intermediate representation after all if your final types are customized and you have to convert the above types to your custom types after parsing.
@@ -407,5 +479,23 @@ For better performance, in previous case the `ast.Visitor` will be the better ch
But `ast.Visitor` is not a very handy API. You might need to write a lot of code to implement your visitor and carefully maintain the tree hierarchy during decoding. Please read the comments in [ast/visitor.go](https://github.com/bytedance/sonic/blob/main/ast/visitor.go) carefully if you decide to use this API. But `ast.Visitor` is not a very handy API. You might need to write a lot of code to implement your visitor and carefully maintain the tree hierarchy during decoding. Please read the comments in [ast/visitor.go](https://github.com/bytedance/sonic/blob/main/ast/visitor.go) carefully if you decide to use this API.
### Buffer Size
Sonic use memory pool in many places like `encoder.Encode`, `ast.Node.MarshalJSON` to improve performance, which may produce more memory usage (in-use) when server's load is high. See [issue 614](https://github.com/bytedance/sonic/issues/614). Therefore, we introduce some options to let user control the behavior of memory pool. See [option](https://pkg.go.dev/github.com/bytedance/sonic@v1.11.9/option#pkg-variables) package.
### Faster JSON Skip
For security, sonic use [FSM](native/skip_one.c) algorithm to validate JSON when decoding raw JSON or encoding `json.Marshaler`, which is much slower (1~10x) than [SIMD-searching-pair](native/skip_one_fast.c) algorithm. If user has many redundant JSON value and DO NOT NEED to strictly validate JSON correctness, you can enable below options:
- `Config.NoValidateSkipJSON`: for faster skipping JSON when decoding, such as unknown fields, json.Unmarshaler(json.RawMessage), mismatched values, and redundant array elements
- `Config.NoValidateJSONMarshaler`: avoid validating JSON when encoding `json.Marshaler`
- `SearchOption.ValidateJSON`: indicates if validate located JSON value when `Get`
## JSON-Path Support (GJSON)
[tidwall/gjson](https://github.com/tidwall/gjson) has provided a comprehensive and popular JSON-Path API, and
a lot of older codes heavily relies on it. Therefore, we provides a wrapper library, which combines gjson's API with sonic's SIMD algorithm to boost up the performance. See [cloudwego/gjson](https://github.com/cloudwego/gjson).
## Community ## Community
Sonic is a subproject of [CloudWeGo](https://www.cloudwego.io/). We are committed to building a cloud native ecosystem. Sonic is a subproject of [CloudWeGo](https://www.cloudwego.io/). We are committed to building a cloud native ecosystem.

92
vendor/github.com/bytedance/sonic/README_ZH_CN.md generated vendored
View File

@@ -6,9 +6,14 @@
## 依赖 ## 依赖
- Go 1.16~1.20 - Go: 1.18~1.25
- Linux / MacOS / Windows（需要 Go1.17 以上） - 注意：Go1.24.0 由于 [issue](https://github.com/golang/go/issues/71672) 不可用，请升级到更高 Go 版本，或添加编译选项 `--ldflags="-checklinkname=0"`
- Amd64 架构 - OS: Linux / MacOS / Windows
- CPU: AMD64 / (ARM64, 需要 Go1.20 以上）
## 接口
详见 [go.dev](https://pkg.go.dev/github.com/bytedance/sonic)
## 特色 ## 特色
@@ -19,7 +24,9 @@
## 基准测试 ## 基准测试
对于**所有大小**的 json 和**所有使用场景**， **Sonic 表现均为最佳**。 对于**所有大小**的 json 和**所有使用场景**， **Sonic 表现均为最佳**。
- [中型](https://github.com/bytedance/sonic/blob/main/decoder/testdata_test.go#L19) (13kB, 300+ 键, 6 层) - [中型](https://github.com/bytedance/sonic/blob/main/decoder/testdata_test.go#L19) (13kB, 300+ 键, 6 层)
```powershell ```powershell
goversion: 1.17.1 goversion: 1.17.1
goos: darwin goos: darwin
@@ -84,12 +91,13 @@ BenchmarkLoadNode_Parallel/LoadAll()-16 5493 ns/op 2370.6
BenchmarkLoadNode/Interface()-16 17722 ns/op 734.85 MB/s 13323 B/op 88 allocs/op BenchmarkLoadNode/Interface()-16 17722 ns/op 734.85 MB/s 13323 B/op 88 allocs/op
BenchmarkLoadNode_Parallel/Interface()-16 10330 ns/op 1260.70 MB/s 15178 B/op 88 allocs/op BenchmarkLoadNode_Parallel/Interface()-16 10330 ns/op 1260.70 MB/s 15178 B/op 88 allocs/op
``` ```
- [小型](https://github.com/bytedance/sonic/blob/main/testdata/small.go) (400B, 11 个键, 3 层) - [小型](https://github.com/bytedance/sonic/blob/main/testdata/small.go) (400B, 11 个键, 3 层)
![small benchmarks](./docs/imgs/bench-small.png) ![small benchmarks](./docs/imgs/bench-small.png)
- [大型](https://github.com/bytedance/sonic/blob/main/testdata/twitter.json) (635kB, 10000+ 个键, 6 层) - [大型](https://github.com/bytedance/sonic/blob/main/testdata/twitter.json) (635kB, 10000+ 个键, 6 层)
![large benchmarks](./docs/imgs/bench-large.png) ![large benchmarks](./docs/imgs/bench-large.png)
要查看基准测试代码，请参阅 [bench.sh](https://github.com/bytedance/sonic/blob/main/bench.sh) 。 要查看基准测试代码，请参阅 [bench.sh](https://github.com/bytedance/sonic/blob/main/scripts/bench.sh) 。
## 工作原理 ## 工作原理
@@ -100,6 +108,7 @@ BenchmarkLoadNode_Parallel/Interface()-16 10330 ns/op 1260.7
### 序列化/反序列化 ### 序列化/反序列化
默认的行为基本上与 `encoding/json` 相一致，除了 HTML 转义形式（参见 [Escape HTML](https://github.com/bytedance/sonic/blob/main/README.md#escape-html)) 和 `SortKeys` 功能（参见 [Sort Keys](https://github.com/bytedance/sonic/blob/main/README.md#sort-keys)）**没有**遵循 [RFC8259](https://datatracker.ietf.org/doc/html/rfc8259) 。 默认的行为基本上与 `encoding/json` 相一致，除了 HTML 转义形式（参见 [Escape HTML](https://github.com/bytedance/sonic/blob/main/README.md#escape-html)) 和 `SortKeys` 功能（参见 [Sort Keys](https://github.com/bytedance/sonic/blob/main/README.md#sort-keys)）**没有**遵循 [RFC8259](https://datatracker.ietf.org/doc/html/rfc8259) 。
```go ```go
import "github.com/bytedance/sonic" import "github.com/bytedance/sonic"
@@ -113,7 +122,9 @@ err := sonic.Unmarshal(output, &data)
### 流式输入输出 ### 流式输入输出
Sonic 支持解码 `io.Reader` 中输入的 json，或将对象编码为 json 后输出至 `io.Writer`，以处理多个值并减少内存消耗。 Sonic 支持解码 `io.Reader` 中输入的 json，或将对象编码为 json 后输出至 `io.Writer`，以处理多个值并减少内存消耗。
- 编码器 - 编码器
```go ```go
var o1 = map[string]interface{}{ var o1 = map[string]interface{}{
"a": "b", "a": "b",
@@ -128,7 +139,9 @@ fmt.Println(w.String())
// {"a":"b"} // {"a":"b"}
// 1 // 1
``` ```
- 解码器 - 解码器
```go ```go
var o = map[string]interface{}{} var o = map[string]interface{}{}
var r = strings.NewReader(`{"a":"b"}{"1":"2"}`) var r = strings.NewReader(`{"a":"b"}{"1":"2"}`)
@@ -172,6 +185,7 @@ fm := root.Interface().(float64) // jn == jm
### 对键排序 ### 对键排序
考虑到排序带来的性能损失（约 10% ）， sonic 默认不会启用这个功能。如果你的组件依赖这个行为（如 [zstd](https://github.com/facebook/zstd)) ，可以仿照下面的例子： 考虑到排序带来的性能损失（约 10% ）， sonic 默认不会启用这个功能。如果你的组件依赖这个行为（如 [zstd](https://github.com/facebook/zstd)) ，可以仿照下面的例子：
```go ```go
import "github.com/bytedance/sonic" import "github.com/bytedance/sonic"
import "github.com/bytedance/sonic/encoder" import "github.com/bytedance/sonic/encoder"
@@ -188,6 +202,7 @@ err := root.SortKeys()
### HTML 转义 ### HTML 转义
考虑到性能损失（约15%）， sonic 默认不会启用这个功能。你可以使用 `encoder.EscapeHTML` 选项来开启（与 `encoding/json.HTMLEscape` 行为一致）。 考虑到性能损失（约15%）， sonic 默认不会启用这个功能。你可以使用 `encoder.EscapeHTML` 选项来开启（与 `encoding/json.HTMLEscape` 行为一致）。
```go ```go
import "github.com/bytedance/sonic" import "github.com/bytedance/sonic"
@@ -196,11 +211,13 @@ ret, err := Encode(v, EscapeHTML) // ret == `{"\u0026\u0026":{"X":"\u003c\u003e"
``` ```
### 紧凑格式 ### 紧凑格式
Sonic 默认将基本类型（ `struct` ， `map` 等）编码为紧凑格式的 JSON ，除非使用 `json.RawMessage` or `json.Marshaler` 进行编码： sonic 确保输出的 JSON 合法，但出于性能考虑，**不会**加工成紧凑格式。我们提供选项 `encoder.CompactMarshaler` 来添加此过程， Sonic 默认将基本类型（ `struct` ， `map` 等）编码为紧凑格式的 JSON ，除非使用 `json.RawMessage` or `json.Marshaler` 进行编码： sonic 确保输出的 JSON 合法，但出于性能考虑，**不会**加工成紧凑格式。我们提供选项 `encoder.CompactMarshaler` 来添加此过程，
### 打印错误 ### 打印错误
如果输入的 JSON 存在无效的语法，sonic 将返回 `decoder.SyntaxError`，该错误支持错误位置的美化输出。 如果输入的 JSON 存在无效的语法，sonic 将返回 `decoder.SyntaxError`，该错误支持错误位置的美化输出。
```go ```go
import "github.com/bytedance/sonic" import "github.com/bytedance/sonic"
import "github.com/bytedance/sonic/decoder" import "github.com/bytedance/sonic/decoder"
@@ -228,6 +245,7 @@ if err != nil {
#### 类型不匹配 [Sonic v1.6.0] #### 类型不匹配 [Sonic v1.6.0]
如果给定键中存在**类型不匹配**的值， sonic 会抛出 `decoder.MismatchTypeError` （如果有多个，只会报告最后一个），但仍会跳过错误的值并解码下一个 JSON 。 如果给定键中存在**类型不匹配**的值， sonic 会抛出 `decoder.MismatchTypeError` （如果有多个，只会报告最后一个），但仍会跳过错误的值并解码下一个 JSON 。
```go ```go
import "github.com/bytedance/sonic" import "github.com/bytedance/sonic"
import "github.com/bytedance/sonic/decoder" import "github.com/bytedance/sonic/decoder"
@@ -240,13 +258,15 @@ err := UnmarshalString(`{"A":"1","B":1}`, &data)
println(err.Error()) // Mismatch type int with value string "at index 5: mismatched type with value\n\n\t{\"A\":\"1\",\"B\":1}\n\t.....^.........\n" println(err.Error()) // Mismatch type int with value string "at index 5: mismatched type with value\n\n\t{\"A\":\"1\",\"B\":1}\n\t.....^.........\n"
fmt.Printf("%+v", data) // {A:0 B:1} fmt.Printf("%+v", data) // {A:0 B:1}
``` ```
### `Ast.Node` ### `Ast.Node`
Sonic/ast.Node 是完全独立的 JSON 抽象语法树库。它实现了序列化和反序列化，并提供了获取和修改通用数据的鲁棒的 API。 Sonic/ast.Node 是完全独立的 JSON 抽象语法树库。它实现了序列化和反序列化，并提供了获取和修改JSON数据的鲁棒的 API。
#### 查找/索引 #### 查找/索引
通过给定的路径搜索 JSON 片段，路径必须为非负整数，字符串或 `nil` 。 通过给定的路径搜索 JSON 片段，路径必须为非负整数，字符串或 `nil` 。
```go ```go
import "github.com/bytedance/sonic" import "github.com/bytedance/sonic"
@@ -260,11 +280,29 @@ raw := root.Raw() // == string(input)
root, err := sonic.Get(input, "key1", 1, "key2") root, err := sonic.Get(input, "key1", 1, "key2")
sub := root.Get("key3").Index(2).Int64() // == 3 sub := root.Get("key3").Index(2).Int64() // == 3
``` ```
**注意**：由于 `Index()` 使用偏移量来定位数据，比使用扫描的 `Get()` 要快的多，建议尽可能的使用 `Index` 。 Sonic 也提供了另一个 API， `IndexOrGet()` ，以偏移量为基础并且也确保键的匹配。 **注意**：由于 `Index()` 使用偏移量来定位数据，比使用扫描的 `Get()` 要快的多，建议尽可能的使用 `Index` 。 Sonic 也提供了另一个 API， `IndexOrGet()` ，以偏移量为基础并且也确保键的匹配。
#### 查找选项
`ast.Searcher`提供了一些选项，以满足用户的不同需求:
```go
opts := ast.SearchOption{CopyReturn: true…}
val, err := sonic.GetWithOptions(JSON, opts, "key")
```
- CopyReturn
指示搜索器复制结果JSON字符串，而不是从输入引用。如果用户缓存结果，这有助于减少内存使用
- ConcurentRead
因为`ast.Node`使用`Lazy-Load`设计，默认不支持并发读取。如果您想同时读取，请指定它。
- ValidateJSON
指示搜索器来验证整个JSON。默认情况下启用该选项, 但是对于查找速度有一定影响。
#### 修改 #### 修改
使用 ` Set()` / `Unset()` 修改 json 的内容 使用 `Set()` / `Unset()` 修改 json 的内容
```go ```go
import "github.com/bytedance/sonic" import "github.com/bytedance/sonic"
@@ -281,7 +319,9 @@ println(root.Get("key4").Check()) // "value not exist"
``` ```
#### 序列化 #### 序列化
要将 `ast.Node` 编码为 json ，使用 `MarshalJson()` 或者 `json.Marshal()` （必须传递指向节点的指针） 要将 `ast.Node` 编码为 json ，使用 `MarshalJson()` 或者 `json.Marshal()` （必须传递指向节点的指针）
```go ```go
import ( import (
"encoding/json" "encoding/json"
@@ -295,6 +335,7 @@ println(string(buf) == string(exp)) // true
``` ```
#### APIs #### APIs
- 合法性检查： `Check()`, `Error()`, `Valid()`, `Exist()` - 合法性检查： `Check()`, `Error()`, `Valid()`, `Exist()`
- 索引： `Index()`, `Get()`, `IndexPair()`, `IndexOrGet()`, `GetByPath()` - 索引： `Index()`, `Get()`, `IndexPair()`, `IndexOrGet()`, `GetByPath()`
- 转换至 go 内置类型： `Int64()`, `Float64()`, `String()`, `Number()`, `Bool()`, `Map[UseNumber|UseNode]()`, `Array[UseNumber|UseNode]()`, `Interface[UseNumber|UseNode]()` - 转换至 go 内置类型： `Int64()`, `Float64()`, `String()`, `Number()`, `Bool()`, `Map[UseNumber|UseNode]()`, `Array[UseNumber|UseNode]()`, `Interface[UseNumber|UseNode]()`
@@ -303,7 +344,9 @@ println(string(buf) == string(exp)) // true
- 修改： `Set()`, `SetByIndex()`, `Add()` - 修改： `Set()`, `SetByIndex()`, `Add()`
### `Ast.Visitor` ### `Ast.Visitor`
Sonic 提供了一个高级的 API 用于直接全量解析 JSON 到非标准容器里 (既不是 `struct` 也不是 `map[string]interface{}`) 且不需要借助任何中间表示 (`ast.Node` 或 `interface{}`)。举个例子，你可能定义了下述的类型，它们看起来像 `interface{}`，但实际上并不是： Sonic 提供了一个高级的 API 用于直接全量解析 JSON 到非标准容器里 (既不是 `struct` 也不是 `map[string]interface{}`) 且不需要借助任何中间表示 (`ast.Node` 或 `interface{}`)。举个例子，你可能定义了下述的类型，它们看起来像 `interface{}`，但实际上并不是：
```go ```go
type UserNode interface {} type UserNode interface {}
@@ -318,7 +361,9 @@ type (
UserArray struct{ Value []UserNode } UserArray struct{ Value []UserNode }
) )
``` ```
Sonic 提供了下述的 API 来返回 **“对 JSON AST 的前序遍历”**。`ast.Visitor` 是一个 SAX 风格的接口，这在某些 C++ 的 JSON 解析库中被使用到。你需要自己实现一个 `ast.Visitor`，将它传递给 `ast.Preorder()` 方法。在你的实现中你可以使用自定义的类型来表示 JSON 的值。在你的 `ast.Visitor` 中，可能需要有一个 O(n) 空间复杂度的容器（比如说栈）来记录 object / array 的层级。 Sonic 提供了下述的 API 来返回 **“对 JSON AST 的前序遍历”**。`ast.Visitor` 是一个 SAX 风格的接口，这在某些 C++ 的 JSON 解析库中被使用到。你需要自己实现一个 `ast.Visitor`，将它传递给 `ast.Preorder()` 方法。在你的实现中你可以使用自定义的类型来表示 JSON 的值。在你的 `ast.Visitor` 中，可能需要有一个 O(n) 空间复杂度的容器（比如说栈）来记录 object / array 的层级。
```go ```go
func Preorder(str string, visitor Visitor, opts *VisitorOptions) error func Preorder(str string, visitor Visitor, opts *VisitorOptions) error
@@ -335,23 +380,24 @@ type Visitor interface {
OnArrayEnd() error OnArrayEnd() error
} }
``` ```
详细用法参看 [ast/visitor.go](https://github.com/bytedance/sonic/blob/main/ast/visitor.go)，我们还为 `UserNode` 实现了一个示例 `ast.Visitor`，你可以在 [ast/visitor_test.go](https://github.com/bytedance/sonic/blob/main/ast/visitor_test.go) 中找到它。 详细用法参看 [ast/visitor.go](https://github.com/bytedance/sonic/blob/main/ast/visitor.go)，我们还为 `UserNode` 实现了一个示例 `ast.Visitor`，你可以在 [ast/visitor_test.go](https://github.com/bytedance/sonic/blob/main/ast/visitor_test.go) 中找到它。
## 兼容性 ## 兼容性
由于开发高性能代码的困难性， Sonic **不**保证对所有环境的支持。对于在不同环境中使用 Sonic 构建应用程序的开发者，我们有以下建议：
- 在 **Mac M1** 上开发：确保在您的计算机上安装了 Rosetta 2，并在构建时设置 `GOARCH=amd64` 。 Rosetta 2 可以自动将 x86 二进制文件转换为 arm64 二进制文件，并在 Mac M1 上运行 x86 应用程序。 对于想要使用sonic来满足不同场景的开发人员，我们提供了一些集成配置:
- 在 **Linux arm64** 上开发：您可以安装 qemu 并使用 `qemu-x86_64 -cpu max` 命令来将 x86 二进制文件转换为 arm64 二进制文件。qemu可以实现与Mac M1上的Rosetta 2类似的转换效果。
对于希望在不使用 qemu 下使用 sonic 的开发者，或者希望处理 JSON 时与 `encoding/JSON` 严格保持一致的开发者，我们在 `sonic.API` 中提供了一些兼容性 API - `ConfigDefault`: sonic的默认配置 (`EscapeHTML=false`， `SortKeys=false`…) 保证性能同时兼顾安全性。
- `ConfigDefault`: 在支持 sonic 的环境下 sonic 的默认配置（`EscapeHTML=false`，`SortKeys=false`等）。行为与具有相应配置的 `encoding/json` 一致，一些选项，如 `SortKeys=false` 将无效。 - `ConfigStd`: 与 `encoding/json` 保证完全兼容的配置
- `ConfigStd`: 在支持 sonic 的环境下与标准库兼容的配置（`EscapeHTML=true`，`SortKeys=true`等）。行为与 `encoding/json` 一致。 - `ConfigFastest`: 最快的配置(`NoQuoteTextMarshaler=true...`) 保证性能最优但是会缺少一些安全性检查（validate UTF8 等）
- `ConfigFastest`: 在支持 sonic 的环境下运行最快的配置（`NoQuoteTextMarshaler=true`）。行为与具有相应配置的 `encoding/json` 一致，某些选项将无效。 Sonic **不**确保支持所有环境，由于开发高性能代码的困难。在不支持sonic的环境中，实现将回落到 `encoding/json`。因此上述配置将全部等于`ConfigStd`。
## 注意事项 ## 注意事项
### 预热 ### 预热
由于 Sonic 使用 [golang-asm](https://github.com/twitchyliquid64/golang-asm) 作为 JIT 汇编器，这个库并不适用于运行时编译，第一次运行一个大型模式可能会导致请求超时甚至进程内存溢出。为了更好地稳定性，我们建议在运行大型模式或在内存有限的应用中，在使用 `Marshal()/Unmarshal()` 前运行 `Pretouch()`。 由于 Sonic 使用 [golang-asm](https://github.com/twitchyliquid64/golang-asm) 作为 JIT 汇编器，这个库并不适用于运行时编译，第一次运行一个大型模式可能会导致请求超时甚至进程内存溢出。为了更好地稳定性，我们建议在运行大型模式或在内存有限的应用中，在使用 `Marshal()/Unmarshal()` 前运行 `Pretouch()`。
```go ```go
import ( import (
"reflect" "reflect"
@@ -381,16 +427,17 @@ func init() {
当解码 **没有转义字符的字符串**时， sonic 会从原始的 JSON 缓冲区内引用而不是复制到新的一个缓冲区中。这对 CPU 的性能方面很有帮助，但是可能因此在解码后对象仍在使用的时候将整个 JSON 缓冲区保留在内存中。实践中我们发现，通过引用 JSON 缓冲区引入的额外内存通常是解码后对象的 20% 至 80% ，一旦应用长期保留这些对象（如缓存以备重用），服务器所使用的内存可能会增加。我们提供了选项 `decoder.CopyString()` 供用户选择，不引用 JSON 缓冲区。这可能在一定程度上降低 CPU 性能。 当解码 **没有转义字符的字符串**时， sonic 会从原始的 JSON 缓冲区内引用而不是复制到新的一个缓冲区中。这对 CPU 的性能方面很有帮助，但是可能因此在解码后对象仍在使用的时候将整个 JSON 缓冲区保留在内存中。实践中我们发现，通过引用 JSON 缓冲区引入的额外内存通常是解码后对象的 20% 至 80% ，一旦应用长期保留这些对象（如缓存以备重用），服务器所使用的内存可能会增加。我们提供了选项 `decoder.CopyString()` 供用户选择，不引用 JSON 缓冲区。这可能在一定程度上降低 CPU 性能。
### 传递字符串还是字节数组？ ### 传递字符串还是字节数组？
为了和 `encoding/json` 保持一致，我们提供了传递 `[]byte` 作为参数的 API ，但考虑到安全性，字符串到字节的复制是同时进行的，这在原始 JSON 非常大时可能会导致性能损失。因此，你可以使用 `UnmarshalString()` 和 `GetFromString()` 来传递字符串，只要你的原始数据是字符串，或**零拷贝类型转换**对于你的字节数组是安全的。我们也提供了 `MarshalString()` 的 API ，以便对编码的 JSON 字节数组进行**零拷贝类型转换**，因为 sonic 输出的字节始终是重复并且唯一的，所以这样是安全的。 为了和 `encoding/json` 保持一致，我们提供了传递 `[]byte` 作为参数的 API ，但考虑到安全性，字符串到字节的复制是同时进行的，这在原始 JSON 非常大时可能会导致性能损失。因此，你可以使用 `UnmarshalString()` 和 `GetFromString()` 来传递字符串，只要你的原始数据是字符串，或**零拷贝类型转换**对于你的字节数组是安全的。我们也提供了 `MarshalString()` 的 API ，以便对编码的 JSON 字节数组进行**零拷贝类型转换**，因为 sonic 输出的字节始终是重复并且唯一的，所以这样是安全的。
### 加速 `encoding.TextMarshaler` ### 加速 `encoding.TextMarshaler`
为了保证数据安全性， `sonic.Encoder` 默认会对来自 `encoding.TextMarshaler` 接口的字符串进行引用和转义，如果大部分数据都是这种形式那可能会导致很大的性能损失。我们提供了 `encoder.NoQuoteTextMarshaler` 选项来跳过这些操作，但你**必须**保证他们的输出字符串依照 [RFC8259](https://datatracker.ietf.org/doc/html/rfc8259) 进行了转义和引用。 为了保证数据安全性， `sonic.Encoder` 默认会对来自 `encoding.TextMarshaler` 接口的字符串进行引用和转义，如果大部分数据都是这种形式那可能会导致很大的性能损失。我们提供了 `encoder.NoQuoteTextMarshaler` 选项来跳过这些操作，但你**必须**保证他们的输出字符串依照 [RFC8259](https://datatracker.ietf.org/doc/html/rfc8259) 进行了转义和引用。
### 泛型的性能优化 ### 泛型的性能优化
在 **完全解析**的场景下， `Unmarshal()` 表现得比 `Get()`+`Node.Interface()` 更好。但是如果你只有特定 JSON 的部分模式，你可以将 `Get()` 和 `Unmarshal()` 结合使用： 在 **完全解析**的场景下， `Unmarshal()` 表现得比 `Get()`+`Node.Interface()` 更好。但是如果你只有特定 JSON 的部分模式，你可以将 `Get()` 和 `Unmarshal()` 结合使用：
```go ```go
import "github.com/bytedance/sonic" import "github.com/bytedance/sonic"
@@ -398,7 +445,9 @@ node, err := sonic.GetFromString(_TwitterJson, "statuses", 3, "user")
var user User // your partial schema... var user User // your partial schema...
err = sonic.UnmarshalString(node.Raw(), &user) err = sonic.UnmarshalString(node.Raw(), &user)
``` ```
甚至如果你没有任何模式，可以用 `ast.Node` 代替 `map` 或 `interface` 作为泛型的容器： 甚至如果你没有任何模式，可以用 `ast.Node` 代替 `map` 或 `interface` 作为泛型的容器：
```go ```go
import "github.com/bytedance/sonic" import "github.com/bytedance/sonic"
@@ -409,7 +458,9 @@ err = user.Check()
// err = user.LoadAll() // only call this when you want to use 'user' concurrently... // err = user.LoadAll() // only call this when you want to use 'user' concurrently...
go someFunc(user) go someFunc(user)
``` ```
为什么？因为 `ast.Node` 使用 `array` 来存储其子节点： 为什么？因为 `ast.Node` 使用 `array` 来存储其子节点：
- 在插入（反序列化）和扫描（序列化）数据时，`Array` 的性能比 `Map` **好得多**； - 在插入（反序列化）和扫描（序列化）数据时，`Array` 的性能比 `Map` **好得多**；
- **哈希**（`map[x]`）的效率不如**索引**（`array[x]`）高效，而 `ast.Node` 可以在数组和对象上使用索引； - **哈希**（`map[x]`）的效率不如**索引**（`array[x]`）高效，而 `ast.Node` 可以在数组和对象上使用索引；
- 使用 `Interface()` / `Map()` 意味着 sonic 必须解析所有的底层值，而 `ast.Node` 可以**按需解析**它们。 - 使用 `Interface()` / `Map()` 意味着 sonic 必须解析所有的底层值，而 `ast.Node` 可以**按需解析**它们。
@@ -417,6 +468,7 @@ go someFunc(user)
**注意**：由于 `ast.Node` 的惰性加载设计，其**不能**直接保证并发安全性，但你可以调用 `Node.Load()` / `Node.LoadAll()` 来实现并发安全。尽管可能会带来性能损失，但仍比转换成 `map` 或 `interface{}` 更为高效。 **注意**：由于 `ast.Node` 的惰性加载设计，其**不能**直接保证并发安全性，但你可以调用 `Node.Load()` / `Node.LoadAll()` 来实现并发安全。尽管可能会带来性能损失，但仍比转换成 `map` 或 `interface{}` 更为高效。
### 使用 `ast.Node` 还是 `ast.Visitor`？ ### 使用 `ast.Node` 还是 `ast.Visitor`？
对于泛型数据的解析，`ast.Node` 在大多数场景上应该能够满足你的需求。 对于泛型数据的解析，`ast.Node` 在大多数场景上应该能够满足你的需求。
然而，`ast.Node` 是一种针对部分解析 JSON 而设计的泛型容器，它包含一些特殊设计，比如惰性加载，如果你希望像 `Unmarshal()` 那样直接解析整个 JSON，这些设计可能并不合适。尽管 `ast.Node` 相较于 `map` 或 `interface{}` 来说是更好的一种泛型容器，但它毕竟也是一种中间表示，如果你的最终类型是自定义的，你还得在解析完成后将上述类型转化成你自定义的类型。 然而，`ast.Node` 是一种针对部分解析 JSON 而设计的泛型容器，它包含一些特殊设计，比如惰性加载，如果你希望像 `Unmarshal()` 那样直接解析整个 JSON，这些设计可能并不合适。尽管 `ast.Node` 相较于 `map` 或 `interface{}` 来说是更好的一种泛型容器，但它毕竟也是一种中间表示，如果你的最终类型是自定义的，你还得在解析完成后将上述类型转化成你自定义的类型。
@@ -425,6 +477,18 @@ go someFunc(user)
但是，`ast.Visitor` 并不是一个很易用的 API。你可能需要写大量的代码去实现自己的 `ast.Visitor`，并且需要在解析过程中仔细维护树的层级。如果你决定要使用这个 API，请先仔细阅读 [ast/visitor.go](https://github.com/bytedance/sonic/blob/main/ast/visitor.go) 中的注释。 但是，`ast.Visitor` 并不是一个很易用的 API。你可能需要写大量的代码去实现自己的 `ast.Visitor`，并且需要在解析过程中仔细维护树的层级。如果你决定要使用这个 API，请先仔细阅读 [ast/visitor.go](https://github.com/bytedance/sonic/blob/main/ast/visitor.go) 中的注释。
### 缓冲区大小
Sonic在许多地方使用内存池，如`encoder.Encode`, `ast.Node.MarshalJSON`等来提高性能，这可能会在服务器负载高时产生更多的内存使用(in-use)。参见[issue 614](https://github.com/bytedance/sonic/issues/614)。因此，我们引入了一些选项来让用户配置内存池的行为。参见[option](https://pkg.go.dev/github.com/bytedance/sonic@v1.11.9/option#pkg-variables)包。
### 更快的 JSON Skip
为了安全起见，在跳过原始JSON 时，sonic decoder 默认使用[FSM](native/skip_one.c)算法扫描来跳过同时校验 JSON。它相比[SIMD-searching-pair](native/skip_one_fast.c)算法跳过要慢得多(1~10倍)。如果用户有很多冗余的JSON值，并且不需要严格验证JSON的正确性，你可以启用以下选项:
- `Config.NoValidateSkipJSON`: 用于在解码时更快地跳过JSON，例如未知字段，`json.RawMessage`，不匹配的值和冗余的数组元素等
- `Config.NoValidateJSONMarshaler`: 编码JSON时避免验证JSON。封送拆收器
- `SearchOption.ValidateJSON`: 指示当`Get`时是否验证定位的JSON值
## 社区 ## 社区
Sonic 是 [CloudWeGo](https://www.cloudwego.io/) 下的一个子项目。我们致力于构建云原生生态系统。 Sonic 是 [CloudWeGo](https://www.cloudwego.io/) 下的一个子项目。我们致力于构建云原生生态系统。

82
vendor/github.com/bytedance/sonic/api.go generated vendored
View File

@@ -20,8 +20,19 @@ import (
`io` `io`
`github.com/bytedance/sonic/ast` `github.com/bytedance/sonic/ast`
`github.com/bytedance/sonic/internal/rt`
) )
const (
// UseStdJSON indicates you are using fallback implementation (encoding/json)
UseStdJSON = iota
// UseSonicJSON indicates you are using real sonic implementation
UseSonicJSON
)
// APIKind is the kind of API, 0 is std json, 1 is sonic.
const APIKind = apiKind
// Config is a combination of sonic/encoder.Options and sonic/decoder.Options // Config is a combination of sonic/encoder.Options and sonic/decoder.Options
type Config struct { type Config struct {
// EscapeHTML indicates encoder to escape all HTML characters // EscapeHTML indicates encoder to escape all HTML characters
@@ -66,13 +77,30 @@ type Config struct {
// CopyString indicates decoder to decode string values by copying instead of referring. // CopyString indicates decoder to decode string values by copying instead of referring.
CopyString bool CopyString bool
// ValidateString indicates decoder and encoder to valid string values: decoder will return errors // ValidateString indicates decoder and encoder to validate string values: decoder will return errors
// when unescaped control chars(\u0000-\u001f) in the string value of JSON. // when unescaped control chars(\u0000-\u001f) in the string value of JSON.
ValidateString bool ValidateString bool
// NoValidateJSONMarshaler indicates that the encoder should not validate the output string
// after encoding the JSONMarshaler to JSON.
NoValidateJSONMarshaler bool
// NoValidateJSONSkip indicates the decoder should not validate the JSON value when skipping it,
// such as unknown-fields, mismatched-type, redundant elements..
NoValidateJSONSkip bool
// NoEncoderNewline indicates that the encoder should not add a newline after every message
NoEncoderNewline bool
// Encode Infinity or Nan float into `null`, instead of returning an error.
EncodeNullForInfOrNan bool
// CaseSensitive indicates that the decoder should not ignore the case of object keys.
CaseSensitive bool
} }
var ( var (
// ConfigDefault is the default config of APIs, aiming at efficiency and safty. // ConfigDefault is the default config of APIs, aiming at efficiency and safety.
ConfigDefault = Config{}.Froze() ConfigDefault = Config{}.Froze()
// ConfigStd is the standard config of APIs, aiming at being compatible with encoding/json. // ConfigStd is the standard config of APIs, aiming at being compatible with encoding/json.
@@ -86,14 +114,15 @@ var (
// ConfigFastest is the fastest config of APIs, aiming at speed. // ConfigFastest is the fastest config of APIs, aiming at speed.
ConfigFastest = Config{ ConfigFastest = Config{
NoQuoteTextMarshaler: true, NoValidateJSONMarshaler: true,
NoValidateJSONSkip: true,
}.Froze() }.Froze()
) )
// API is a binding of specific config. // API is a binding of specific config.
// This interface is inspired by github.com/json-iterator/go, // This interface is inspired by github.com/json-iterator/go,
// and has same behaviors under equavilent config. // and has same behaviors under equivalent config.
type API interface { type API interface {
// MarshalToString returns the JSON encoding string of v // MarshalToString returns the JSON encoding string of v
MarshalToString(v interface{}) (string, error) MarshalToString(v interface{}) (string, error)
@@ -109,7 +138,7 @@ type API interface {
NewEncoder(writer io.Writer) Encoder NewEncoder(writer io.Writer) Encoder
// NewDecoder create a Decoder holding reader // NewDecoder create a Decoder holding reader
NewDecoder(reader io.Reader) Decoder NewDecoder(reader io.Reader) Decoder
// Valid validates the JSON-encoded bytes and reportes if it is valid // Valid validates the JSON-encoded bytes and reports if it is valid
Valid(data []byte) bool Valid(data []byte) bool
} }
@@ -148,6 +177,13 @@ func Marshal(val interface{}) ([]byte, error) {
return ConfigDefault.Marshal(val) return ConfigDefault.Marshal(val)
} }
// MarshalIndent is like Marshal but applies Indent to format the output.
// Each JSON element in the output will begin on a new line beginning with prefix
// followed by one or more copies of indent according to the indentation nesting.
func MarshalIndent(v interface{}, prefix, indent string) ([]byte, error) {
return ConfigDefault.MarshalIndent(v, prefix, indent)
}
// MarshalString returns the JSON encoding string of v. // MarshalString returns the JSON encoding string of v.
func MarshalString(val interface{}) (string, error) { func MarshalString(val interface{}) (string, error) {
return ConfigDefault.MarshalToString(val) return ConfigDefault.MarshalToString(val)
@@ -165,27 +201,49 @@ func UnmarshalString(buf string, val interface{}) error {
return ConfigDefault.UnmarshalFromString(buf, val) return ConfigDefault.UnmarshalFromString(buf, val)
} }
// Get searches the given path from json, // Get searches and locates the given path from src json,
// and returns its representing ast.Node. // and returns a ast.Node representing the partially json.
// //
// Each path arg must be integer or string: // Each path arg must be integer or string:
// - Integer is target index(>=0), means searching current node as array. // - Integer is target index(>=0), means searching current node as array.
// - String is target key, means searching current node as object. // - String is target key, means searching current node as object.
// //
// //
// Note, the api expects the json is well-formed at least, // Notice: It expects the src json is **Well-formed** and **Immutable** when calling,
// otherwise it may return unexpected result. // otherwise it may return unexpected result.
// Considering memory safety, the returned JSON is **Copied** from the input
func Get(src []byte, path ...interface{}) (ast.Node, error) { func Get(src []byte, path ...interface{}) (ast.Node, error) {
return GetFromString(string(src), path...) return GetCopyFromString(rt.Mem2Str(src), path...)
} }
// GetFromString is same with Get except src is string, //GetWithOptions searches and locates the given path from src json,
// which can reduce unnecessary memory copy. // with specific options of ast.Searcher
func GetWithOptions(src []byte, opts ast.SearchOptions, path ...interface{}) (ast.Node, error) {
s := ast.NewSearcher(rt.Mem2Str(src))
s.SearchOptions = opts
return s.GetByPath(path...)
}
// GetFromString is same with Get except src is string.
//
// WARNING: The returned JSON is **Referenced** from the input.
// Caching or long-time holding the returned node may cause OOM.
// If your src is big, consider use GetFromStringCopy().
func GetFromString(src string, path ...interface{}) (ast.Node, error) { func GetFromString(src string, path ...interface{}) (ast.Node, error) {
return ast.NewSearcher(src).GetByPath(path...) return ast.NewSearcher(src).GetByPath(path...)
} }
// GetCopyFromString is same with Get except src is string
func GetCopyFromString(src string, path ...interface{}) (ast.Node, error) {
return ast.NewSearcher(src).GetByPathCopy(path...)
}
// Valid reports whether data is a valid JSON encoding. // Valid reports whether data is a valid JSON encoding.
func Valid(data []byte) bool { func Valid(data []byte) bool {
return ConfigDefault.Valid(data) return ConfigDefault.Valid(data)
} }
// Valid reports whether data is a valid JSON encoding.
func ValidString(data string) bool {
return ConfigDefault.Valid(rt.Str2Mem(data))
}

157
vendor/github.com/bytedance/sonic/ast/api_amd64.go generated vendored
View File

@@ -1,157 +0,0 @@
// +build amd64,go1.16,!go1.22
/*
* Copyright 2022 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ast
import (
`runtime`
`unsafe`
`github.com/bytedance/sonic/encoder`
`github.com/bytedance/sonic/internal/native`
`github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/internal/rt`
uq `github.com/bytedance/sonic/unquote`
`github.com/chenzhuoyu/base64x`
)
var typeByte = rt.UnpackEface(byte(0)).Type
//go:nocheckptr
func quote(buf *[]byte, val string) {
*buf = append(*buf, '"')
if len(val) == 0 {
*buf = append(*buf, '"')
return
}
sp := rt.IndexChar(val, 0)
nb := len(val)
b := (*rt.GoSlice)(unsafe.Pointer(buf))
// input buffer
for nb > 0 {
// output buffer
dp := unsafe.Pointer(uintptr(b.Ptr) + uintptr(b.Len))
dn := b.Cap - b.Len
// call native.Quote, dn is byte count it outputs
ret := native.Quote(sp, nb, dp, &dn, 0)
// update *buf length
b.Len += dn
// no need more output
if ret >= 0 {
break
}
// double buf size
*b = growslice(typeByte, *b, b.Cap*2)
// ret is the complement of consumed input
ret = ^ret
// update input buffer
nb -= ret
sp = unsafe.Pointer(uintptr(sp) + uintptr(ret))
}
runtime.KeepAlive(buf)
runtime.KeepAlive(sp)
*buf = append(*buf, '"')
}
func unquote(src string) (string, types.ParsingError) {
return uq.String(src)
}
func decodeBase64(src string) ([]byte, error) {
return base64x.StdEncoding.DecodeString(src)
}
func encodeBase64(src []byte) string {
return base64x.StdEncoding.EncodeToString(src)
}
func (self *Parser) decodeValue() (val types.JsonState) {
sv := (*rt.GoString)(unsafe.Pointer(&self.s))
flag := types.F_USE_NUMBER
if self.dbuf != nil {
flag = 0
val.Dbuf = self.dbuf
val.Dcap = types.MaxDigitNums
}
self.p = native.Value(sv.Ptr, sv.Len, self.p, &val, uint64(flag))
return
}
func (self *Parser) skip() (int, types.ParsingError) {
fsm := types.NewStateMachine()
start := native.SkipOne(&self.s, &self.p, fsm, 0)
types.FreeStateMachine(fsm)
if start < 0 {
return self.p, types.ParsingError(-start)
}
return start, 0
}
func (self *Node) encodeInterface(buf *[]byte) error {
//WARN: NOT compatible with json.Encoder
return encoder.EncodeInto(buf, self.packAny(), 0)
}
func (self *Parser) skipFast() (int, types.ParsingError) {
start := native.SkipOneFast(&self.s, &self.p)
if start < 0 {
return self.p, types.ParsingError(-start)
}
return start, 0
}
func (self *Parser) getByPath(path ...interface{}) (int, types.ParsingError) {
fsm := types.NewStateMachine()
start := native.GetByPath(&self.s, &self.p, &path, fsm)
types.FreeStateMachine(fsm)
runtime.KeepAlive(path)
if start < 0 {
return self.p, types.ParsingError(-start)
}
return start, 0
}
func (self *Searcher) GetByPath(path ...interface{}) (Node, error) {
var err types.ParsingError
var start int
self.parser.p = 0
start, err = self.parser.getByPath(path...)
if err != 0 {
// for compatibility with old version
if err == types.ERR_NOT_FOUND {
return Node{}, ErrNotExist
}
if err == types.ERR_UNSUPPORT_TYPE {
panic("path must be either int(>=0) or string")
}
return Node{}, self.parser.syntaxError(err)
}
t := switchRawType(self.parser.s[start])
if t == _V_NONE {
return Node{}, self.parser.ExportError(err)
}
return newRawNode(self.parser.s[start:self.parser.p], t), nil
}

93
vendor/github.com/bytedance/sonic/ast/api_compat.go generated vendored
View File

@@ -1,57 +1,39 @@
// +build !amd64 !go1.16 go1.22 // +build !amd64,!arm64 go1.26 !go1.17 arm64,!go1.20
/* /*
* Copyright 2022 ByteDance Inc. * Copyright 2022 ByteDance Inc.
* *
* Licensed under the Apache License, Version 2.0 (the "License"); * Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License. * you may not use this file except in compliance with the License.
* You may obtain a copy of the License at * You may obtain a copy of the License at
* *
* http://www.apache.org/licenses/LICENSE-2.0 * http://www.apache.org/licenses/LICENSE-2.0
* *
* Unless required by applicable law or agreed to in writing, software * Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, * distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and * See the License for the specific language governing permissions and
* limitations under the License. * limitations under the License.
*/ */
package ast package ast
import ( import (
`encoding/base64`
`encoding/json` `encoding/json`
`fmt` `unicode/utf8`
`github.com/bytedance/sonic/internal/native/types` `github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/internal/rt` `github.com/bytedance/sonic/internal/compat`
) )
func init() { func init() {
println("WARNING: sonic only supports Go1.16~1.20 && CPU amd64, but your environment is not suitable") compat.Warn("sonic/ast")
} }
func quote(buf *[]byte, val string) { func quote(buf *[]byte, val string) {
quoteString(buf, val) quoteString(buf, val)
} }
func unquote(src string) (string, types.ParsingError) {
sp := rt.IndexChar(src, -1)
out, ok := unquoteBytes(rt.BytesFrom(sp, len(src)+2, len(src)+2))
if !ok {
return "", types.ERR_INVALID_ESCAPE
}
return rt.Mem2Str(out), 0
}
func decodeBase64(src string) ([]byte, error) {
return base64.StdEncoding.DecodeString(src)
}
func encodeBase64(src []byte) string {
return base64.StdEncoding.EncodeToString(src)
}
func (self *Parser) decodeValue() (val types.JsonState) { func (self *Parser) decodeValue() (val types.JsonState) {
e, v := decodeValue(self.s, self.p, self.dbuf == nil) e, v := decodeValue(self.s, self.p, self.dbuf == nil)
if e < 0 { if e < 0 {
@@ -88,37 +70,34 @@ func (self *Node) encodeInterface(buf *[]byte) error {
return nil return nil
} }
func (self *Searcher) GetByPath(path ...interface{}) (Node, error) { func (self *Parser) getByPath(validate bool, path ...interface{}) (int, types.ParsingError) {
self.parser.p = 0
var err types.ParsingError
for _, p := range path { for _, p := range path {
if idx, ok := p.(int); ok && idx >= 0 { if idx, ok := p.(int); ok && idx >= 0 {
if err = self.parser.searchIndex(idx); err != 0 { if err := self.searchIndex(idx); err != 0 {
return Node{}, self.parser.ExportError(err) return self.p, err
} }
} else if key, ok := p.(string); ok { } else if key, ok := p.(string); ok {
if err = self.parser.searchKey(key); err != 0 { if err := self.searchKey(key); err != 0 {
return Node{}, self.parser.ExportError(err) return self.p, err
} }
} else { } else {
panic("path must be either int(>=0) or string") panic("path must be either int(>=0) or string")
} }
} }
var start = self.parser.p var start int
if start, err = self.parser.skip(); err != 0 { var e types.ParsingError
return Node{}, self.parser.ExportError(err) if validate {
start, e = self.skip()
} else {
start, e = self.skipFast()
} }
ns := len(self.parser.s) if e != 0 {
if self.parser.p > ns || start >= ns || start>=self.parser.p { return self.p, e
return Node{}, fmt.Errorf("skip %d char out of json boundary", start)
} }
return start, 0
t := switchRawType(self.parser.s[start]) }
if t == _V_NONE {
return Node{}, self.parser.ExportError(err) func validate_utf8(str string) bool {
} return utf8.ValidString(str)
return newRawNode(self.parser.s[start:self.parser.p], t), nil
} }

128
vendor/github.com/bytedance/sonic/ast/decode.go generated vendored
View File

@@ -17,36 +17,35 @@
package ast package ast
import ( import (
`encoding/base64` "encoding/base64"
`runtime` "runtime"
`strconv` "strconv"
`unsafe` "unsafe"
`github.com/bytedance/sonic/internal/native/types` "github.com/bytedance/sonic/internal/native/types"
`github.com/bytedance/sonic/internal/rt` "github.com/bytedance/sonic/internal/rt"
"github.com/bytedance/sonic/internal/utils"
"github.com/bytedance/sonic/unquote"
) )
const _blankCharsMask = (1 << ' ') | (1 << '\t') | (1 << '\r') | (1 << '\n')
var bytesNull = []byte("null")
const ( const (
bytesNull = "null" strNull = "null"
bytesTrue = "true" bytesTrue = "true"
bytesFalse = "false" bytesFalse = "false"
bytesObject = "{}" bytesObject = "{}"
bytesArray = "[]" bytesArray = "[]"
) )
func isSpace(c byte) bool {
return (int(1<<c) & _blankCharsMask) != 0
}
//go:nocheckptr //go:nocheckptr
func skipBlank(src string, pos int) int { func skipBlank(src string, pos int) int {
se := uintptr(rt.IndexChar(src, len(src))) se := uintptr(rt.IndexChar(src, len(src)))
sp := uintptr(rt.IndexChar(src, pos)) sp := uintptr(rt.IndexChar(src, pos))
for sp < se { for sp < se {
if !isSpace(*(*byte)(unsafe.Pointer(sp))) { if !utils.IsSpace(*(*byte)(unsafe.Pointer(sp))) {
break break
} }
sp += 1 sp += 1
@@ -63,7 +62,7 @@ func decodeNull(src string, pos int) (ret int) {
if ret > len(src) { if ret > len(src) {
return -int(types.ERR_EOF) return -int(types.ERR_EOF)
} }
if src[pos:ret] == bytesNull { if src[pos:ret] == strNull {
return ret return ret
} else { } else {
return -int(types.ERR_INVALID_CHAR) return -int(types.ERR_INVALID_CHAR)
@@ -103,13 +102,13 @@ func decodeString(src string, pos int) (ret int, v string) {
return ret, v return ret, v
} }
vv, ok := unquoteBytes(rt.Str2Mem(src[pos:ret])) result, err := unquote.String(src[pos:ret])
if !ok { if err != 0 {
return -int(types.ERR_INVALID_CHAR), "" return -int(types.ERR_INVALID_CHAR), ""
} }
runtime.KeepAlive(src) runtime.KeepAlive(src)
return ret, rt.Mem2Str(vv) return ret, result
} }
func decodeBinary(src string, pos int) (ret int, v []byte) { func decodeBinary(src string, pos int) (ret int, v []byte) {
@@ -287,67 +286,7 @@ func decodeValue(src string, pos int, skipnum bool) (ret int, v types.JsonState)
//go:nocheckptr //go:nocheckptr
func skipNumber(src string, pos int) (ret int) { func skipNumber(src string, pos int) (ret int) {
sp := uintptr(rt.IndexChar(src, pos)) return utils.SkipNumber(src, pos)
se := uintptr(rt.IndexChar(src, len(src)))
if uintptr(sp) >= se {
return -int(types.ERR_EOF)
}
if c := *(*byte)(unsafe.Pointer(sp)); c == '-' {
sp += 1
}
ss := sp
var pointer bool
var exponent bool
var lastIsDigit bool
var nextNeedDigit = true
for ; sp < se; sp += uintptr(1) {
c := *(*byte)(unsafe.Pointer(sp))
if isDigit(c) {
lastIsDigit = true
nextNeedDigit = false
continue
} else if nextNeedDigit {
return -int(types.ERR_INVALID_CHAR)
} else if c == '.' {
if !lastIsDigit || pointer || exponent || sp == ss {
return -int(types.ERR_INVALID_CHAR)
}
pointer = true
lastIsDigit = false
nextNeedDigit = true
continue
} else if c == 'e' || c == 'E' {
if !lastIsDigit || exponent {
return -int(types.ERR_INVALID_CHAR)
}
if sp == se-1 {
return -int(types.ERR_EOF)
}
exponent = true
lastIsDigit = false
nextNeedDigit = false
continue
} else if c == '-' || c == '+' {
if prev := *(*byte)(unsafe.Pointer(sp - 1)); prev != 'e' && prev != 'E' {
return -int(types.ERR_INVALID_CHAR)
}
lastIsDigit = false
nextNeedDigit = true
continue
} else {
break
}
}
if nextNeedDigit {
return -int(types.ERR_EOF)
}
runtime.KeepAlive(src)
return int(uintptr(sp) - uintptr((*rt.GoString)(unsafe.Pointer(&src)).Ptr))
} }
//go:nocheckptr //go:nocheckptr
@@ -583,3 +522,36 @@ func skipArray(src string, pos int) (ret int, start int) {
pos++ pos++
} }
} }
// DecodeString decodes a JSON string from pos and return golang string.
// - needEsc indicates if to unescaped escaping chars
// - hasEsc tells if the returned string has escaping chars
// - validStr enables validating UTF8 charset
//
func _DecodeString(src string, pos int, needEsc bool, validStr bool) (v string, ret int, hasEsc bool) {
p := NewParserObj(src)
p.p = pos
switch val := p.decodeValue(); val.Vt {
case types.V_STRING:
str := p.s[val.Iv : p.p-1]
if validStr && !validate_utf8(str) {
return "", -int(types.ERR_INVALID_UTF8), false
}
/* fast path: no escape sequence */
if val.Ep == -1 {
return str, p.p, false
} else if !needEsc {
return str, p.p, true
}
/* unquote the string */
out, err := unquote.String(str)
/* check for errors */
if err != 0 {
return "", -int(err), true
} else {
return out, p.p, true
}
default:
return "", -int(_ERR_UNSUPPORT_TYPE), false
}
}

93
vendor/github.com/bytedance/sonic/ast/encode.go generated vendored
View File

@@ -17,12 +17,12 @@
package ast package ast
import ( import (
`sync` "sync"
`unicode/utf8` "unicode/utf8"
)
const ( "github.com/bytedance/gopkg/lang/dirtmake"
_MaxBuffer = 1024 // 1KB buffer size "github.com/bytedance/sonic/internal/rt"
"github.com/bytedance/sonic/option"
) )
func quoteString(e *[]byte, s string) { func quoteString(e *[]byte, s string) {
@@ -30,7 +30,7 @@ func quoteString(e *[]byte, s string) {
start := 0 start := 0
for i := 0; i < len(s); { for i := 0; i < len(s); {
if b := s[i]; b < utf8.RuneSelf { if b := s[i]; b < utf8.RuneSelf {
if safeSet[b] { if rt.SafeSet[b] {
i++ i++
continue continue
} }
@@ -54,8 +54,8 @@ func quoteString(e *[]byte, s string) {
// user-controlled strings are rendered into JSON // user-controlled strings are rendered into JSON
// and served to some browsers. // and served to some browsers.
*e = append(*e, `u00`...) *e = append(*e, `u00`...)
*e = append(*e, hex[b>>4]) *e = append(*e, rt.Hex[b>>4])
*e = append(*e, hex[b&0xF]) *e = append(*e, rt.Hex[b&0xF])
} }
i++ i++
start = i start = i
@@ -76,7 +76,7 @@ func quoteString(e *[]byte, s string) {
*e = append(*e, s[start:i]...) *e = append(*e, s[start:i]...)
} }
*e = append(*e, `\u202`...) *e = append(*e, `\u202`...)
*e = append(*e, hex[c&0xF]) *e = append(*e, rt.Hex[c&0xF])
i += size i += size
start = i start = i
continue continue
@@ -92,16 +92,29 @@ func quoteString(e *[]byte, s string) {
var bytesPool = sync.Pool{} var bytesPool = sync.Pool{}
func (self *Node) MarshalJSON() ([]byte, error) { func (self *Node) MarshalJSON() ([]byte, error) {
if self == nil {
return bytesNull, nil
}
// fast path for raw node
if self.isRaw() {
return rt.Str2Mem(self.toString()), nil
}
buf := newBuffer() buf := newBuffer()
err := self.encode(buf) err := self.encode(buf)
if err != nil { if err != nil {
freeBuffer(buf) freeBuffer(buf)
return nil, err return nil, err
} }
var ret []byte
ret := make([]byte, len(*buf)) if !rt.CanSizeResue(cap(*buf)) {
copy(ret, *buf) ret = *buf
freeBuffer(buf) } else {
ret = dirtmake.Bytes(len(*buf), len(*buf))
copy(ret, *buf)
freeBuffer(buf)
}
return ret, err return ret, err
} }
@@ -109,21 +122,24 @@ func newBuffer() *[]byte {
if ret := bytesPool.Get(); ret != nil { if ret := bytesPool.Get(); ret != nil {
return ret.(*[]byte) return ret.(*[]byte)
} else { } else {
buf := make([]byte, 0, _MaxBuffer) buf := make([]byte, 0, option.DefaultAstBufferSize)
return &buf return &buf
} }
} }
func freeBuffer(buf *[]byte) { func freeBuffer(buf *[]byte) {
if !rt.CanSizeResue(cap(*buf)) {
return
}
*buf = (*buf)[:0] *buf = (*buf)[:0]
bytesPool.Put(buf) bytesPool.Put(buf)
} }
func (self *Node) encode(buf *[]byte) error { func (self *Node) encode(buf *[]byte) error {
if self.IsRaw() { if self.isRaw() {
return self.encodeRaw(buf) return self.encodeRaw(buf)
} }
switch self.Type() { switch int(self.itype()) {
case V_NONE : return ErrNotExist case V_NONE : return ErrNotExist
case V_ERROR : return self.Check() case V_ERROR : return self.Check()
case V_NULL : return self.encodeNull(buf) case V_NULL : return self.encodeNull(buf)
@@ -139,16 +155,21 @@ func (self *Node) encode(buf *[]byte) error {
} }
func (self *Node) encodeRaw(buf *[]byte) error { func (self *Node) encodeRaw(buf *[]byte) error {
raw, err := self.Raw() lock := self.rlock()
if err != nil { if !self.isRaw() {
return err self.runlock()
return self.encode(buf)
}
raw := self.toString()
if lock {
self.runlock()
} }
*buf = append(*buf, raw...) *buf = append(*buf, raw...)
return nil return nil
} }
func (self *Node) encodeNull(buf *[]byte) error { func (self *Node) encodeNull(buf *[]byte) error {
*buf = append(*buf, bytesNull...) *buf = append(*buf, strNull...)
return nil return nil
} }
@@ -193,20 +214,9 @@ func (self *Node) encodeArray(buf *[]byte) error {
*buf = append(*buf, '[') *buf = append(*buf, '[')
var s = (*linkedNodes)(self.p)
var started bool var started bool
if nb > 0 { for i := 0; i < nb; i++ {
n := s.At(0) n := self.nodeAt(i)
if n.Exists() {
if err := n.encode(buf); err != nil {
return err
}
started = true
}
}
for i := 1; i < nb; i++ {
n := s.At(i)
if !n.Exists() { if !n.Exists() {
continue continue
} }
@@ -250,21 +260,10 @@ func (self *Node) encodeObject(buf *[]byte) error {
*buf = append(*buf, '{') *buf = append(*buf, '{')
var s = (*linkedPairs)(self.p)
var started bool var started bool
if nb > 0 { for i := 0; i < nb; i++ {
n := s.At(0) n := self.pairAt(i)
if n.Value.Exists() { if n == nil || !n.Value.Exists() {
if err := n.encode(buf); err != nil {
return err
}
started = true
}
}
for i := 1; i < nb; i++ {
n := s.At(i)
if !n.Value.Exists() {
continue continue
} }
if started { if started {

6
vendor/github.com/bytedance/sonic/ast/error.go generated vendored
View File

@@ -17,6 +17,10 @@ func newError(err types.ParsingError, msg string) *Node {
} }
} }
func newErrorPair(err SyntaxError) *Pair {
return &Pair{0, "", *newSyntaxError(err)}
}
// Error returns error message if the node is invalid // Error returns error message if the node is invalid
func (self Node) Error() string { func (self Node) Error() string {
if self.t != V_ERROR { if self.t != V_ERROR {
@@ -79,7 +83,7 @@ func (self SyntaxError) description() string {
/* check for empty source */ /* check for empty source */
if self.Src == "" { if self.Src == "" {
return fmt.Sprintf("no sources available: %#v", self) return fmt.Sprintf("no sources available, the input json is empty: %#v", self)
} }
/* prevent slicing before the beginning */ /* prevent slicing before the beginning */

39
vendor/github.com/bytedance/sonic/ast/iterator.go generated vendored
View File

@@ -17,30 +17,48 @@
package ast package ast
import ( import (
`fmt` "fmt"
`github.com/bytedance/sonic/internal/native/types` "github.com/bytedance/sonic/internal/caching"
"github.com/bytedance/sonic/internal/native/types"
) )
type Pair struct { type Pair struct {
hash uint64
Key string Key string
Value Node Value Node
} }
func NewPair(key string, val Node) Pair {
return Pair{
hash: caching.StrHash(key),
Key: key,
Value: val,
}
}
// Values returns iterator for array's children traversal // Values returns iterator for array's children traversal
func (self *Node) Values() (ListIterator, error) { func (self *Node) Values() (ListIterator, error) {
if err := self.should(types.V_ARRAY, "an array"); err != nil { if err := self.should(types.V_ARRAY); err != nil {
return ListIterator{}, err return ListIterator{}, err
} }
return ListIterator{Iterator{p: self}}, nil return self.values(), nil
}
func (self *Node) values() ListIterator {
return ListIterator{Iterator{p: self}}
} }
// Properties returns iterator for object's children traversal // Properties returns iterator for object's children traversal
func (self *Node) Properties() (ObjectIterator, error) { func (self *Node) Properties() (ObjectIterator, error) {
if err := self.should(types.V_OBJECT, "an object"); err != nil { if err := self.should(types.V_OBJECT); err != nil {
return ObjectIterator{}, err return ObjectIterator{}, err
} }
return ObjectIterator{Iterator{p: self}}, nil return self.properties(), nil
}
func (self *Node) properties() ObjectIterator {
return ObjectIterator{Iterator{p: self}}
} }
type Iterator struct { type Iterator struct {
@@ -114,7 +132,7 @@ next_start:
} else { } else {
n := self.p.pairAt(self.i) n := self.p.pairAt(self.i)
self.i++ self.i++
if !n.Value.Exists() { if n == nil || !n.Value.Exists() {
goto next_start goto next_start
} }
return n return n
@@ -155,11 +173,14 @@ type Scanner func(path Sequence, node *Node) bool
// ForEach scans one V_OBJECT node's children from JSON head to tail, // ForEach scans one V_OBJECT node's children from JSON head to tail,
// and pass the Sequence and Node of corresponding JSON value. // and pass the Sequence and Node of corresponding JSON value.
// //
// Especailly, if the node is not V_ARRAY or V_OBJECT, // Especially, if the node is not V_ARRAY or V_OBJECT,
// the node itself will be returned and Sequence.Index == -1. // the node itself will be returned and Sequence.Index == -1.
// //
// NOTICE: A unsetted node WON'T trigger sc, but its index still counts into Path.Index // NOTICE: An unset node WON'T trigger sc, but its index still counts into Path.Index
func (self *Node) ForEach(sc Scanner) error { func (self *Node) ForEach(sc Scanner) error {
if err := self.checkRaw(); err != nil {
return err
}
switch self.itype() { switch self.itype() {
case types.V_ARRAY: case types.V_ARRAY:
iter, err := self.Values() iter, err := self.Values()

716
vendor/github.com/bytedance/sonic/ast/node.go generated vendored
View File

File diff suppressed because it is too large Load Diff

171
vendor/github.com/bytedance/sonic/ast/parser.go generated vendored
View File

@@ -17,14 +17,18 @@
package ast package ast
import ( import (
`fmt` "fmt"
"sync"
"sync/atomic"
`github.com/bytedance/sonic/internal/native/types` "github.com/bytedance/sonic/internal/native/types"
`github.com/bytedance/sonic/internal/rt` "github.com/bytedance/sonic/internal/rt"
"github.com/bytedance/sonic/internal/utils"
"github.com/bytedance/sonic/unquote"
) )
const ( const (
_DEFAULT_NODE_CAP int = 8 _DEFAULT_NODE_CAP int = 16
_APPEND_GROW_SHIFT = 1 _APPEND_GROW_SHIFT = 1
) )
@@ -45,6 +49,7 @@ type Parser struct {
p int p int
s string s string
noLazy bool noLazy bool
loadOnce bool
skipValue bool skipValue bool
dbuf *byte dbuf *byte
} }
@@ -60,7 +65,7 @@ func (self *Parser) delim() types.ParsingError {
return types.ERR_EOF return types.ERR_EOF
} }
/* check for the delimtier */ /* check for the delimiter */
if self.s[p] != ':' { if self.s[p] != ':' {
return types.ERR_INVALID_CHAR return types.ERR_INVALID_CHAR
} }
@@ -79,7 +84,7 @@ func (self *Parser) object() types.ParsingError {
return types.ERR_EOF return types.ERR_EOF
} }
/* check for the delimtier */ /* check for the delimiter */
if self.s[p] != '{' { if self.s[p] != '{' {
return types.ERR_INVALID_CHAR return types.ERR_INVALID_CHAR
} }
@@ -98,7 +103,7 @@ func (self *Parser) array() types.ParsingError {
return types.ERR_EOF return types.ERR_EOF
} }
/* check for the delimtier */ /* check for the delimiter */
if self.s[p] != '[' { if self.s[p] != '[' {
return types.ERR_INVALID_CHAR return types.ERR_INVALID_CHAR
} }
@@ -110,11 +115,15 @@ func (self *Parser) array() types.ParsingError {
func (self *Parser) lspace(sp int) int { func (self *Parser) lspace(sp int) int {
ns := len(self.s) ns := len(self.s)
for ; sp<ns && isSpace(self.s[sp]); sp+=1 {} for ; sp<ns && utils.IsSpace(self.s[sp]); sp+=1 {}
return sp return sp
} }
func (self *Parser) backward() {
for ; self.p >= 0 && utils.IsSpace(self.s[self.p]); self.p-=1 {}
}
func (self *Parser) decodeArray(ret *linkedNodes) (Node, types.ParsingError) { func (self *Parser) decodeArray(ret *linkedNodes) (Node, types.ParsingError) {
sp := self.p sp := self.p
ns := len(self.s) ns := len(self.s)
@@ -148,7 +157,7 @@ func (self *Parser) decodeArray(ret *linkedNodes) (Node, types.ParsingError) {
if t == _V_NONE { if t == _V_NONE {
return Node{}, types.ERR_INVALID_CHAR return Node{}, types.ERR_INVALID_CHAR
} }
val = newRawNode(self.s[start:self.p], t) val = newRawNode(self.s[start:self.p], t, false)
}else{ }else{
/* decode the value */ /* decode the value */
if val, err = self.Parse(); err != 0 { if val, err = self.Parse(); err != 0 {
@@ -157,7 +166,7 @@ func (self *Parser) decodeArray(ret *linkedNodes) (Node, types.ParsingError) {
} }
/* add the value to result */ /* add the value to result */
ret.Add(val) ret.Push(val)
self.p = self.lspace(self.p) self.p = self.lspace(self.p)
/* check for EOF */ /* check for EOF */
@@ -210,7 +219,7 @@ func (self *Parser) decodeObject(ret *linkedPairs) (Node, types.ParsingError) {
/* check for escape sequence */ /* check for escape sequence */
if njs.Ep != -1 { if njs.Ep != -1 {
if key, err = unquote(key); err != 0 { if key, err = unquote.String(key); err != 0 {
return Node{}, err return Node{}, err
} }
} }
@@ -234,7 +243,7 @@ func (self *Parser) decodeObject(ret *linkedPairs) (Node, types.ParsingError) {
if t == _V_NONE { if t == _V_NONE {
return Node{}, types.ERR_INVALID_CHAR return Node{}, types.ERR_INVALID_CHAR
} }
val = newRawNode(self.s[start:self.p], t) val = newRawNode(self.s[start:self.p], t, false)
} else { } else {
/* decode the value */ /* decode the value */
if val, err = self.Parse(); err != 0 { if val, err = self.Parse(); err != 0 {
@@ -244,7 +253,7 @@ func (self *Parser) decodeObject(ret *linkedPairs) (Node, types.ParsingError) {
/* add the value to result */ /* add the value to result */
// FIXME: ret's address may change here, thus previous referred node in ret may be invalid !! // FIXME: ret's address may change here, thus previous referred node in ret may be invalid !!
ret.Add(Pair{Key: key, Value: val}) ret.Push(NewPair(key, val))
self.p = self.lspace(self.p) self.p = self.lspace(self.p)
/* check for EOF */ /* check for EOF */
@@ -275,7 +284,7 @@ func (self *Parser) decodeString(iv int64, ep int) (Node, types.ParsingError) {
} }
/* unquote the string */ /* unquote the string */
out, err := unquote(s) out, err := unquote.String(s)
/* check for errors */ /* check for errors */
if err != 0 { if err != 0 {
@@ -291,6 +300,10 @@ func (self *Parser) Pos() int {
return self.p return self.p
} }
// Parse returns a ast.Node representing the parser's JSON.
// NOTICE: the specific parsing lazy dependens parser's option
// It only parse first layer and first child for Object or Array be default
func (self *Parser) Parse() (Node, types.ParsingError) { func (self *Parser) Parse() (Node, types.ParsingError) {
switch val := self.decodeValue(); val.Vt { switch val := self.decodeValue(); val.Vt {
case types.V_EOF : return Node{}, types.ERR_EOF case types.V_EOF : return Node{}, types.ERR_EOF
@@ -299,22 +312,48 @@ func (self *Parser) Parse() (Node, types.ParsingError) {
case types.V_FALSE : return falseNode, 0 case types.V_FALSE : return falseNode, 0
case types.V_STRING : return self.decodeString(val.Iv, val.Ep) case types.V_STRING : return self.decodeString(val.Iv, val.Ep)
case types.V_ARRAY: case types.V_ARRAY:
s := self.p - 1;
if p := skipBlank(self.s, self.p); p >= self.p && self.s[p] == ']' { if p := skipBlank(self.s, self.p); p >= self.p && self.s[p] == ']' {
self.p = p + 1 self.p = p + 1
return Node{t: types.V_ARRAY}, 0 return Node{t: types.V_ARRAY}, 0
} }
if self.noLazy { if self.noLazy {
if self.loadOnce {
self.noLazy = false
}
return self.decodeArray(new(linkedNodes)) return self.decodeArray(new(linkedNodes))
} }
// NOTICE: loadOnce always keep raw json for object or array
if self.loadOnce {
self.p = s
s, e := self.skipFast()
if e != 0 {
return Node{}, e
}
return newRawNode(self.s[s:self.p], types.V_ARRAY, true), 0
}
return newLazyArray(self), 0 return newLazyArray(self), 0
case types.V_OBJECT: case types.V_OBJECT:
s := self.p - 1;
if p := skipBlank(self.s, self.p); p >= self.p && self.s[p] == '}' { if p := skipBlank(self.s, self.p); p >= self.p && self.s[p] == '}' {
self.p = p + 1 self.p = p + 1
return Node{t: types.V_OBJECT}, 0 return Node{t: types.V_OBJECT}, 0
} }
// NOTICE: loadOnce always keep raw json for object or array
if self.noLazy { if self.noLazy {
if self.loadOnce {
self.noLazy = false
}
return self.decodeObject(new(linkedPairs)) return self.decodeObject(new(linkedPairs))
} }
if self.loadOnce {
self.p = s
s, e := self.skipFast()
if e != 0 {
return Node{}, e
}
return newRawNode(self.s[s:self.p], types.V_OBJECT, true), 0
}
return newLazyObject(self), 0 return newLazyObject(self), 0
case types.V_DOUBLE : return NewNumber(self.s[val.Ep:self.p]), 0 case types.V_DOUBLE : return NewNumber(self.s[val.Ep:self.p]), 0
case types.V_INTEGER : return NewNumber(self.s[val.Ep:self.p]), 0 case types.V_INTEGER : return NewNumber(self.s[val.Ep:self.p]), 0
@@ -355,7 +394,7 @@ func (self *Parser) searchKey(match string) types.ParsingError {
/* check for escape sequence */ /* check for escape sequence */
if njs.Ep != -1 { if njs.Ep != -1 {
if key, err = unquote(key); err != 0 { if key, err = unquote.String(key); err != 0 {
return err return err
} }
} }
@@ -471,11 +510,11 @@ func (self *Node) skipNextNode() *Node {
if t == _V_NONE { if t == _V_NONE {
return newSyntaxError(parser.syntaxError(types.ERR_INVALID_CHAR)) return newSyntaxError(parser.syntaxError(types.ERR_INVALID_CHAR))
} }
val = newRawNode(parser.s[start:parser.p], t) val = newRawNode(parser.s[start:parser.p], t, false)
} }
/* add the value to result */ /* add the value to result */
ret.Add(val) ret.Push(val)
self.l++ self.l++
parser.p = parser.lspace(parser.p) parser.p = parser.lspace(parser.p)
@@ -510,7 +549,7 @@ func (self *Node) skipNextPair() (*Pair) {
/* check for EOF */ /* check for EOF */
if parser.p = parser.lspace(sp); parser.p >= ns { if parser.p = parser.lspace(sp); parser.p >= ns {
return &Pair{"", *newSyntaxError(parser.syntaxError(types.ERR_EOF))} return newErrorPair(parser.syntaxError(types.ERR_EOF))
} }
/* check for empty object */ /* check for empty object */
@@ -527,7 +566,7 @@ func (self *Node) skipNextPair() (*Pair) {
/* decode the key */ /* decode the key */
if njs = parser.decodeValue(); njs.Vt != types.V_STRING { if njs = parser.decodeValue(); njs.Vt != types.V_STRING {
return &Pair{"", *newSyntaxError(parser.syntaxError(types.ERR_INVALID_CHAR))} return newErrorPair(parser.syntaxError(types.ERR_INVALID_CHAR))
} }
/* extract the key */ /* extract the key */
@@ -536,35 +575,35 @@ func (self *Node) skipNextPair() (*Pair) {
/* check for escape sequence */ /* check for escape sequence */
if njs.Ep != -1 { if njs.Ep != -1 {
if key, err = unquote(key); err != 0 { if key, err = unquote.String(key); err != 0 {
return &Pair{key, *newSyntaxError(parser.syntaxError(err))} return newErrorPair(parser.syntaxError(err))
} }
} }
/* expect a ':' delimiter */ /* expect a ':' delimiter */
if err = parser.delim(); err != 0 { if err = parser.delim(); err != 0 {
return &Pair{key, *newSyntaxError(parser.syntaxError(err))} return newErrorPair(parser.syntaxError(err))
} }
/* skip the value */ /* skip the value */
if start, err := parser.skipFast(); err != 0 { if start, err := parser.skipFast(); err != 0 {
return &Pair{key, *newSyntaxError(parser.syntaxError(err))} return newErrorPair(parser.syntaxError(err))
} else { } else {
t := switchRawType(parser.s[start]) t := switchRawType(parser.s[start])
if t == _V_NONE { if t == _V_NONE {
return &Pair{key, *newSyntaxError(parser.syntaxError(types.ERR_INVALID_CHAR))} return newErrorPair(parser.syntaxError(types.ERR_INVALID_CHAR))
} }
val = newRawNode(parser.s[start:parser.p], t) val = newRawNode(parser.s[start:parser.p], t, false)
} }
/* add the value to result */ /* add the value to result */
ret.Add(Pair{Key: key, Value: val}) ret.Push(NewPair(key, val))
self.l++ self.l++
parser.p = parser.lspace(parser.p) parser.p = parser.lspace(parser.p)
/* check for EOF */ /* check for EOF */
if parser.p >= ns { if parser.p >= ns {
return &Pair{key, *newSyntaxError(parser.syntaxError(types.ERR_EOF))} return newErrorPair(parser.syntaxError(types.ERR_EOF))
} }
/* check for the next character */ /* check for the next character */
@@ -577,7 +616,7 @@ func (self *Node) skipNextPair() (*Pair) {
self.setObject(ret) self.setObject(ret)
return ret.At(ret.Len()-1) return ret.At(ret.Len()-1)
default: default:
return &Pair{key, *newSyntaxError(parser.syntaxError(types.ERR_INVALID_CHAR))} return newErrorPair(parser.syntaxError(types.ERR_INVALID_CHAR))
} }
} }
@@ -601,7 +640,7 @@ func Loads(src string) (int, interface{}, error) {
} }
} }
// LoadsUseNumber parse all json into interface{}, with numeric nodes casted to json.Number // LoadsUseNumber parse all json into interface{}, with numeric nodes cast to json.Number
func LoadsUseNumber(src string) (int, interface{}, error) { func LoadsUseNumber(src string) (int, interface{}, error) {
ps := &Parser{s: src} ps := &Parser{s: src}
np, err := ps.Parse() np, err := ps.Parse()
@@ -653,3 +692,77 @@ func (self *Parser) ExportError(err types.ParsingError) error {
Code: err, Code: err,
}.Description()) }.Description())
} }
func backward(src string, i int) int {
for ; i>=0 && utils.IsSpace(src[i]); i-- {}
return i
}
func newRawNode(str string, typ types.ValueType, lock bool) Node {
ret := Node{
t: typ | _V_RAW,
p: rt.StrPtr(str),
l: uint(len(str)),
}
if lock {
ret.m = new(sync.RWMutex)
}
return ret
}
var typeJumpTable = [256]types.ValueType{
'"' : types.V_STRING,
'-' : _V_NUMBER,
'0' : _V_NUMBER,
'1' : _V_NUMBER,
'2' : _V_NUMBER,
'3' : _V_NUMBER,
'4' : _V_NUMBER,
'5' : _V_NUMBER,
'6' : _V_NUMBER,
'7' : _V_NUMBER,
'8' : _V_NUMBER,
'9' : _V_NUMBER,
'[' : types.V_ARRAY,
'f' : types.V_FALSE,
'n' : types.V_NULL,
't' : types.V_TRUE,
'{' : types.V_OBJECT,
}
func switchRawType(c byte) types.ValueType {
return typeJumpTable[c]
}
func (self *Node) loadt() types.ValueType {
return (types.ValueType)(atomic.LoadInt64(&self.t))
}
func (self *Node) lock() bool {
if m := self.m; m != nil {
m.Lock()
return true
}
return false
}
func (self *Node) unlock() {
if m := self.m; m != nil {
m.Unlock()
}
}
func (self *Node) rlock() bool {
if m := self.m; m != nil {
m.RLock()
return true
}
return false
}
func (self *Node) runlock() {
if m := self.m; m != nil {
m.RUnlock()
}
}

127
vendor/github.com/bytedance/sonic/ast/search.go generated vendored
View File

@@ -16,8 +16,28 @@
package ast package ast
import (
`github.com/bytedance/sonic/internal/rt`
`github.com/bytedance/sonic/internal/native/types`
)
// SearchOptions controls Searcher's behavior
type SearchOptions struct {
// ValidateJSON indicates the searcher to validate the entire JSON
ValidateJSON bool
// CopyReturn indicates the searcher to copy the result JSON instead of refer from the input
// This can help to reduce memory usage if you cache the results
CopyReturn bool
// ConcurrentRead indicates the searcher to return a concurrently-READ-safe node,
// including: GetByPath/Get/Index/GetOrIndex/Int64/Bool/Float64/String/Number/Interface/Array/Map/Raw/MarshalJSON
ConcurrentRead bool
}
type Searcher struct { type Searcher struct {
parser Parser parser Parser
SearchOptions
} }
func NewSearcher(str string) *Searcher { func NewSearcher(str string) *Searcher {
@@ -26,5 +46,112 @@ func NewSearcher(str string) *Searcher {
s: str, s: str,
noLazy: false, noLazy: false,
}, },
SearchOptions: SearchOptions{
ValidateJSON: true,
},
} }
} }
// GetByPathCopy search in depth from top json and returns a **Copied** json node at the path location
func (self *Searcher) GetByPathCopy(path ...interface{}) (Node, error) {
self.CopyReturn = true
return self.getByPath(path...)
}
// GetByPathNoCopy search in depth from top json and returns a **Referenced** json node at the path location
//
// WARN: this search directly refer partial json from top json, which has faster speed,
// may consumes more memory.
func (self *Searcher) GetByPath(path ...interface{}) (Node, error) {
return self.getByPath(path...)
}
func (self *Searcher) getByPath(path ...interface{}) (Node, error) {
var err types.ParsingError
var start int
self.parser.p = 0
start, err = self.parser.getByPath(self.ValidateJSON, path...)
if err != 0 {
// for compatibility with old version
if err == types.ERR_NOT_FOUND {
return Node{}, ErrNotExist
}
if err == types.ERR_UNSUPPORT_TYPE {
panic("path must be either int(>=0) or string")
}
return Node{}, self.parser.syntaxError(err)
}
t := switchRawType(self.parser.s[start])
if t == _V_NONE {
return Node{}, self.parser.ExportError(err)
}
// copy string to reducing memory usage
var raw string
if self.CopyReturn {
raw = rt.Mem2Str([]byte(self.parser.s[start:self.parser.p]))
} else {
raw = self.parser.s[start:self.parser.p]
}
return newRawNode(raw, t, self.ConcurrentRead), nil
}
// GetByPath searches a path and returns relaction and types of target
func _GetByPath(src string, path ...interface{}) (start int, end int, typ int, err error) {
p := NewParserObj(src)
s, e := p.getByPath(false, path...)
if e != 0 {
// for compatibility with old version
if e == types.ERR_NOT_FOUND {
return -1, -1, 0, ErrNotExist
}
if e == types.ERR_UNSUPPORT_TYPE {
panic("path must be either int(>=0) or string")
}
return -1, -1, 0, p.syntaxError(e)
}
t := switchRawType(p.s[s])
if t == _V_NONE {
return -1, -1, 0, ErrNotExist
}
if t == _V_NUMBER {
p.p = 1 + backward(p.s, p.p-1)
}
return s, p.p, int(t), nil
}
// ValidSyntax check if a json has a valid JSON syntax,
// while not validate UTF-8 charset
func _ValidSyntax(json string) bool {
p := NewParserObj(json)
_, e := p.skip()
if e != 0 {
return false
}
if skipBlank(p.s, p.p) != -int(types.ERR_EOF) {
return false
}
return true
}
// SkipFast skip a json value in fast-skip algs,
// while not strictly validate JSON syntax and UTF-8 charset.
func _SkipFast(src string, i int) (int, int, error) {
p := NewParserObj(src)
p.p = i
s, e := p.skipFast()
if e != 0 {
return -1, -1, p.ExportError(e)
}
t := switchRawType(p.s[s])
if t == _V_NONE {
return -1, -1, ErrNotExist
}
if t == _V_NUMBER {
p.p = 1 + backward(p.s, p.p-1)
}
return s, p.p, nil
}

55
vendor/github.com/bytedance/sonic/ast/stubs_go115.go generated vendored
View File

@@ -1,55 +0,0 @@
// +build !go1.20
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ast
import (
`unsafe`
`unicode/utf8`
`github.com/bytedance/sonic/internal/rt`
)
//go:noescape
//go:linkname memmove runtime.memmove
//goland:noinspection GoUnusedParameter
func memmove(to unsafe.Pointer, from unsafe.Pointer, n uintptr)
//go:linkname unsafe_NewArray reflect.unsafe_NewArray
//goland:noinspection GoUnusedParameter
func unsafe_NewArray(typ *rt.GoType, n int) unsafe.Pointer
//go:linkname growslice runtime.growslice
//goland:noinspection GoUnusedParameter
func growslice(et *rt.GoType, old rt.GoSlice, cap int) rt.GoSlice
//go:nosplit
func mem2ptr(s []byte) unsafe.Pointer {
return (*rt.GoSlice)(unsafe.Pointer(&s)).Ptr
}
var (
//go:linkname safeSet encoding/json.safeSet
safeSet [utf8.RuneSelf]bool
//go:linkname hex encoding/json.hex
hex string
)
//go:linkname unquoteBytes encoding/json.unquoteBytes
func unquoteBytes(s []byte) (t []byte, ok bool)

55
vendor/github.com/bytedance/sonic/ast/stubs_go120.go generated vendored
View File

@@ -1,55 +0,0 @@
// +build go1.20
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package ast
import (
`unsafe`
`unicode/utf8`
`github.com/bytedance/sonic/internal/rt`
)
//go:noescape
//go:linkname memmove runtime.memmove
//goland:noinspection GoUnusedParameter
func memmove(to unsafe.Pointer, from unsafe.Pointer, n uintptr)
//go:linkname unsafe_NewArray reflect.unsafe_NewArray
//goland:noinspection GoUnusedParameter
func unsafe_NewArray(typ *rt.GoType, n int) unsafe.Pointer
//go:linkname growslice reflect.growslice
//goland:noinspection GoUnusedParameter
func growslice(et *rt.GoType, old rt.GoSlice, cap int) rt.GoSlice
//go:nosplit
func mem2ptr(s []byte) unsafe.Pointer {
return (*rt.GoSlice)(unsafe.Pointer(&s)).Ptr
}
var (
//go:linkname safeSet encoding/json.safeSet
safeSet [utf8.RuneSelf]bool
//go:linkname hex encoding/json.hex
hex string
)
//go:linkname unquoteBytes encoding/json.unquoteBytes
func unquoteBytes(s []byte) (t []byte, ok bool)

16
vendor/github.com/bytedance/sonic/compat.go generated vendored
View File

@@ -1,4 +1,4 @@
// +build !amd64 !go1.16 go1.22 // +build !amd64,!arm64 go1.26 !go1.17 arm64,!go1.20
/* /*
* Copyright 2021 ByteDance Inc. * Copyright 2021 ByteDance Inc.
@@ -27,6 +27,8 @@ import (
`github.com/bytedance/sonic/option` `github.com/bytedance/sonic/option`
) )
const apiKind = UseStdJSON
type frozenConfig struct { type frozenConfig struct {
Config Config
} }
@@ -85,7 +87,17 @@ func (cfg frozenConfig) UnmarshalFromString(buf string, val interface{}) error {
if cfg.DisallowUnknownFields { if cfg.DisallowUnknownFields {
dec.DisallowUnknownFields() dec.DisallowUnknownFields()
} }
return dec.Decode(val) err := dec.Decode(val)
if err != nil {
return err
}
// check the trailing chars
offset := dec.InputOffset()
if t, err := dec.Token(); !(t == nil && err == io.EOF) {
return &json.SyntaxError{ Offset: offset}
}
return nil
} }
// Unmarshal is implemented by sonic // Unmarshal is implemented by sonic

68
vendor/github.com/bytedance/sonic/decoder/decoder_amd64.go generated vendored
View File

@@ -1,68 +0,0 @@
// +build amd64,go1.16,!go1.22
/*
* Copyright 2023 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package decoder
import (
`github.com/bytedance/sonic/internal/decoder`
)
// Decoder is the decoder context object
type Decoder = decoder.Decoder
// SyntaxError represents json syntax error
type SyntaxError = decoder.SyntaxError
// MismatchTypeError represents dismatching between json and object
type MismatchTypeError = decoder.MismatchTypeError
// Options for decode.
type Options = decoder.Options
const (
OptionUseInt64 Options = decoder.OptionUseInt64
OptionUseNumber Options = decoder.OptionUseNumber
OptionUseUnicodeErrors Options = decoder.OptionUseUnicodeErrors
OptionDisableUnknown Options = decoder.OptionDisableUnknown
OptionCopyString Options = decoder.OptionCopyString
OptionValidateString Options = decoder.OptionValidateString
)
// StreamDecoder is the decoder context object for streaming input.
type StreamDecoder = decoder.StreamDecoder
var (
// NewDecoder creates a new decoder instance.
NewDecoder = decoder.NewDecoder
// NewStreamDecoder adapts to encoding/json.NewDecoder API.
//
// NewStreamDecoder returns a new decoder that reads from r.
NewStreamDecoder = decoder.NewStreamDecoder
// Pretouch compiles vt ahead-of-time to avoid JIT compilation on-the-fly, in
// order to reduce the first-hit latency.
//
// Opts are the compile options, for example, "option.WithCompileRecursiveDepth" is
// a compile option to set the depth of recursive compile for the nested struct type.
Pretouch = decoder.Pretouch
// Skip skips only one json value, and returns first non-blank character position and its ending position if it is valid.
// Otherwise, returns negative error code using start and invalid character position using end
Skip = decoder.Skip
)

45
vendor/github.com/bytedance/sonic/decoder/decoder_compat.go generated vendored
View File

@@ -1,4 +1,5 @@
// +build !amd64 !go1.16 go1.22 //go:build (!amd64 && !arm64) || go1.26 || !go1.17 || (arm64 && !go1.20)
// +build !amd64,!arm64 go1.26 !go1.17 arm64,!go1.20
/* /*
* Copyright 2023 ByteDance Inc. * Copyright 2023 ByteDance Inc.
@@ -19,29 +20,33 @@
package decoder package decoder
import ( import (
`bytes` "bytes"
`encoding/json` "encoding/json"
`io` "io"
`reflect` "reflect"
`unsafe` "unsafe"
`github.com/bytedance/sonic/internal/native/types` "github.com/bytedance/sonic/internal/decoder/consts"
`github.com/bytedance/sonic/option` "github.com/bytedance/sonic/internal/native/types"
"github.com/bytedance/sonic/option"
"github.com/bytedance/sonic/internal/compat"
) )
func init() { func init() {
println("WARNING: sonic only supports Go1.16~1.20 && CPU amd64, but your environment is not suitable") compat.Warn("sonic/decoder")
} }
const ( const (
_F_use_int64 = 0 _F_use_int64 = consts.F_use_int64
_F_disable_urc = 2 _F_disable_urc = consts.F_disable_unknown
_F_disable_unknown = 3 _F_disable_unknown = consts.F_disable_unknown
_F_copy_string = 4 _F_copy_string = consts.F_copy_string
_F_use_number = types.B_USE_NUMBER _F_use_number = consts.F_use_number
_F_validate_string = types.B_VALIDATE_STRING _F_validate_string = consts.F_validate_string
_F_allow_control = types.B_ALLOW_CONTROL _F_allow_control = consts.F_allow_control
_F_no_validate_json = consts.F_no_validate_json
_F_case_sensitive = consts.F_case_sensitive
) )
type Options uint64 type Options uint64
@@ -53,6 +58,8 @@ const (
OptionDisableUnknown Options = 1 << _F_disable_unknown OptionDisableUnknown Options = 1 << _F_disable_unknown
OptionCopyString Options = 1 << _F_copy_string OptionCopyString Options = 1 << _F_copy_string
OptionValidateString Options = 1 << _F_validate_string OptionValidateString Options = 1 << _F_validate_string
OptionNoValidateJSON Options = 1 << _F_no_validate_json
OptionCaseSensitive Options = 1 << _F_case_sensitive
) )
func (self *Decoder) SetOptions(opts Options) { func (self *Decoder) SetOptions(opts Options) {
@@ -112,10 +119,10 @@ func (self *Decoder) CheckTrailings() error {
func (self *Decoder) Decode(val interface{}) error { func (self *Decoder) Decode(val interface{}) error {
r := bytes.NewBufferString(self.s) r := bytes.NewBufferString(self.s)
dec := json.NewDecoder(r) dec := json.NewDecoder(r)
if (self.f | uint64(OptionUseNumber)) != 0 { if (self.f & uint64(OptionUseNumber)) != 0 {
dec.UseNumber() dec.UseNumber()
} }
if (self.f | uint64(OptionDisableUnknown)) != 0 { if (self.f & uint64(OptionDisableUnknown)) != 0 {
dec.DisallowUnknownFields() dec.DisallowUnknownFields()
} }
return dec.Decode(val) return dec.Decode(val)
@@ -190,5 +197,5 @@ func (s SyntaxError) Error() string {
return (*json.SyntaxError)(unsafe.Pointer(&s)).Error() return (*json.SyntaxError)(unsafe.Pointer(&s)).Error()
} }
// MismatchTypeError represents dismatching between json and object // MismatchTypeError represents mismatching between json and object
type MismatchTypeError json.UnmarshalTypeError type MismatchTypeError json.UnmarshalTypeError

108
vendor/github.com/bytedance/sonic/encoder/encoder_amd64.go generated vendored
View File

@@ -1,108 +0,0 @@
// +build amd64,go1.16,!go1.22
/*
* Copyright 2023 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package encoder
import (
`github.com/bytedance/sonic/internal/encoder`
)
// Encoder represents a specific set of encoder configurations.
type Encoder = encoder.Encoder
// StreamEncoder uses io.Writer as input.
type StreamEncoder = encoder.StreamEncoder
// Options is a set of encoding options.
type Options = encoder.Options
const (
// SortMapKeys indicates that the keys of a map needs to be sorted
// before serializing into JSON.
// WARNING: This hurts performance A LOT, USE WITH CARE.
SortMapKeys Options = encoder.SortMapKeys
// EscapeHTML indicates encoder to escape all HTML characters
// after serializing into JSON (see https://pkg.go.dev/encoding/json#HTMLEscape).
// WARNING: This hurts performance A LOT, USE WITH CARE.
EscapeHTML Options = encoder.EscapeHTML
// CompactMarshaler indicates that the output JSON from json.Marshaler
// is always compact and needs no validation
CompactMarshaler Options = encoder.CompactMarshaler
// NoQuoteTextMarshaler indicates that the output text from encoding.TextMarshaler
// is always escaped string and needs no quoting
NoQuoteTextMarshaler Options = encoder.NoQuoteTextMarshaler
// NoNullSliceOrMap indicates all empty Array or Object are encoded as '[]' or '{}',
// instead of 'null'
NoNullSliceOrMap Options = encoder.NoNullSliceOrMap
// ValidateString indicates that encoder should validate the input string
// before encoding it into JSON.
ValidateString Options = encoder.ValidateString
// CompatibleWithStd is used to be compatible with std encoder.
CompatibleWithStd Options = encoder.CompatibleWithStd
)
var (
// Encode returns the JSON encoding of val, encoded with opts.
Encode = encoder.Encode
// EncodeInto is like Encode but uses a user-supplied buffer instead of allocating a new one.
EncodeIndented = encoder.EncodeIndented
// EncodeIndented is like Encode but applies Indent to format the output.
// Each JSON element in the output will begin on a new line beginning with prefix
// followed by one or more copies of indent according to the indentation nesting.
EncodeInto = encoder.EncodeInto
// HTMLEscape appends to dst the JSON-encoded src with <, >, &, U+2028 and U+2029
// characters inside string literals changed to \u003c, \u003e, \u0026, \u2028, \u2029
// so that the JSON will be safe to embed inside HTML <script> tags.
// For historical reasons, web browsers don't honor standard HTML
// escaping within <script> tags, so an alternative JSON encoding must
// be used.
HTMLEscape = encoder.HTMLEscape
// Pretouch compiles vt ahead-of-time to avoid JIT compilation on-the-fly, in
// order to reduce the first-hit latency.
//
// Opts are the compile options, for example, "option.WithCompileRecursiveDepth" is
// a compile option to set the depth of recursive compile for the nested struct type.
Pretouch = encoder.Pretouch
// Quote returns the JSON-quoted version of s.
Quote = encoder.Quote
// Valid validates json and returns first non-blank character position,
// if it is only one valid json value.
// Otherwise returns invalid character position using start.
//
// Note: it does not check for the invalid UTF-8 characters.
Valid = encoder.Valid
// NewStreamEncoder adapts to encoding/json.NewDecoder API.
//
// NewStreamEncoder returns a new encoder that write to w.
NewStreamEncoder = encoder.NewStreamEncoder
)

59
vendor/github.com/bytedance/sonic/encoder/encoder_compat.go generated vendored
View File

@@ -1,4 +1,4 @@
// +build !amd64 !go1.16 go1.22 // +build !amd64,!arm64 go1.26 !go1.17 arm64,!go1.20
/* /*
* Copyright 2023 ByteDance Inc. * Copyright 2023 ByteDance Inc.
@@ -19,18 +19,22 @@
package encoder package encoder
import ( import (
`io` `io`
`bytes` `bytes`
`encoding/json` `encoding/json`
`reflect` `reflect`
`github.com/bytedance/sonic/option` `github.com/bytedance/sonic/option`
`github.com/bytedance/sonic/internal/compat`
) )
func init() { func init() {
println("WARNING: sonic only supports Go1.16~1.20 && CPU amd64, but your environment is not suitable") compat.Warn("sonic/encoder")
} }
// EnableFallback indicates if encoder use fallback
const EnableFallback = true
// Options is a set of encoding options. // Options is a set of encoding options.
type Options uint64 type Options uint64
@@ -41,6 +45,8 @@ const (
bitNoQuoteTextMarshaler bitNoQuoteTextMarshaler
bitNoNullSliceOrMap bitNoNullSliceOrMap
bitValidateString bitValidateString
bitNoValidateJSONMarshaler
bitNoEncoderNewline
// used for recursive compile // used for recursive compile
bitPointerValue = 63 bitPointerValue = 63
@@ -73,6 +79,13 @@ const (
// before encoding it into JSON. // before encoding it into JSON.
ValidateString Options = 1 << bitValidateString ValidateString Options = 1 << bitValidateString
// NoValidateJSONMarshaler indicates that the encoder should not validate the output string
// after encoding the JSONMarshaler to JSON.
NoValidateJSONMarshaler Options = 1 << bitNoValidateJSONMarshaler
// NoEncoderNewline indicates that the encoder should not add a newline after every message
NoEncoderNewline Options = 1 << bitNoEncoderNewline
// CompatibleWithStd is used to be compatible with std encoder. // CompatibleWithStd is used to be compatible with std encoder.
CompatibleWithStd Options = SortMapKeys | EscapeHTML | CompactMarshaler CompatibleWithStd Options = SortMapKeys | EscapeHTML | CompactMarshaler
) )
@@ -116,6 +129,24 @@ func (self *Encoder) SetValidateString(f bool) {
} }
} }
// SetNoValidateJSONMarshaler specifies if option NoValidateJSONMarshaler opens
func (self *Encoder) SetNoValidateJSONMarshaler(f bool) {
if f {
self.Opts |= NoValidateJSONMarshaler
} else {
self.Opts &= ^NoValidateJSONMarshaler
}
}
// SetNoEncoderNewline specifies if option NoEncoderNewline opens
func (self *Encoder) SetNoEncoderNewline(f bool) {
if f {
self.Opts |= NoEncoderNewline
} else {
self.Opts &= ^NoEncoderNewline
}
}
// SetCompactMarshaler specifies if option CompactMarshaler opens // SetCompactMarshaler specifies if option CompactMarshaler opens
func (self *Encoder) SetCompactMarshaler(f bool) { func (self *Encoder) SetCompactMarshaler(f bool) {
if f { if f {
@@ -161,15 +192,19 @@ func Encode(val interface{}, opts Options) ([]byte, error) {
// EncodeInto is like Encode but uses a user-supplied buffer instead of allocating // EncodeInto is like Encode but uses a user-supplied buffer instead of allocating
// a new one. // a new one.
func EncodeInto(buf *[]byte, val interface{}, opts Options) error { func EncodeInto(buf *[]byte, val interface{}, opts Options) error {
if buf == nil { if buf == nil {
panic("user-supplied buffer buf is nil") panic("user-supplied buffer buf is nil")
} }
w := bytes.NewBuffer(*buf) w := bytes.NewBuffer(*buf)
enc := json.NewEncoder(w) enc := json.NewEncoder(w)
enc.SetEscapeHTML((opts & EscapeHTML) != 0) enc.SetEscapeHTML((opts & EscapeHTML) != 0)
err := enc.Encode(val) err := enc.Encode(val)
*buf = w.Bytes() *buf = w.Bytes()
return err l := len(*buf)
if l > 0 && (opts & NoEncoderNewline != 0) && (*buf)[l-1] == '\n' {
*buf = (*buf)[:l-1]
}
return err
} }
// HTMLEscape appends to dst the JSON-encoded src with <, >, &, U+2028 and U+2029 // HTMLEscape appends to dst the JSON-encoded src with <, >, &, U+2028 and U+2029

8
vendor/github.com/bytedance/sonic/internal/caching/hashing.go generated vendored
View File

@@ -23,16 +23,12 @@ import (
) )
var ( var (
V_strhash = rt.UnpackEface(strhash) V_strhash = rt.UnpackEface(rt.Strhash)
S_strhash = *(*uintptr)(V_strhash.Value) S_strhash = *(*uintptr)(V_strhash.Value)
) )
//go:noescape
//go:linkname strhash runtime.strhash
func strhash(_ unsafe.Pointer, _ uintptr) uintptr
func StrHash(s string) uint64 { func StrHash(s string) uint64 {
if v := strhash(unsafe.Pointer(&s), 0); v == 0 { if v := rt.Strhash(unsafe.Pointer(&s), 0); v == 0 {
return 1 return 1
} else { } else {
return uint64(v) return uint64(v)

4
vendor/github.com/bytedance/sonic/internal/cpu/features.go generated vendored
View File

@@ -24,7 +24,6 @@ import (
) )
var ( var (
HasAVX = cpuid.CPU.Has(cpuid.AVX)
HasAVX2 = cpuid.CPU.Has(cpuid.AVX2) HasAVX2 = cpuid.CPU.Has(cpuid.AVX2)
HasSSE = cpuid.CPU.Has(cpuid.SSE) HasSSE = cpuid.CPU.Has(cpuid.SSE)
) )
@@ -33,7 +32,8 @@ func init() {
switch v := os.Getenv("SONIC_MODE"); v { switch v := os.Getenv("SONIC_MODE"); v {
case "" : break case "" : break
case "auto" : break case "auto" : break
case "noavx" : HasAVX = false; fallthrough case "noavx" : HasAVX2 = false
// will also disable avx, act as `noavx`, we remain it to make sure forward compatibility
case "noavx2" : HasAVX2 = false case "noavx2" : HasAVX2 = false
default : panic(fmt.Sprintf("invalid mode: '%s', should be one of 'auto', 'noavx', 'noavx2'", v)) default : panic(fmt.Sprintf("invalid mode: '%s', should be one of 'auto', 'noavx', 'noavx2'", v))
} }

0
vendor/github.com/bytedance/sonic/internal/decoder/asm.s generated vendored
View File

1155
vendor/github.com/bytedance/sonic/internal/decoder/compiler.go generated vendored
View File

File diff suppressed because it is too large Load Diff

70
vendor/github.com/bytedance/sonic/internal/decoder/debug.go generated vendored
View File

@@ -1,70 +0,0 @@
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package decoder
import (
`os`
`runtime`
`runtime/debug`
`strings`
`github.com/bytedance/sonic/internal/jit`
)
var (
debugSyncGC = os.Getenv("SONIC_SYNC_GC") != ""
debugAsyncGC = os.Getenv("SONIC_NO_ASYNC_GC") == ""
)
var (
_Instr_End _Instr = newInsOp(_OP_nil_1)
_F_gc = jit.Func(runtime.GC)
_F_force_gc = jit.Func(debug.FreeOSMemory)
_F_println = jit.Func(println_wrapper)
_F_print = jit.Func(print)
)
func println_wrapper(i int, op1 int, op2 int){
println(i, " Intrs ", op1, _OpNames[op1], "next: ", op2, _OpNames[op2])
}
func print(i int){
println(i)
}
func (self *_Assembler) force_gc() {
self.call_go(_F_gc)
self.call_go(_F_force_gc)
}
func (self *_Assembler) debug_instr(i int, v *_Instr) {
if debugSyncGC {
if (i+1 == len(self.p)) {
self.print_gc(i, v, &_Instr_End)
} else {
next := &(self.p[i+1])
self.print_gc(i, v, next)
name := _OpNames[next.op()]
if strings.Contains(name, "save") {
return
}
}
self.force_gc()
}
}

255
vendor/github.com/bytedance/sonic/internal/decoder/decoder.go generated vendored
View File

@@ -1,255 +0,0 @@
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package decoder
import (
`unsafe`
`encoding/json`
`reflect`
`runtime`
`github.com/bytedance/sonic/internal/native`
`github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/internal/rt`
`github.com/bytedance/sonic/option`
`github.com/bytedance/sonic/utf8`
)
const (
_F_use_int64 = 0
_F_disable_urc = 2
_F_disable_unknown = 3
_F_copy_string = 4
_F_use_number = types.B_USE_NUMBER
_F_validate_string = types.B_VALIDATE_STRING
_F_allow_control = types.B_ALLOW_CONTROL
)
type Options uint64
const (
OptionUseInt64 Options = 1 << _F_use_int64
OptionUseNumber Options = 1 << _F_use_number
OptionUseUnicodeErrors Options = 1 << _F_disable_urc
OptionDisableUnknown Options = 1 << _F_disable_unknown
OptionCopyString Options = 1 << _F_copy_string
OptionValidateString Options = 1 << _F_validate_string
)
func (self *Decoder) SetOptions(opts Options) {
if (opts & OptionUseNumber != 0) && (opts & OptionUseInt64 != 0) {
panic("can't set OptionUseInt64 and OptionUseNumber both!")
}
self.f = uint64(opts)
}
// Decoder is the decoder context object
type Decoder struct {
i int
f uint64
s string
}
// NewDecoder creates a new decoder instance.
func NewDecoder(s string) *Decoder {
return &Decoder{s: s}
}
// Pos returns the current decoding position.
func (self *Decoder) Pos() int {
return self.i
}
func (self *Decoder) Reset(s string) {
self.s = s
self.i = 0
// self.f = 0
}
func (self *Decoder) CheckTrailings() error {
pos := self.i
buf := self.s
/* skip all the trailing spaces */
if pos != len(buf) {
for pos < len(buf) && (types.SPACE_MASK & (1 << buf[pos])) != 0 {
pos++
}
}
/* then it must be at EOF */
if pos == len(buf) {
return nil
}
/* junk after JSON value */
return SyntaxError {
Src : buf,
Pos : pos,
Code : types.ERR_INVALID_CHAR,
}
}
// Decode parses the JSON-encoded data from current position and stores the result
// in the value pointed to by val.
func (self *Decoder) Decode(val interface{}) error {
/* validate json if needed */
if (self.f & (1 << _F_validate_string)) != 0 && !utf8.ValidateString(self.s){
dbuf := utf8.CorrectWith(nil, rt.Str2Mem(self.s), "\ufffd")
self.s = rt.Mem2Str(dbuf)
}
vv := rt.UnpackEface(val)
vp := vv.Value
/* check for nil type */
if vv.Type == nil {
return &json.InvalidUnmarshalError{}
}
/* must be a non-nil pointer */
if vp == nil || vv.Type.Kind() != reflect.Ptr {
return &json.InvalidUnmarshalError{Type: vv.Type.Pack()}
}
etp := rt.PtrElem(vv.Type)
/* check the defined pointer type for issue 379 */
if vv.Type.IsNamed() {
newp := vp
etp = vv.Type
vp = unsafe.Pointer(&newp)
}
/* create a new stack, and call the decoder */
sb := newStack()
nb, err := decodeTypedPointer(self.s, self.i, etp, vp, sb, self.f)
/* return the stack back */
self.i = nb
freeStack(sb)
/* avoid GC ahead */
runtime.KeepAlive(vv)
return err
}
// UseInt64 indicates the Decoder to unmarshal an integer into an interface{} as an
// int64 instead of as a float64.
func (self *Decoder) UseInt64() {
self.f |= 1 << _F_use_int64
self.f &^= 1 << _F_use_number
}
// UseNumber indicates the Decoder to unmarshal a number into an interface{} as a
// json.Number instead of as a float64.
func (self *Decoder) UseNumber() {
self.f &^= 1 << _F_use_int64
self.f |= 1 << _F_use_number
}
// UseUnicodeErrors indicates the Decoder to return an error when encounter invalid
// UTF-8 escape sequences.
func (self *Decoder) UseUnicodeErrors() {
self.f |= 1 << _F_disable_urc
}
// DisallowUnknownFields indicates the Decoder to return an error when the destination
// is a struct and the input contains object keys which do not match any
// non-ignored, exported fields in the destination.
func (self *Decoder) DisallowUnknownFields() {
self.f |= 1 << _F_disable_unknown
}
// CopyString indicates the Decoder to decode string values by copying instead of referring.
func (self *Decoder) CopyString() {
self.f |= 1 << _F_copy_string
}
// ValidateString causes the Decoder to validate string values when decoding string value
// in JSON. Validation is that, returning error when unescaped control chars(0x00-0x1f) or
// invalid UTF-8 chars in the string value of JSON.
func (self *Decoder) ValidateString() {
self.f |= 1 << _F_validate_string
}
// Pretouch compiles vt ahead-of-time to avoid JIT compilation on-the-fly, in
// order to reduce the first-hit latency.
//
// Opts are the compile options, for example, "option.WithCompileRecursiveDepth" is
// a compile option to set the depth of recursive compile for the nested struct type.
func Pretouch(vt reflect.Type, opts ...option.CompileOption) error {
cfg := option.DefaultCompileOptions()
for _, opt := range opts {
opt(&cfg)
}
return pretouchRec(map[reflect.Type]bool{vt:true}, cfg)
}
func pretouchType(_vt reflect.Type, opts option.CompileOptions) (map[reflect.Type]bool, error) {
/* compile function */
compiler := newCompiler().apply(opts)
decoder := func(vt *rt.GoType, _ ...interface{}) (interface{}, error) {
if pp, err := compiler.compile(_vt); err != nil {
return nil, err
} else {
as := newAssembler(pp)
as.name = _vt.String()
return as.Load(), nil
}
}
/* find or compile */
vt := rt.UnpackType(_vt)
if val := programCache.Get(vt); val != nil {
return nil, nil
} else if _, err := programCache.Compute(vt, decoder); err == nil {
return compiler.rec, nil
} else {
return nil, err
}
}
func pretouchRec(vtm map[reflect.Type]bool, opts option.CompileOptions) error {
if opts.RecursiveDepth < 0 || len(vtm) == 0 {
return nil
}
next := make(map[reflect.Type]bool)
for vt := range(vtm) {
sub, err := pretouchType(vt, opts)
if err != nil {
return err
}
for svt := range(sub) {
next[svt] = true
}
}
opts.RecursiveDepth -= 1
return pretouchRec(next, opts)
}
// Skip skips only one json value, and returns first non-blank character position and its ending position if it is valid.
// Otherwise, returns negative error code using start and invalid character position using end
func Skip(data []byte) (start int, end int) {
s := rt.Mem2Str(data)
p := 0
m := types.NewStateMachine()
ret := native.SkipOne(&s, &p, m, uint64(0))
types.FreeStateMachine(m)
return ret, p
}

191
vendor/github.com/bytedance/sonic/internal/decoder/errors.go generated vendored
View File

@@ -1,191 +0,0 @@
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package decoder
import (
`encoding/json`
`errors`
`fmt`
`reflect`
`strconv`
`strings`
`github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/internal/rt`
)
type SyntaxError struct {
Pos int
Src string
Code types.ParsingError
Msg string
}
func (self SyntaxError) Error() string {
return fmt.Sprintf("%q", self.Description())
}
func (self SyntaxError) Description() string {
return "Syntax error " + self.description()
}
func (self SyntaxError) description() string {
/* check for empty source */
if self.Src == "" {
return fmt.Sprintf("no sources available: %#v", self)
}
p, x, q, y := calcBounds(len(self.Src), self.Pos)
/* compose the error description */
return fmt.Sprintf(
"at index %d: %s\n\n\t%s\n\t%s^%s\n",
self.Pos,
self.Message(),
self.Src[p:q],
strings.Repeat(".", x),
strings.Repeat(".", y),
)
}
func calcBounds(size int, pos int) (lbound int, lwidth int, rbound int, rwidth int) {
if pos >= size || pos < 0 {
return 0, 0, size, 0
}
i := 16
lbound = pos - i
rbound = pos + i
/* prevent slicing before the beginning */
if lbound < 0 {
lbound, rbound, i = 0, rbound - lbound, i + lbound
}
/* prevent slicing beyond the end */
if n := size; rbound > n {
n = rbound - n
rbound = size
/* move the left bound if possible */
if lbound > n {
i += n
lbound -= n
}
}
/* left and right length */
lwidth = clamp_zero(i)
rwidth = clamp_zero(rbound - lbound - i - 1)
return
}
func (self SyntaxError) Message() string {
if self.Msg == "" {
return self.Code.Message()
}
return self.Msg
}
func clamp_zero(v int) int {
if v < 0 {
return 0
} else {
return v
}
}
/** JIT Error Helpers **/
var stackOverflow = &json.UnsupportedValueError {
Str : "Value nesting too deep",
Value : reflect.ValueOf("..."),
}
func error_wrap(src string, pos int, code types.ParsingError) error {
return *error_wrap_heap(src, pos, code)
}
//go:noinline
func error_wrap_heap(src string, pos int, code types.ParsingError) *SyntaxError {
return &SyntaxError {
Pos : pos,
Src : src,
Code : code,
}
}
func error_type(vt *rt.GoType) error {
return &json.UnmarshalTypeError{Type: vt.Pack()}
}
type MismatchTypeError struct {
Pos int
Src string
Type reflect.Type
}
func swithchJSONType (src string, pos int) string {
var val string
switch src[pos] {
case 'f': fallthrough
case 't': val = "bool"
case '"': val = "string"
case '{': val = "object"
case '[': val = "array"
case '-', '0', '1', '2', '3', '4', '5', '6', '7', '8', '9': val = "number"
}
return val
}
func (self MismatchTypeError) Error() string {
se := SyntaxError {
Pos : self.Pos,
Src : self.Src,
Code : types.ERR_MISMATCH,
}
return fmt.Sprintf("Mismatch type %s with value %s %q", self.Type.String(), swithchJSONType(self.Src, self.Pos), se.description())
}
func (self MismatchTypeError) Description() string {
se := SyntaxError {
Pos : self.Pos,
Src : self.Src,
Code : types.ERR_MISMATCH,
}
return fmt.Sprintf("Mismatch type %s with value %s %s", self.Type.String(), swithchJSONType(self.Src, self.Pos), se.description())
}
func error_mismatch(src string, pos int, vt *rt.GoType) error {
return &MismatchTypeError {
Pos : pos,
Src : src,
Type : vt.Pack(),
}
}
func error_field(name string) error {
return errors.New("json: unknown field " + strconv.Quote(name))
}
func error_value(value string, vtype reflect.Type) error {
return &json.UnmarshalTypeError {
Type : vtype,
Value : value,
}
}

143
vendor/github.com/bytedance/sonic/internal/decoder/pools.go generated vendored
View File

@@ -1,143 +0,0 @@
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package decoder
import (
`sync`
`unsafe`
`github.com/bytedance/sonic/internal/caching`
`github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/internal/rt`
)
const (
_MinSlice = 2
_MaxStack = 4096 // 4k slots
_MaxStackBytes = _MaxStack * _PtrBytes
_MaxDigitNums = types.MaxDigitNums // used in atof fallback algorithm
)
const (
_PtrBytes = _PTR_SIZE / 8
_FsmOffset = (_MaxStack + 1) * _PtrBytes
_DbufOffset = _FsmOffset + int64(unsafe.Sizeof(types.StateMachine{})) + types.MAX_RECURSE * _PtrBytes
_StackSize = unsafe.Sizeof(_Stack{})
)
var (
stackPool = sync.Pool{}
valueCache = []unsafe.Pointer(nil)
fieldCache = []*caching.FieldMap(nil)
fieldCacheMux = sync.Mutex{}
programCache = caching.CreateProgramCache()
)
type _Stack struct {
sp uintptr
sb [_MaxStack]unsafe.Pointer
mm types.StateMachine
vp [types.MAX_RECURSE]unsafe.Pointer
dp [_MaxDigitNums]byte
}
type _Decoder func(
s string,
i int,
vp unsafe.Pointer,
sb *_Stack,
fv uint64,
sv string, // DO NOT pass value to this arguement, since it is only used for local _VAR_sv
vk unsafe.Pointer, // DO NOT pass value to this arguement, since it is only used for local _VAR_vk
) (int, error)
var _KeepAlive struct {
s string
i int
vp unsafe.Pointer
sb *_Stack
fv uint64
sv string
vk unsafe.Pointer
ret int
err error
frame_decoder [_FP_offs]byte
frame_generic [_VD_offs]byte
}
var (
argPtrs = []bool{true, false, false, true, true, false, true, false, true}
localPtrs = []bool{}
)
var (
argPtrs_generic = []bool{true}
localPtrs_generic = []bool{}
)
func newStack() *_Stack {
if ret := stackPool.Get(); ret == nil {
return new(_Stack)
} else {
return ret.(*_Stack)
}
}
func resetStack(p *_Stack) {
memclrNoHeapPointers(unsafe.Pointer(p), _StackSize)
}
func freeStack(p *_Stack) {
p.sp = 0
stackPool.Put(p)
}
func freezeValue(v unsafe.Pointer) uintptr {
valueCache = append(valueCache, v)
return uintptr(v)
}
func freezeFields(v *caching.FieldMap) int64 {
fieldCacheMux.Lock()
fieldCache = append(fieldCache, v)
fieldCacheMux.Unlock()
return referenceFields(v)
}
func referenceFields(v *caching.FieldMap) int64 {
return int64(uintptr(unsafe.Pointer(v)))
}
func makeDecoder(vt *rt.GoType, _ ...interface{}) (interface{}, error) {
if pp, err := newCompiler().compile(vt.Pack()); err != nil {
return nil, err
} else {
return newAssembler(pp).Load(), nil
}
}
func findOrCompile(vt *rt.GoType) (_Decoder, error) {
if val := programCache.Get(vt); val != nil {
return val.(_Decoder), nil
} else if ret, err := programCache.Compute(vt, makeDecoder); err == nil {
return ret.(_Decoder), nil
} else {
return nil, err
}
}

46
vendor/github.com/bytedance/sonic/internal/decoder/primitives.go generated vendored
View File

@@ -1,46 +0,0 @@
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package decoder
import (
`encoding`
`encoding/json`
`unsafe`
`github.com/bytedance/sonic/internal/native`
`github.com/bytedance/sonic/internal/rt`
)
func decodeTypedPointer(s string, i int, vt *rt.GoType, vp unsafe.Pointer, sb *_Stack, fv uint64) (int, error) {
if fn, err := findOrCompile(vt); err != nil {
return 0, err
} else {
rt.MoreStack(_FP_size + _VD_size + native.MaxFrameSize)
rt.StopProf()
ret, err := fn(s, i, vp, sb, fv, "", nil)
rt.StartProf()
return ret, err
}
}
func decodeJsonUnmarshaler(vv interface{}, s string) error {
return vv.(json.Unmarshaler).UnmarshalJSON(rt.Str2Mem(s))
}
func decodeTextUnmarshaler(vv interface{}, s string) error {
return vv.(encoding.TextUnmarshaler).UnmarshalText(rt.Str2Mem(s))
}

228
vendor/github.com/bytedance/sonic/internal/decoder/stream.go generated vendored
View File

@@ -1,228 +0,0 @@
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package decoder
import (
`bytes`
`io`
`sync`
`github.com/bytedance/sonic/internal/native`
`github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/option`
)
var (
minLeftBufferShift uint = 1
)
// StreamDecoder is the decoder context object for streaming input.
type StreamDecoder struct {
r io.Reader
buf []byte
scanp int
scanned int64
err error
Decoder
}
var bufPool = sync.Pool{
New: func () interface{} {
return make([]byte, 0, option.DefaultDecoderBufferSize)
},
}
// NewStreamDecoder adapts to encoding/json.NewDecoder API.
//
// NewStreamDecoder returns a new decoder that reads from r.
func NewStreamDecoder(r io.Reader) *StreamDecoder {
return &StreamDecoder{r : r}
}
// Decode decodes input stream into val with corresponding data.
// Redundantly bytes may be read and left in its buffer, and can be used at next call.
// Either io error from underlying io.Reader (except io.EOF)
// or syntax error from data will be recorded and stop subsequently decoding.
func (self *StreamDecoder) Decode(val interface{}) (err error) {
if self.err != nil {
return self.err
}
var buf = self.buf[self.scanp:]
var p = 0
var recycle bool
if cap(buf) == 0 {
buf = bufPool.Get().([]byte)
recycle = true
}
var first = true
var repeat = true
read_more:
for {
l := len(buf)
realloc(&buf)
n, err := self.r.Read(buf[l:cap(buf)])
buf = buf[:l+n]
if err != nil {
repeat = false
if err == io.EOF {
if len(buf) == 0 {
return err
}
break
}
self.err = err
return err
}
if n > 0 || first {
break
}
}
first = false
l := len(buf)
if l > 0 {
self.Decoder.Reset(string(buf))
var x int
if ret := native.SkipOneFast(&self.s, &x); ret < 0 {
if repeat {
goto read_more
} else {
err = SyntaxError{x, self.s, types.ParsingError(-ret), ""}
self.err = err
return
}
}
err = self.Decoder.Decode(val)
if err != nil {
self.err = err
}
p = self.Decoder.Pos()
self.scanned += int64(p)
self.scanp = 0
}
if l > p {
// remain undecoded bytes, so copy them into self.buf
self.buf = append(self.buf[:0], buf[p:]...)
} else {
self.buf = nil
recycle = true
}
if recycle {
buf = buf[:0]
bufPool.Put(buf)
}
return err
}
func (self StreamDecoder) repeatable(err error) bool {
if ee, ok := err.(SyntaxError); ok &&
(ee.Code == types.ERR_EOF || (ee.Code == types.ERR_INVALID_CHAR && self.i >= len(self.s)-1)) {
return true
}
return false
}
// InputOffset returns the input stream byte offset of the current decoder position.
// The offset gives the location of the end of the most recently returned token and the beginning of the next token.
func (self *StreamDecoder) InputOffset() int64 {
return self.scanned + int64(self.scanp)
}
// Buffered returns a reader of the data remaining in the Decoder's buffer.
// The reader is valid until the next call to Decode.
func (self *StreamDecoder) Buffered() io.Reader {
return bytes.NewReader(self.buf[self.scanp:])
}
// More reports whether there is another element in the
// current array or object being parsed.
func (self *StreamDecoder) More() bool {
if self.err != nil {
return false
}
c, err := self.peek()
return err == nil && c != ']' && c != '}'
}
func (self *StreamDecoder) peek() (byte, error) {
var err error
for {
for i := self.scanp; i < len(self.buf); i++ {
c := self.buf[i]
if isSpace(c) {
continue
}
self.scanp = i
return c, nil
}
// buffer has been scanned, now report any error
if err != nil {
if err != io.EOF {
self.err = err
}
return 0, err
}
err = self.refill()
}
}
func isSpace(c byte) bool {
return types.SPACE_MASK & (1 << c) != 0
}
func (self *StreamDecoder) refill() error {
// Make room to read more into the buffer.
// First slide down data already consumed.
if self.scanp > 0 {
self.scanned += int64(self.scanp)
n := copy(self.buf, self.buf[self.scanp:])
self.buf = self.buf[:n]
self.scanp = 0
}
// Grow buffer if not large enough.
realloc(&self.buf)
// Read. Delay error for next iteration (after scan).
n, err := self.r.Read(self.buf[len(self.buf):cap(self.buf)])
self.buf = self.buf[0 : len(self.buf)+n]
return err
}
func realloc(buf *[]byte) {
l := uint(len(*buf))
c := uint(cap(*buf))
if c - l <= c >> minLeftBufferShift {
e := l+(l>>minLeftBufferShift)
if e < option.DefaultDecoderBufferSize {
e = option.DefaultDecoderBufferSize
}
tmp := make([]byte, l, e)
copy(tmp, *buf)
*buf = tmp
}
}

111
vendor/github.com/bytedance/sonic/internal/decoder/stubs_go120.go generated vendored
View File

@@ -1,111 +0,0 @@
// +build go1.20
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package decoder
import (
`unsafe`
`reflect`
_ `github.com/chenzhuoyu/base64x`
`github.com/bytedance/sonic/internal/rt`
)
//go:linkname _subr__b64decode github.com/chenzhuoyu/base64x._subr__b64decode
var _subr__b64decode uintptr
// runtime.maxElementSize
const _max_map_element_size uintptr = 128
func mapfast(vt reflect.Type) bool {
return vt.Elem().Size() <= _max_map_element_size
}
//go:nosplit
//go:linkname throw runtime.throw
//goland:noinspection GoUnusedParameter
func throw(s string)
//go:linkname convT64 runtime.convT64
//goland:noinspection GoUnusedParameter
func convT64(v uint64) unsafe.Pointer
//go:linkname convTslice runtime.convTslice
//goland:noinspection GoUnusedParameter
func convTslice(v []byte) unsafe.Pointer
//go:linkname convTstring runtime.convTstring
//goland:noinspection GoUnusedParameter
func convTstring(v string) unsafe.Pointer
//go:noescape
//go:linkname memequal runtime.memequal
//goland:noinspection GoUnusedParameter
func memequal(a unsafe.Pointer, b unsafe.Pointer, size uintptr) bool
//go:noescape
//go:linkname memmove runtime.memmove
//goland:noinspection GoUnusedParameter
func memmove(to unsafe.Pointer, from unsafe.Pointer, n uintptr)
//go:linkname mallocgc runtime.mallocgc
//goland:noinspection GoUnusedParameter
func mallocgc(size uintptr, typ *rt.GoType, needzero bool) unsafe.Pointer
//go:linkname makeslice runtime.makeslice
//goland:noinspection GoUnusedParameter
func makeslice(et *rt.GoType, len int, cap int) unsafe.Pointer
//go:noescape
//go:linkname growslice reflect.growslice
//goland:noinspection GoUnusedParameter
func growslice(et *rt.GoType, old rt.GoSlice, cap int) rt.GoSlice
//go:linkname makemap_small runtime.makemap_small
func makemap_small() unsafe.Pointer
//go:linkname mapassign runtime.mapassign
//goland:noinspection GoUnusedParameter
func mapassign(t *rt.GoMapType, h unsafe.Pointer, k unsafe.Pointer) unsafe.Pointer
//go:linkname mapassign_fast32 runtime.mapassign_fast32
//goland:noinspection GoUnusedParameter
func mapassign_fast32(t *rt.GoMapType, h unsafe.Pointer, k uint32) unsafe.Pointer
//go:linkname mapassign_fast64 runtime.mapassign_fast64
//goland:noinspection GoUnusedParameter
func mapassign_fast64(t *rt.GoMapType, h unsafe.Pointer, k uint64) unsafe.Pointer
//go:linkname mapassign_fast64ptr runtime.mapassign_fast64ptr
//goland:noinspection GoUnusedParameter
func mapassign_fast64ptr(t *rt.GoMapType, h unsafe.Pointer, k unsafe.Pointer) unsafe.Pointer
//go:linkname mapassign_faststr runtime.mapassign_faststr
//goland:noinspection GoUnusedParameter
func mapassign_faststr(t *rt.GoMapType, h unsafe.Pointer, s string) unsafe.Pointer
//go:nosplit
//go:linkname memclrHasPointers runtime.memclrHasPointers
//goland:noinspection GoUnusedParameter
func memclrHasPointers(ptr unsafe.Pointer, n uintptr)
//go:noescape
//go:linkname memclrNoHeapPointers runtime.memclrNoHeapPointers
//goland:noinspection GoUnusedParameter
func memclrNoHeapPointers(ptr unsafe.Pointer, n uintptr)

58
vendor/github.com/bytedance/sonic/internal/decoder/types.go generated vendored
View File

@@ -1,58 +0,0 @@
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package decoder
import (
`encoding`
`encoding/base64`
`encoding/json`
`reflect`
`unsafe`
`github.com/bytedance/sonic/internal/rt`
)
var (
byteType = reflect.TypeOf(byte(0))
intType = reflect.TypeOf(int(0))
int8Type = reflect.TypeOf(int8(0))
int16Type = reflect.TypeOf(int16(0))
int32Type = reflect.TypeOf(int32(0))
int64Type = reflect.TypeOf(int64(0))
uintType = reflect.TypeOf(uint(0))
uint8Type = reflect.TypeOf(uint8(0))
uint16Type = reflect.TypeOf(uint16(0))
uint32Type = reflect.TypeOf(uint32(0))
uint64Type = reflect.TypeOf(uint64(0))
float32Type = reflect.TypeOf(float32(0))
float64Type = reflect.TypeOf(float64(0))
stringType = reflect.TypeOf("")
bytesType = reflect.TypeOf([]byte(nil))
jsonNumberType = reflect.TypeOf(json.Number(""))
base64CorruptInputError = reflect.TypeOf(base64.CorruptInputError(0))
)
var (
errorType = reflect.TypeOf((*error)(nil)).Elem()
jsonUnmarshalerType = reflect.TypeOf((*json.Unmarshaler)(nil)).Elem()
encodingTextUnmarshalerType = reflect.TypeOf((*encoding.TextUnmarshaler)(nil)).Elem()
)
func rtype(t reflect.Type) (*rt.GoItab, *rt.GoType) {
p := (*rt.GoIface)(unsafe.Pointer(&t))
return p.Itab, (*rt.GoType)(p.Value)
}

39
vendor/github.com/bytedance/sonic/internal/decoder/utils.go generated vendored
View File

@@ -1,39 +0,0 @@
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package decoder
import (
`unsafe`
`github.com/bytedance/sonic/loader`
)
//go:nosplit
func pbool(v bool) uintptr {
return freezeValue(unsafe.Pointer(&v))
}
//go:nosplit
func ptodec(p loader.Function) _Decoder {
return *(*_Decoder)(unsafe.Pointer(&p))
}
func assert_eq(v int64, exp int64, msg string) {
if v != exp {
panic(msg)
}
}

0
vendor/github.com/bytedance/sonic/internal/encoder/asm.s generated vendored
View File

1384
vendor/github.com/bytedance/sonic/internal/encoder/compiler.go generated vendored
View File

File diff suppressed because it is too large Load Diff

66
vendor/github.com/bytedance/sonic/internal/encoder/debug_go116.go generated vendored
View File

@@ -1,66 +0,0 @@
// +build go1.16,!go1.17
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package encoder
import (
`os`
`strings`
`runtime`
`runtime/debug`
`github.com/bytedance/sonic/internal/jit`
)
var (
debugSyncGC = os.Getenv("SONIC_SYNC_GC") != ""
debugAsyncGC = os.Getenv("SONIC_NO_ASYNC_GC") == ""
)
var (
_Instr_End _Instr = newInsOp(_OP_null)
_F_gc = jit.Func(runtime.GC)
_F_force_gc = jit.Func(debug.FreeOSMemory)
_F_println = jit.Func(println_wrapper)
)
func println_wrapper(i int, op1 int, op2 int){
println(i, " Intrs ", op1, _OpNames[op1], "next: ", op2, _OpNames[op2])
}
func (self *_Assembler) force_gc() {
self.call_go(_F_gc)
self.call_go(_F_force_gc)
}
func (self *_Assembler) debug_instr(i int, v *_Instr) {
if debugSyncGC {
if (i+1 == len(self.p)) {
self.print_gc(i, v, &_Instr_End)
} else {
next := &(self.p[i+1])
self.print_gc(i, v, next)
name := _OpNames[next.op()]
if strings.Contains(name, "save") {
return
}
}
self.force_gc()
}
}

205
vendor/github.com/bytedance/sonic/internal/encoder/debug_go117.go generated vendored
View File

@@ -1,205 +0,0 @@
// +build go1.17,!go1.22
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package encoder
import (
`fmt`
`os`
`runtime`
`strings`
`unsafe`
`github.com/bytedance/sonic/internal/jit`
`github.com/twitchyliquid64/golang-asm/obj`
)
const _FP_debug = 128
var (
debugSyncGC = os.Getenv("SONIC_SYNC_GC") != ""
debugAsyncGC = os.Getenv("SONIC_NO_ASYNC_GC") == ""
debugCheckPtr = os.Getenv("SONIC_CHECK_POINTER") != ""
)
var (
_Instr_End = newInsOp(_OP_is_nil)
_F_gc = jit.Func(gc)
_F_println = jit.Func(println_wrapper)
_F_print = jit.Func(print)
)
func (self *_Assembler) dsave(r ...obj.Addr) {
for i, v := range r {
if i > _FP_debug / 8 - 1 {
panic("too many registers to save")
} else {
self.Emit("MOVQ", v, jit.Ptr(_SP, _FP_fargs + _FP_saves + _FP_locals + int64(i) * 8))
}
}
}
func (self *_Assembler) dload(r ...obj.Addr) {
for i, v := range r {
if i > _FP_debug / 8 - 1 {
panic("too many registers to load")
} else {
self.Emit("MOVQ", jit.Ptr(_SP, _FP_fargs + _FP_saves + _FP_locals + int64(i) * 8), v)
}
}
}
func println_wrapper(i int, op1 int, op2 int){
println(i, " Intrs ", op1, _OpNames[op1], "next: ", op2, _OpNames[op2])
}
func print(i int){
println(i)
}
func gc() {
if !debugSyncGC {
return
}
runtime.GC()
// debug.FreeOSMemory()
}
func (self *_Assembler) dcall(fn obj.Addr) {
self.Emit("MOVQ", fn, _R10) // MOVQ ${fn}, R10
self.Rjmp("CALL", _R10) // CALL R10
}
func (self *_Assembler) debug_gc() {
if !debugSyncGC {
return
}
self.dsave(_REG_debug...)
self.dcall(_F_gc)
self.dload(_REG_debug...)
}
func (self *_Assembler) debug_instr(i int, v *_Instr) {
if debugSyncGC {
if i+1 == len(self.p) {
self.print_gc(i, v, &_Instr_End)
} else {
next := &(self.p[i+1])
self.print_gc(i, v, next)
name := _OpNames[next.op()]
if strings.Contains(name, "save") {
return
}
}
// self.debug_gc()
}
}
//go:noescape
//go:linkname checkptrBase runtime.checkptrBase
func checkptrBase(p unsafe.Pointer) uintptr
//go:noescape
//go:linkname findObject runtime.findObject
func findObject(p, refBase, refOff uintptr) (base uintptr, s unsafe.Pointer, objIndex uintptr)
var (
_F_checkptr = jit.Func(checkptr)
_F_printptr = jit.Func(printptr)
)
var (
_R10 = jit.Reg("R10")
)
var _REG_debug = []obj.Addr {
jit.Reg("AX"),
jit.Reg("BX"),
jit.Reg("CX"),
jit.Reg("DX"),
jit.Reg("DI"),
jit.Reg("SI"),
jit.Reg("BP"),
jit.Reg("SP"),
jit.Reg("R8"),
jit.Reg("R9"),
jit.Reg("R10"),
jit.Reg("R11"),
jit.Reg("R12"),
jit.Reg("R13"),
jit.Reg("R14"),
jit.Reg("R15"),
}
func checkptr(ptr uintptr) {
if ptr == 0 {
return
}
fmt.Printf("pointer: %x\n", ptr)
f := checkptrBase(unsafe.Pointer(uintptr(ptr)))
if f == 0 {
fmt.Printf("! unknown-based pointer: %x\n", ptr)
} else if f == 1 {
fmt.Printf("! stack pointer: %x\n", ptr)
} else {
fmt.Printf("base: %x\n", f)
}
findobj(ptr)
}
func findobj(ptr uintptr) {
base, s, objIndex := findObject(ptr, 0, 0)
if s != nil && base == 0 {
fmt.Printf("! invalid pointer: %x\n", ptr)
}
fmt.Printf("objIndex: %d\n", objIndex)
}
func (self *_Assembler) check_ptr(ptr obj.Addr, lea bool) {
if !debugCheckPtr {
return
}
self.dsave(_REG_debug...)
if lea {
self.Emit("LEAQ", ptr, _R10)
} else {
self.Emit("MOVQ", ptr, _R10)
}
self.Emit("MOVQ", _R10, jit.Ptr(_SP, 0))
self.dcall(_F_checkptr)
self.dload(_REG_debug...)
}
func printptr(i int, ptr uintptr) {
fmt.Printf("[%d] ptr: %x\n", i, ptr)
}
func (self *_Assembler) print_ptr(i int, ptr obj.Addr, lea bool) {
self.dsave(_REG_debug...)
if lea {
self.Emit("LEAQ", ptr, _R10)
} else {
self.Emit("MOVQ", ptr, _R10)
}
self.Emit("MOVQ", jit.Imm(int64(i)), _AX)
self.Emit("MOVQ", _R10, _BX)
self.dcall(_F_printptr)
self.dload(_REG_debug...)
}

186
vendor/github.com/bytedance/sonic/internal/encoder/encoder.go generated vendored
View File

@@ -17,63 +17,63 @@
package encoder package encoder
import ( import (
`bytes` "bytes"
`encoding/json` "encoding/json"
`reflect` "reflect"
`runtime` "runtime"
`unsafe` "unsafe"
`github.com/bytedance/sonic/internal/native` "github.com/bytedance/sonic/utf8"
`github.com/bytedance/sonic/internal/native/types` "github.com/bytedance/sonic/internal/encoder/alg"
`github.com/bytedance/sonic/internal/rt` "github.com/bytedance/sonic/internal/encoder/vars"
`github.com/bytedance/sonic/utf8` "github.com/bytedance/sonic/internal/rt"
`github.com/bytedance/sonic/option` "github.com/bytedance/sonic/option"
"github.com/bytedance/gopkg/lang/dirtmake"
) )
// Options is a set of encoding options. // Options is a set of encoding options.
type Options uint64 type Options uint64
const (
bitSortMapKeys = iota
bitEscapeHTML
bitCompactMarshaler
bitNoQuoteTextMarshaler
bitNoNullSliceOrMap
bitValidateString
// used for recursive compile
bitPointerValue = 63
)
const ( const (
// SortMapKeys indicates that the keys of a map needs to be sorted // SortMapKeys indicates that the keys of a map needs to be sorted
// before serializing into JSON. // before serializing into JSON.
// WARNING: This hurts performance A LOT, USE WITH CARE. // WARNING: This hurts performance A LOT, USE WITH CARE.
SortMapKeys Options = 1 << bitSortMapKeys SortMapKeys Options = 1 << alg.BitSortMapKeys
// EscapeHTML indicates encoder to escape all HTML characters // EscapeHTML indicates encoder to escape all HTML characters
// after serializing into JSON (see https://pkg.go.dev/encoding/json#HTMLEscape). // after serializing into JSON (see https://pkg.go.dev/encoding/json#HTMLEscape).
// WARNING: This hurts performance A LOT, USE WITH CARE. // WARNING: This hurts performance A LOT, USE WITH CARE.
EscapeHTML Options = 1 << bitEscapeHTML EscapeHTML Options = 1 << alg.BitEscapeHTML
// CompactMarshaler indicates that the output JSON from json.Marshaler // CompactMarshaler indicates that the output JSON from json.Marshaler
// is always compact and needs no validation // is always compact and needs no validation
CompactMarshaler Options = 1 << bitCompactMarshaler CompactMarshaler Options = 1 << alg.BitCompactMarshaler
// NoQuoteTextMarshaler indicates that the output text from encoding.TextMarshaler // NoQuoteTextMarshaler indicates that the output text from encoding.TextMarshaler
// is always escaped string and needs no quoting // is always escaped string and needs no quoting
NoQuoteTextMarshaler Options = 1 << bitNoQuoteTextMarshaler NoQuoteTextMarshaler Options = 1 << alg.BitNoQuoteTextMarshaler
// NoNullSliceOrMap indicates all empty Array or Object are encoded as '[]' or '{}', // NoNullSliceOrMap indicates all empty Array or Object are encoded as '[]' or '{}',
// instead of 'null' // instead of 'null'.
NoNullSliceOrMap Options = 1 << bitNoNullSliceOrMap // NOTE: The priority of this option is lower than json tag `omitempty`.
NoNullSliceOrMap Options = 1 << alg.BitNoNullSliceOrMap
// ValidateString indicates that encoder should validate the input string // ValidateString indicates that encoder should validate the input string
// before encoding it into JSON. // before encoding it into JSON.
ValidateString Options = 1 << bitValidateString ValidateString Options = 1 << alg.BitValidateString
// NoValidateJSONMarshaler indicates that the encoder should not validate the output string
// after encoding the JSONMarshaler to JSON.
NoValidateJSONMarshaler Options = 1 << alg.BitNoValidateJSONMarshaler
// NoEncoderNewline indicates that the encoder should not add a newline after every message
NoEncoderNewline Options = 1 << alg.BitNoEncoderNewline
// CompatibleWithStd is used to be compatible with std encoder. // CompatibleWithStd is used to be compatible with std encoder.
CompatibleWithStd Options = SortMapKeys | EscapeHTML | CompactMarshaler CompatibleWithStd Options = SortMapKeys | EscapeHTML | CompactMarshaler
// Encode Infinity or Nan float into `null`, instead of returning an error.
EncodeNullForInfOrNan Options = 1 << alg.BitEncodeNullForInfOrNan
) )
// Encoder represents a specific set of encoder configurations. // Encoder represents a specific set of encoder configurations.
@@ -115,6 +115,25 @@ func (self *Encoder) SetValidateString(f bool) {
} }
} }
// SetNoValidateJSONMarshaler specifies if option NoValidateJSONMarshaler opens
func (self *Encoder) SetNoValidateJSONMarshaler(f bool) {
if f {
self.Opts |= NoValidateJSONMarshaler
} else {
self.Opts &= ^NoValidateJSONMarshaler
}
}
// SetNoEncoderNewline specifies if option NoEncoderNewline opens
func (self *Encoder) SetNoEncoderNewline(f bool) {
if f {
self.Opts |= NoEncoderNewline
} else {
self.Opts &= ^NoEncoderNewline
}
}
// SetCompactMarshaler specifies if option CompactMarshaler opens // SetCompactMarshaler specifies if option CompactMarshaler opens
func (self *Encoder) SetCompactMarshaler(f bool) { func (self *Encoder) SetCompactMarshaler(f bool) {
if f { if f {
@@ -143,53 +162,45 @@ func (enc *Encoder) SetIndent(prefix, indent string) {
// Quote returns the JSON-quoted version of s. // Quote returns the JSON-quoted version of s.
func Quote(s string) string { func Quote(s string) string {
var n int buf := make([]byte, 0, len(s)+2)
var p []byte buf = alg.Quote(buf, s, false)
return rt.Mem2Str(buf)
/* check for empty string */
if s == "" {
return `""`
}
/* allocate space for result */
n = len(s) + 2
p = make([]byte, 0, n)
/* call the encoder */
_ = encodeString(&p, s)
return rt.Mem2Str(p)
} }
// Encode returns the JSON encoding of val, encoded with opts. // Encode returns the JSON encoding of val, encoded with opts.
func Encode(val interface{}, opts Options) ([]byte, error) { func Encode(val interface{}, opts Options) ([]byte, error) {
var ret []byte var ret []byte
buf := newBytes() buf := vars.NewBytes()
err := encodeInto(&buf, val, opts) err := encodeIntoCheckRace(buf, val, opts)
/* check for errors */ /* check for errors */
if err != nil { if err != nil {
freeBytes(buf) vars.FreeBytes(buf)
return nil, err return nil, err
} }
/* htmlescape or correct UTF-8 if opts enable */ /* htmlescape or correct UTF-8 if opts enable */
old := buf old := buf
buf = encodeFinish(old, opts) *buf = encodeFinish(*old, opts)
pbuf := ((*rt.GoSlice)(unsafe.Pointer(&buf))).Ptr pbuf := ((*rt.GoSlice)(unsafe.Pointer(buf))).Ptr
pold := ((*rt.GoSlice)(unsafe.Pointer(&old))).Ptr pold := ((*rt.GoSlice)(unsafe.Pointer(old))).Ptr
/* return when allocated a new buffer */ /* return when allocated a new buffer */
if pbuf != pold { if pbuf != pold {
freeBytes(old) vars.FreeBytes(old)
return buf, nil return *buf, nil
} }
/* make a copy of the result */ /* make a copy of the result */
ret = make([]byte, len(buf)) if rt.CanSizeResue(cap(*buf)) {
copy(ret, buf) ret = dirtmake.Bytes(len(*buf), len(*buf))
copy(ret, *buf)
vars.FreeBytes(buf)
} else {
ret = *buf
}
freeBytes(buf)
/* return the buffer into pool */ /* return the buffer into pool */
return ret, nil return ret, nil
} }
@@ -197,7 +208,7 @@ func Encode(val interface{}, opts Options) ([]byte, error) {
// EncodeInto is like Encode but uses a user-supplied buffer instead of allocating // EncodeInto is like Encode but uses a user-supplied buffer instead of allocating
// a new one. // a new one.
func EncodeInto(buf *[]byte, val interface{}, opts Options) error { func EncodeInto(buf *[]byte, val interface{}, opts Options) error {
err := encodeInto(buf, val, opts) err := encodeIntoCheckRace(buf, val, opts)
if err != nil { if err != nil {
return err return err
} }
@@ -206,15 +217,15 @@ func EncodeInto(buf *[]byte, val interface{}, opts Options) error {
} }
func encodeInto(buf *[]byte, val interface{}, opts Options) error { func encodeInto(buf *[]byte, val interface{}, opts Options) error {
stk := newStack() stk := vars.NewStack()
efv := rt.UnpackEface(val) efv := rt.UnpackEface(val)
err := encodeTypedPointer(buf, efv.Type, &efv.Value, stk, uint64(opts)) err := encodeTypedPointer(buf, efv.Type, &efv.Value, stk, uint64(opts))
/* return the stack into pool */ /* return the stack into pool */
if err != nil { if err != nil {
resetStack(stk) vars.ResetStack(stk)
} }
freeStack(stk) vars.FreeStack(stk)
/* avoid GC ahead */ /* avoid GC ahead */
runtime.KeepAlive(buf) runtime.KeepAlive(buf)
@@ -226,13 +237,12 @@ func encodeFinish(buf []byte, opts Options) []byte {
if opts & EscapeHTML != 0 { if opts & EscapeHTML != 0 {
buf = HTMLEscape(nil, buf) buf = HTMLEscape(nil, buf)
} }
if opts & ValidateString != 0 && !utf8.Validate(buf) { if (opts & ValidateString != 0) && !utf8.Validate(buf) {
buf = utf8.CorrectWith(nil, buf, `\ufffd`) buf = utf8.CorrectWith(nil, buf, `\ufffd`)
} }
return buf return buf
} }
var typeByte = rt.UnpackType(reflect.TypeOf(byte(0)))
// HTMLEscape appends to dst the JSON-encoded src with <, >, &, U+2028 and U+2029 // HTMLEscape appends to dst the JSON-encoded src with <, >, &, U+2028 and U+2029
// characters inside string literals changed to \u003c, \u003e, \u0026, \u2028, \u2029 // characters inside string literals changed to \u003c, \u003e, \u0026, \u2028, \u2029
@@ -241,7 +251,7 @@ var typeByte = rt.UnpackType(reflect.TypeOf(byte(0)))
// escaping within <script> tags, so an alternative JSON encoding must // escaping within <script> tags, so an alternative JSON encoding must
// be used. // be used.
func HTMLEscape(dst []byte, src []byte) []byte { func HTMLEscape(dst []byte, src []byte) []byte {
return htmlEscape(dst, src) return alg.HtmlEscape(dst, src)
} }
// EncodeIndented is like Encode but applies Indent to format the output. // EncodeIndented is like Encode but applies Indent to format the output.
@@ -249,37 +259,40 @@ func HTMLEscape(dst []byte, src []byte) []byte {
// followed by one or more copies of indent according to the indentation nesting. // followed by one or more copies of indent according to the indentation nesting.
func EncodeIndented(val interface{}, prefix string, indent string, opts Options) ([]byte, error) { func EncodeIndented(val interface{}, prefix string, indent string, opts Options) ([]byte, error) {
var err error var err error
var out []byte
var buf *bytes.Buffer var buf *bytes.Buffer
/* encode into the buffer */ /* encode into the buffer */
out = newBytes() out := vars.NewBytes()
err = EncodeInto(&out, val, opts) err = EncodeInto(out, val, opts)
/* check for errors */ /* check for errors */
if err != nil { if err != nil {
freeBytes(out) vars.FreeBytes(out)
return nil, err return nil, err
} }
/* indent the JSON */ /* indent the JSON */
buf = newBuffer() buf = vars.NewBuffer()
err = json.Indent(buf, out, prefix, indent) err = json.Indent(buf, *out, prefix, indent)
vars.FreeBytes(out)
/* check for errors */ /* check for errors */
if err != nil { if err != nil {
freeBytes(out) vars.FreeBuffer(buf)
freeBuffer(buf)
return nil, err return nil, err
} }
/* copy to the result buffer */ /* copy to the result buffer */
ret := make([]byte, buf.Len()) var ret []byte
copy(ret, buf.Bytes()) if rt.CanSizeResue(cap(buf.Bytes())) {
ret = make([]byte, buf.Len())
copy(ret, buf.Bytes())
/* return the buffers into pool */
vars.FreeBuffer(buf)
} else {
ret = buf.Bytes()
}
/* return the buffers into pool */
freeBytes(out)
freeBuffer(buf)
return ret, nil return ret, nil
} }
@@ -302,26 +315,5 @@ func Pretouch(vt reflect.Type, opts ...option.CompileOption) error {
// //
// Note: it does not check for the invalid UTF-8 characters. // Note: it does not check for the invalid UTF-8 characters.
func Valid(data []byte) (ok bool, start int) { func Valid(data []byte) (ok bool, start int) {
n := len(data) return alg.Valid(data)
if n == 0 {
return false, -1
}
s := rt.Mem2Str(data)
p := 0
m := types.NewStateMachine()
ret := native.ValidateOne(&s, &p, m)
types.FreeStateMachine(m)
if ret < 0 {
return false, p-1
}
/* check for trailing spaces */
for ;p < n; p++ {
if (types.SPACE_MASK & (1 << data[p])) == 0 {
return false, p
}
}
return true, ret
} }

65
vendor/github.com/bytedance/sonic/internal/encoder/errors.go generated vendored
View File

@@ -1,65 +0,0 @@
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package encoder
import (
`encoding/json`
`fmt`
`reflect`
`strconv`
`unsafe`
`github.com/bytedance/sonic/internal/rt`
)
var _ERR_too_deep = &json.UnsupportedValueError {
Str : "Value nesting too deep",
Value : reflect.ValueOf("..."),
}
var _ERR_nan_or_infinite = &json.UnsupportedValueError {
Str : "NaN or ±Infinite",
Value : reflect.ValueOf("NaN or ±Infinite"),
}
func error_type(vtype reflect.Type) error {
return &json.UnsupportedTypeError{Type: vtype}
}
func error_number(number json.Number) error {
return &json.UnsupportedValueError {
Str : "invalid number literal: " + strconv.Quote(string(number)),
Value : reflect.ValueOf(number),
}
}
func error_marshaler(ret []byte, pos int) error {
return fmt.Errorf("invalid Marshaler output json syntax at %d: %q", pos, ret)
}
const (
panicNilPointerOfNonEmptyString int = 1 + iota
)
func goPanic(code int, val unsafe.Pointer) {
switch(code){
case panicNilPointerOfNonEmptyString:
panic(fmt.Sprintf("val: %#v has nil pointer while its length is not zero!", (*rt.GoString)(val)))
default:
panic("encoder error!")
}
}

199
vendor/github.com/bytedance/sonic/internal/encoder/mapiter.go generated vendored
View File

@@ -1,199 +0,0 @@
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package encoder
import (
"encoding"
"reflect"
"sync"
"unsafe"
"github.com/bytedance/sonic/internal/native"
"github.com/bytedance/sonic/internal/rt"
)
type _MapPair struct {
k string // when the map key is integer, k is pointed to m
v unsafe.Pointer
m [32]byte
}
type _MapIterator struct {
it rt.GoMapIterator // must be the first field
kv rt.GoSlice // slice of _MapPair
ki int
}
var (
iteratorPool = sync.Pool{}
iteratorPair = rt.UnpackType(reflect.TypeOf(_MapPair{}))
)
func init() {
if unsafe.Offsetof(_MapIterator{}.it) != 0 {
panic("_MapIterator.it is not the first field")
}
}
func newIterator() *_MapIterator {
if v := iteratorPool.Get(); v == nil {
return new(_MapIterator)
} else {
return resetIterator(v.(*_MapIterator))
}
}
func resetIterator(p *_MapIterator) *_MapIterator {
p.ki = 0
p.it = rt.GoMapIterator{}
p.kv.Len = 0
return p
}
func (self *_MapIterator) at(i int) *_MapPair {
return (*_MapPair)(unsafe.Pointer(uintptr(self.kv.Ptr) + uintptr(i) * unsafe.Sizeof(_MapPair{})))
}
func (self *_MapIterator) add() (p *_MapPair) {
p = self.at(self.kv.Len)
self.kv.Len++
return
}
func (self *_MapIterator) data() (p []_MapPair) {
*(*rt.GoSlice)(unsafe.Pointer(&p)) = self.kv
return
}
func (self *_MapIterator) append(t *rt.GoType, k unsafe.Pointer, v unsafe.Pointer) (err error) {
p := self.add()
p.v = v
/* check for strings */
if tk := t.Kind(); tk != reflect.String {
return self.appendGeneric(p, t, tk, k)
}
/* fast path for strings */
p.k = *(*string)(k)
return nil
}
func (self *_MapIterator) appendGeneric(p *_MapPair, t *rt.GoType, v reflect.Kind, k unsafe.Pointer) error {
switch v {
case reflect.Int : p.k = rt.Mem2Str(p.m[:native.I64toa(&p.m[0], int64(*(*int)(k)))]) ; return nil
case reflect.Int8 : p.k = rt.Mem2Str(p.m[:native.I64toa(&p.m[0], int64(*(*int8)(k)))]) ; return nil
case reflect.Int16 : p.k = rt.Mem2Str(p.m[:native.I64toa(&p.m[0], int64(*(*int16)(k)))]) ; return nil
case reflect.Int32 : p.k = rt.Mem2Str(p.m[:native.I64toa(&p.m[0], int64(*(*int32)(k)))]) ; return nil
case reflect.Int64 : p.k = rt.Mem2Str(p.m[:native.I64toa(&p.m[0], *(*int64)(k))]) ; return nil
case reflect.Uint : p.k = rt.Mem2Str(p.m[:native.U64toa(&p.m[0], uint64(*(*uint)(k)))]) ; return nil
case reflect.Uint8 : p.k = rt.Mem2Str(p.m[:native.U64toa(&p.m[0], uint64(*(*uint8)(k)))]) ; return nil
case reflect.Uint16 : p.k = rt.Mem2Str(p.m[:native.U64toa(&p.m[0], uint64(*(*uint16)(k)))]) ; return nil
case reflect.Uint32 : p.k = rt.Mem2Str(p.m[:native.U64toa(&p.m[0], uint64(*(*uint32)(k)))]) ; return nil
case reflect.Uint64 : p.k = rt.Mem2Str(p.m[:native.U64toa(&p.m[0], *(*uint64)(k))]) ; return nil
case reflect.Uintptr : p.k = rt.Mem2Str(p.m[:native.U64toa(&p.m[0], uint64(*(*uintptr)(k)))]) ; return nil
case reflect.Interface : return self.appendInterface(p, t, k)
case reflect.Struct, reflect.Ptr : return self.appendConcrete(p, t, k)
default : panic("unexpected map key type")
}
}
func (self *_MapIterator) appendConcrete(p *_MapPair, t *rt.GoType, k unsafe.Pointer) (err error) {
// compiler has already checked that the type implements the encoding.MarshalText interface
if !t.Indirect() {
k = *(*unsafe.Pointer)(k)
}
eface := rt.GoEface{Value: k, Type: t}.Pack()
out, err := eface.(encoding.TextMarshaler).MarshalText()
if err != nil {
return err
}
p.k = rt.Mem2Str(out)
return
}
func (self *_MapIterator) appendInterface(p *_MapPair, t *rt.GoType, k unsafe.Pointer) (err error) {
if len(rt.IfaceType(t).Methods) == 0 {
panic("unexpected map key type")
} else if p.k, err = asText(k); err == nil {
return nil
} else {
return
}
}
func iteratorStop(p *_MapIterator) {
iteratorPool.Put(p)
}
func iteratorNext(p *_MapIterator) {
i := p.ki
t := &p.it
/* check for unordered iteration */
if i < 0 {
mapiternext(t)
return
}
/* check for end of iteration */
if p.ki >= p.kv.Len {
t.K = nil
t.V = nil
return
}
/* update the key-value pair, and increase the pointer */
t.K = unsafe.Pointer(&p.at(p.ki).k)
t.V = p.at(p.ki).v
p.ki++
}
func iteratorStart(t *rt.GoMapType, m *rt.GoMap, fv uint64) (*_MapIterator, error) {
it := newIterator()
mapiterinit(t, m, &it.it)
/* check for key-sorting, empty map don't need sorting */
if m.Count == 0 || (fv & uint64(SortMapKeys)) == 0 {
it.ki = -1
return it, nil
}
/* pre-allocate space if needed */
if m.Count > it.kv.Cap {
it.kv = growslice(iteratorPair, it.kv, m.Count)
}
/* dump all the key-value pairs */
for ; it.it.K != nil; mapiternext(&it.it) {
if err := it.append(t.Key, it.it.K, it.it.V); err != nil {
iteratorStop(it)
return nil, err
}
}
/* sort the keys, map with only 1 item don't need sorting */
if it.ki = 1; m.Count > 1 {
radixQsort(it.data(), 0, maxDepth(it.kv.Len))
}
/* load the first pair into iterator */
it.it.V = it.at(0).v
it.it.K = unsafe.Pointer(&it.at(0).k)
return it, nil
}

193
vendor/github.com/bytedance/sonic/internal/encoder/pools.go generated vendored
View File

@@ -1,193 +0,0 @@
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package encoder
import (
`bytes`
`sync`
`unsafe`
`errors`
`reflect`
`github.com/bytedance/sonic/internal/caching`
`github.com/bytedance/sonic/option`
`github.com/bytedance/sonic/internal/rt`
)
const (
_MaxStack = 4096 // 4k states
_StackSize = unsafe.Sizeof(_Stack{})
)
var (
bytesPool = sync.Pool{}
stackPool = sync.Pool{}
bufferPool = sync.Pool{}
programCache = caching.CreateProgramCache()
)
type _State struct {
x int
f uint64
p unsafe.Pointer
q unsafe.Pointer
}
type _Stack struct {
sp uint64
sb [_MaxStack]_State
}
type _Encoder func(
rb *[]byte,
vp unsafe.Pointer,
sb *_Stack,
fv uint64,
) error
var _KeepAlive struct {
rb *[]byte
vp unsafe.Pointer
sb *_Stack
fv uint64
err error
frame [_FP_offs]byte
}
var errCallShadow = errors.New("DON'T CALL THIS!")
// Faker func of _Encoder, used to export its stackmap as _Encoder's
func _Encoder_Shadow(rb *[]byte, vp unsafe.Pointer, sb *_Stack, fv uint64) (err error) {
// align to assembler_amd64.go: _FP_offs
var frame [_FP_offs]byte
// must keep all args and frames noticeable to GC
_KeepAlive.rb = rb
_KeepAlive.vp = vp
_KeepAlive.sb = sb
_KeepAlive.fv = fv
_KeepAlive.err = err
_KeepAlive.frame = frame
return errCallShadow
}
func newBytes() []byte {
if ret := bytesPool.Get(); ret != nil {
return ret.([]byte)
} else {
return make([]byte, 0, option.DefaultEncoderBufferSize)
}
}
func newStack() *_Stack {
if ret := stackPool.Get(); ret == nil {
return new(_Stack)
} else {
return ret.(*_Stack)
}
}
func resetStack(p *_Stack) {
memclrNoHeapPointers(unsafe.Pointer(p), _StackSize)
}
func newBuffer() *bytes.Buffer {
if ret := bufferPool.Get(); ret != nil {
return ret.(*bytes.Buffer)
} else {
return bytes.NewBuffer(make([]byte, 0, option.DefaultEncoderBufferSize))
}
}
func freeBytes(p []byte) {
p = p[:0]
bytesPool.Put(p)
}
func freeStack(p *_Stack) {
p.sp = 0
stackPool.Put(p)
}
func freeBuffer(p *bytes.Buffer) {
p.Reset()
bufferPool.Put(p)
}
func makeEncoder(vt *rt.GoType, ex ...interface{}) (interface{}, error) {
if pp, err := newCompiler().compile(vt.Pack(), ex[0].(bool)); err != nil {
return nil, err
} else {
as := newAssembler(pp)
as.name = vt.String()
return as.Load(), nil
}
}
func findOrCompile(vt *rt.GoType, pv bool) (_Encoder, error) {
if val := programCache.Get(vt); val != nil {
return val.(_Encoder), nil
} else if ret, err := programCache.Compute(vt, makeEncoder, pv); err == nil {
return ret.(_Encoder), nil
} else {
return nil, err
}
}
func pretouchType(_vt reflect.Type, opts option.CompileOptions, v uint8) (map[reflect.Type]uint8, error) {
/* compile function */
compiler := newCompiler().apply(opts)
encoder := func(vt *rt.GoType, ex ...interface{}) (interface{}, error) {
if pp, err := compiler.compile(_vt, ex[0].(bool)); err != nil {
return nil, err
} else {
as := newAssembler(pp)
as.name = vt.String()
return as.Load(), nil
}
}
/* find or compile */
vt := rt.UnpackType(_vt)
if val := programCache.Get(vt); val != nil {
return nil, nil
} else if _, err := programCache.Compute(vt, encoder, v == 1); err == nil {
return compiler.rec, nil
} else {
return nil, err
}
}
func pretouchRec(vtm map[reflect.Type]uint8, opts option.CompileOptions) error {
if opts.RecursiveDepth < 0 || len(vtm) == 0 {
return nil
}
next := make(map[reflect.Type]uint8)
for vt, v := range vtm {
sub, err := pretouchType(vt, opts, v)
if err != nil {
return err
}
for svt, v := range sub {
next[svt] = v
}
}
opts.RecursiveDepth -= 1
return pretouchRec(next, opts)
}

168
vendor/github.com/bytedance/sonic/internal/encoder/primitives.go generated vendored
View File

@@ -1,168 +0,0 @@
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package encoder
import (
`encoding`
`encoding/json`
`unsafe`
`github.com/bytedance/sonic/internal/jit`
`github.com/bytedance/sonic/internal/native`
`github.com/bytedance/sonic/internal/rt`
)
/** Encoder Primitives **/
func encodeNil(rb *[]byte) error {
*rb = append(*rb, 'n', 'u', 'l', 'l')
return nil
}
func encodeString(buf *[]byte, val string) error {
var sidx int
var pbuf *rt.GoSlice
var pstr *rt.GoString
/* opening quote */
*buf = append(*buf, '"')
pbuf = (*rt.GoSlice)(unsafe.Pointer(buf))
pstr = (*rt.GoString)(unsafe.Pointer(&val))
/* encode with native library */
for sidx < pstr.Len {
sn := pstr.Len - sidx
dn := pbuf.Cap - pbuf.Len
sp := padd(pstr.Ptr, sidx)
dp := padd(pbuf.Ptr, pbuf.Len)
nb := native.Quote(sp, sn, dp, &dn, 0)
/* check for errors */
if pbuf.Len += dn; nb >= 0 {
break
}
/* not enough space, grow the slice and try again */
sidx += ^nb
*pbuf = growslice(rt.UnpackType(byteType), *pbuf, pbuf.Cap * 2)
}
/* closing quote */
*buf = append(*buf, '"')
return nil
}
func encodeTypedPointer(buf *[]byte, vt *rt.GoType, vp *unsafe.Pointer, sb *_Stack, fv uint64) error {
if vt == nil {
return encodeNil(buf)
} else if fn, err := findOrCompile(vt, (fv&(1<<bitPointerValue)) != 0); err != nil {
return err
} else if vt.Indirect() {
rt.MoreStack(_FP_size + native.MaxFrameSize)
rt.StopProf()
err := fn(buf, *vp, sb, fv)
rt.StartProf()
return err
} else {
rt.MoreStack(_FP_size + native.MaxFrameSize)
rt.StopProf()
err := fn(buf, unsafe.Pointer(vp), sb, fv)
rt.StartProf()
return err
}
}
func encodeJsonMarshaler(buf *[]byte, val json.Marshaler, opt Options) error {
if ret, err := val.MarshalJSON(); err != nil {
return err
} else {
if opt & CompactMarshaler != 0 {
return compact(buf, ret)
}
if ok, s := Valid(ret); !ok {
return error_marshaler(ret, s)
}
*buf = append(*buf, ret...)
return nil
}
}
func encodeTextMarshaler(buf *[]byte, val encoding.TextMarshaler, opt Options) error {
if ret, err := val.MarshalText(); err != nil {
return err
} else {
if opt & NoQuoteTextMarshaler != 0 {
*buf = append(*buf, ret...)
return nil
}
return encodeString(buf, rt.Mem2Str(ret) )
}
}
func htmlEscape(dst []byte, src []byte) []byte {
var sidx int
dst = append(dst, src[:0]...) // avoid check nil dst
sbuf := (*rt.GoSlice)(unsafe.Pointer(&src))
dbuf := (*rt.GoSlice)(unsafe.Pointer(&dst))
/* grow dst if it is shorter */
if cap(dst) - len(dst) < len(src) + native.BufPaddingSize {
cap := len(src) * 3 / 2 + native.BufPaddingSize
*dbuf = growslice(typeByte, *dbuf, cap)
}
for sidx < sbuf.Len {
sp := padd(sbuf.Ptr, sidx)
dp := padd(dbuf.Ptr, dbuf.Len)
sn := sbuf.Len - sidx
dn := dbuf.Cap - dbuf.Len
nb := native.HTMLEscape(sp, sn, dp, &dn)
/* check for errors */
if dbuf.Len += dn; nb >= 0 {
break
}
/* not enough space, grow the slice and try again */
sidx += ^nb
*dbuf = growslice(typeByte, *dbuf, dbuf.Cap * 2)
}
return dst
}
var (
argPtrs = []bool { true, true, true, false }
localPtrs = []bool{}
)
var (
_F_assertI2I = jit.Func(assertI2I)
)
func asText(v unsafe.Pointer) (string, error) {
text := assertI2I(_T_encoding_TextMarshaler, *(*rt.GoIface)(v))
r, e := (*(*encoding.TextMarshaler)(unsafe.Pointer(&text))).MarshalText()
return rt.Mem2Str(r), e
}
func asJson(v unsafe.Pointer) (string, error) {
text := assertI2I(_T_json_Marshaler, *(*rt.GoIface)(v))
r, e := (*(*json.Marshaler)(unsafe.Pointer(&text))).MarshalJSON()
return rt.Mem2Str(r), e
}

206
vendor/github.com/bytedance/sonic/internal/encoder/sort.go generated vendored
View File

@@ -1,206 +0,0 @@
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package encoder
// Algorithm 3-way Radix Quicksort, d means the radix.
// Reference: https://algs4.cs.princeton.edu/51radix/Quick3string.java.html
func radixQsort(kvs []_MapPair, d, maxDepth int) {
for len(kvs) > 11 {
// To avoid the worst case of quickSort (time: O(n^2)), use introsort here.
// Reference: https://en.wikipedia.org/wiki/Introsort and
// https://github.com/golang/go/issues/467
if maxDepth == 0 {
heapSort(kvs, 0, len(kvs))
return
}
maxDepth--
p := pivot(kvs, d)
lt, i, gt := 0, 0, len(kvs)
for i < gt {
c := byteAt(kvs[i].k, d)
if c < p {
swap(kvs, lt, i)
i++
lt++
} else if c > p {
gt--
swap(kvs, i, gt)
} else {
i++
}
}
// kvs[0:lt] < v = kvs[lt:gt] < kvs[gt:len(kvs)]
// Native implemention:
// radixQsort(kvs[:lt], d, maxDepth)
// if p > -1 {
// radixQsort(kvs[lt:gt], d+1, maxDepth)
// }
// radixQsort(kvs[gt:], d, maxDepth)
// Optimize as follows: make recursive calls only for the smaller parts.
// Reference: https://www.geeksforgeeks.org/quicksort-tail-call-optimization-reducing-worst-case-space-log-n/
if p == -1 {
if lt > len(kvs) - gt {
radixQsort(kvs[gt:], d, maxDepth)
kvs = kvs[:lt]
} else {
radixQsort(kvs[:lt], d, maxDepth)
kvs = kvs[gt:]
}
} else {
ml := maxThree(lt, gt-lt, len(kvs)-gt)
if ml == lt {
radixQsort(kvs[lt:gt], d+1, maxDepth)
radixQsort(kvs[gt:], d, maxDepth)
kvs = kvs[:lt]
} else if ml == gt-lt {
radixQsort(kvs[:lt], d, maxDepth)
radixQsort(kvs[gt:], d, maxDepth)
kvs = kvs[lt:gt]
d += 1
} else {
radixQsort(kvs[:lt], d, maxDepth)
radixQsort(kvs[lt:gt], d+1, maxDepth)
kvs = kvs[gt:]
}
}
}
insertRadixSort(kvs, d)
}
func insertRadixSort(kvs []_MapPair, d int) {
for i := 1; i < len(kvs); i++ {
for j := i; j > 0 && lessFrom(kvs[j].k, kvs[j-1].k, d); j-- {
swap(kvs, j, j-1)
}
}
}
func pivot(kvs []_MapPair, d int) int {
m := len(kvs) >> 1
if len(kvs) > 40 {
// Tukey's ``Ninther,'' median of three mediankvs of three.
t := len(kvs) / 8
return medianThree(
medianThree(byteAt(kvs[0].k, d), byteAt(kvs[t].k, d), byteAt(kvs[2*t].k, d)),
medianThree(byteAt(kvs[m].k, d), byteAt(kvs[m-t].k, d), byteAt(kvs[m+t].k, d)),
medianThree(byteAt(kvs[len(kvs)-1].k, d),
byteAt(kvs[len(kvs)-1-t].k, d),
byteAt(kvs[len(kvs)-1-2*t].k, d)))
}
return medianThree(byteAt(kvs[0].k, d), byteAt(kvs[m].k, d), byteAt(kvs[len(kvs)-1].k, d))
}
func medianThree(i, j, k int) int {
if i > j {
i, j = j, i
} // i < j
if k < i {
return i
}
if k > j {
return j
}
return k
}
func maxThree(i, j, k int) int {
max := i
if max < j {
max = j
}
if max < k {
max = k
}
return max
}
// maxDepth returns a threshold at which quicksort should switch
// to heapsort. It returnkvs 2*ceil(lg(n+1)).
func maxDepth(n int) int {
var depth int
for i := n; i > 0; i >>= 1 {
depth++
}
return depth * 2
}
// siftDown implements the heap property on kvs[lo:hi].
// first is an offset into the array where the root of the heap lies.
func siftDown(kvs []_MapPair, lo, hi, first int) {
root := lo
for {
child := 2*root + 1
if child >= hi {
break
}
if child+1 < hi && kvs[first+child].k < kvs[first+child+1].k {
child++
}
if kvs[first+root].k >= kvs[first+child].k {
return
}
swap(kvs, first+root, first+child)
root = child
}
}
func heapSort(kvs []_MapPair, a, b int) {
first := a
lo := 0
hi := b - a
// Build heap with the greatest element at top.
for i := (hi - 1) / 2; i >= 0; i-- {
siftDown(kvs, i, hi, first)
}
// Pop elements, the largest first, into end of kvs.
for i := hi - 1; i >= 0; i-- {
swap(kvs, first, first+i)
siftDown(kvs, lo, i, first)
}
}
// Note that _MapPair.k is NOT pointed to _MapPair.m when map key is integer after swap
func swap(kvs []_MapPair, a, b int) {
kvs[a].k, kvs[b].k = kvs[b].k, kvs[a].k
kvs[a].v, kvs[b].v = kvs[b].v, kvs[a].v
}
// Compare two strings from the pos d.
func lessFrom(a, b string, d int) bool {
l := len(a)
if l > len(b) {
l = len(b)
}
for i := d; i < l; i++ {
if a[i] == b[i] {
continue
}
return a[i] < b[i]
}
return len(a) < len(b)
}
func byteAt(b string, p int) int {
if p < len(b) {
return int(b[p])
}
return -1
}

35
vendor/github.com/bytedance/sonic/internal/encoder/stream.go generated vendored
View File

@@ -17,8 +17,10 @@
package encoder package encoder
import ( import (
`encoding/json` "encoding/json"
`io` "io"
"github.com/bytedance/sonic/internal/encoder/vars"
) )
// StreamEncoder uses io.Writer as input. // StreamEncoder uses io.Writer as input.
@@ -36,49 +38,54 @@ func NewStreamEncoder(w io.Writer) *StreamEncoder {
// Encode encodes interface{} as JSON to io.Writer // Encode encodes interface{} as JSON to io.Writer
func (enc *StreamEncoder) Encode(val interface{}) (err error) { func (enc *StreamEncoder) Encode(val interface{}) (err error) {
out := newBytes() out := vars.NewBytes()
/* encode into the buffer */ /* encode into the buffer */
err = EncodeInto(&out, val, enc.Opts) err = EncodeInto(out, val, enc.Opts)
if err != nil { if err != nil {
goto free_bytes goto free_bytes
} }
if enc.indent != "" || enc.prefix != "" { if enc.indent != "" || enc.prefix != "" {
/* indent the JSON */ /* indent the JSON */
buf := newBuffer() buf := vars.NewBuffer()
err = json.Indent(buf, out, enc.prefix, enc.indent) err = json.Indent(buf, *out, enc.prefix, enc.indent)
if err != nil { if err != nil {
freeBuffer(buf) vars.FreeBuffer(buf)
goto free_bytes goto free_bytes
} }
// according to standard library, terminate each value with a newline... // according to standard library, terminate each value with a newline...
buf.WriteByte('\n') if enc.Opts & NoEncoderNewline == 0 {
buf.WriteByte('\n')
}
/* copy into io.Writer */ /* copy into io.Writer */
_, err = io.Copy(enc.w, buf) _, err = io.Copy(enc.w, buf)
if err != nil { if err != nil {
freeBuffer(buf) vars.FreeBuffer(buf)
goto free_bytes goto free_bytes
} }
} else { } else {
/* copy into io.Writer */ /* copy into io.Writer */
var n int var n int
for len(out) > 0 { buf := *out
n, err = enc.w.Write(out) for len(buf) > 0 {
out = out[n:] n, err = enc.w.Write(buf)
buf = buf[n:]
if err != nil { if err != nil {
goto free_bytes goto free_bytes
} }
} }
// according to standard library, terminate each value with a newline... // according to standard library, terminate each value with a newline...
enc.w.Write([]byte{'\n'}) if enc.Opts & NoEncoderNewline == 0 {
enc.w.Write([]byte{'\n'})
}
} }
free_bytes: free_bytes:
freeBytes(out) vars.FreeBytes(out)
return err return err
} }

65
vendor/github.com/bytedance/sonic/internal/encoder/stubs_go116.go generated vendored
View File

@@ -1,65 +0,0 @@
// +build go1.16,!go1.17
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package encoder
import (
`unsafe`
_ `github.com/chenzhuoyu/base64x`
`github.com/bytedance/sonic/internal/rt`
)
//go:linkname _subr__b64encode github.com/chenzhuoyu/base64x._subr__b64encode
var _subr__b64encode uintptr
//go:noescape
//go:linkname memmove runtime.memmove
//goland:noinspection GoUnusedParameter
func memmove(to unsafe.Pointer, from unsafe.Pointer, n uintptr)
//go:linkname growslice runtime.growslice
//goland:noinspection GoUnusedParameter
func growslice(et *rt.GoType, old rt.GoSlice, cap int) rt.GoSlice
//go:linkname assertI2I runtime.assertI2I
//goland:noinspection GoUnusedParameter
func assertI2I(inter *rt.GoType, i rt.GoIface) rt.GoIface
//go:linkname mapiternext runtime.mapiternext
//goland:noinspection GoUnusedParameter
func mapiternext(it *rt.GoMapIterator)
//go:linkname mapiterinit runtime.mapiterinit
//goland:noinspection GoUnusedParameter
func mapiterinit(t *rt.GoMapType, m *rt.GoMap, it *rt.GoMapIterator)
//go:linkname isValidNumber encoding/json.isValidNumber
//goland:noinspection GoUnusedParameter
func isValidNumber(s string) bool
//go:noescape
//go:linkname memclrNoHeapPointers runtime.memclrNoHeapPointers
//goland:noinspection GoUnusedParameter
func memclrNoHeapPointers(ptr unsafe.Pointer, n uintptr)
var _runtime_writeBarrier uintptr = rt.GcwbAddr()
//go:linkname gcWriteBarrierAX runtime.gcWriteBarrier
func gcWriteBarrierAX()

66
vendor/github.com/bytedance/sonic/internal/encoder/stubs_go117.go generated vendored
View File

@@ -1,66 +0,0 @@
// +build go1.17,!go1.20
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package encoder
import (
`unsafe`
_ `github.com/chenzhuoyu/base64x`
`github.com/bytedance/sonic/internal/rt`
)
//go:linkname _subr__b64encode github.com/chenzhuoyu/base64x._subr__b64encode
var _subr__b64encode uintptr
//go:noescape
//go:linkname memmove runtime.memmove
//goland:noinspection GoUnusedParameter
func memmove(to unsafe.Pointer, from unsafe.Pointer, n uintptr)
//go:linkname growslice runtime.growslice
//goland:noinspection GoUnusedParameter
func growslice(et *rt.GoType, old rt.GoSlice, cap int) rt.GoSlice
//go:linkname assertI2I runtime.assertI2I2
//goland:noinspection GoUnusedParameter
func assertI2I(inter *rt.GoType, i rt.GoIface) rt.GoIface
//go:linkname mapiternext runtime.mapiternext
//goland:noinspection GoUnusedParameter
func mapiternext(it *rt.GoMapIterator)
//go:linkname mapiterinit runtime.mapiterinit
//goland:noinspection GoUnusedParameter
func mapiterinit(t *rt.GoMapType, m *rt.GoMap, it *rt.GoMapIterator)
//go:linkname isValidNumber encoding/json.isValidNumber
//goland:noinspection GoUnusedParameter
func isValidNumber(s string) bool
//go:noescape
//go:linkname memclrNoHeapPointers runtime.memclrNoHeapPointers
//goland:noinspection GoUnusedParameter
func memclrNoHeapPointers(ptr unsafe.Pointer, n uintptr)
//go:linkname _runtime_writeBarrier runtime.writeBarrier
var _runtime_writeBarrier uintptr
//go:linkname gcWriteBarrierAX runtime.gcWriteBarrier
func gcWriteBarrierAX()

66
vendor/github.com/bytedance/sonic/internal/encoder/stubs_go120.go generated vendored
View File

@@ -1,66 +0,0 @@
// +build go1.20,!go1.21
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package encoder
import (
`unsafe`
_ `github.com/chenzhuoyu/base64x`
`github.com/bytedance/sonic/internal/rt`
)
//go:linkname _subr__b64encode github.com/chenzhuoyu/base64x._subr__b64encode
var _subr__b64encode uintptr
//go:noescape
//go:linkname memmove runtime.memmove
//goland:noinspection GoUnusedParameter
func memmove(to unsafe.Pointer, from unsafe.Pointer, n uintptr)
//go:linkname growslice reflect.growslice
//goland:noinspection GoUnusedParameter
func growslice(et *rt.GoType, old rt.GoSlice, cap int) rt.GoSlice
//go:linkname assertI2I runtime.assertI2I2
//goland:noinspection GoUnusedParameter
func assertI2I(inter *rt.GoType, i rt.GoIface) rt.GoIface
//go:linkname mapiternext runtime.mapiternext
//goland:noinspection GoUnusedParameter
func mapiternext(it *rt.GoMapIterator)
//go:linkname mapiterinit runtime.mapiterinit
//goland:noinspection GoUnusedParameter
func mapiterinit(t *rt.GoMapType, m *rt.GoMap, it *rt.GoMapIterator)
//go:linkname isValidNumber encoding/json.isValidNumber
//goland:noinspection GoUnusedParameter
func isValidNumber(s string) bool
//go:noescape
//go:linkname memclrNoHeapPointers runtime.memclrNoHeapPointers
//goland:noinspection GoUnusedParameter
func memclrNoHeapPointers(ptr unsafe.Pointer, n uintptr)
//go:linkname _runtime_writeBarrier runtime.writeBarrier
var _runtime_writeBarrier uintptr
//go:linkname gcWriteBarrierAX runtime.gcWriteBarrier
func gcWriteBarrierAX()

47
vendor/github.com/bytedance/sonic/internal/encoder/types.go generated vendored
View File

@@ -1,47 +0,0 @@
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package encoder
import (
`encoding`
`encoding/json`
`reflect`
)
var (
byteType = reflect.TypeOf(byte(0))
jsonNumberType = reflect.TypeOf(json.Number(""))
jsonUnsupportedValueType = reflect.TypeOf(new(json.UnsupportedValueError))
)
var (
errorType = reflect.TypeOf((*error)(nil)).Elem()
jsonMarshalerType = reflect.TypeOf((*json.Marshaler)(nil)).Elem()
encodingTextMarshalerType = reflect.TypeOf((*encoding.TextMarshaler)(nil)).Elem()
)
func isSimpleByte(vt reflect.Type) bool {
if vt.Kind() != byteType.Kind() {
return false
} else {
return !isEitherMarshaler(vt) && !isEitherMarshaler(reflect.PtrTo(vt))
}
}
func isEitherMarshaler(vt reflect.Type) bool {
return vt.Implements(jsonMarshalerType) || vt.Implements(encodingTextMarshalerType)
}

52
vendor/github.com/bytedance/sonic/internal/encoder/utils.go generated vendored
View File

@@ -1,52 +0,0 @@
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package encoder
import (
`encoding/json`
`unsafe`
`github.com/bytedance/sonic/loader`
)
//go:nosplit
func padd(p unsafe.Pointer, v int) unsafe.Pointer {
return unsafe.Pointer(uintptr(p) + uintptr(v))
}
//go:nosplit
func ptoenc(p loader.Function) _Encoder {
return *(*_Encoder)(unsafe.Pointer(&p))
}
func compact(p *[]byte, v []byte) error {
buf := newBuffer()
err := json.Compact(buf, v)
/* check for errors */
if err != nil {
return err
}
/* add to result */
v = buf.Bytes()
*p = append(*p, v...)
/* return the buffer into pool */
freeBuffer(buf)
return nil
}

13
vendor/github.com/bytedance/sonic/internal/jit/arch_amd64.go generated vendored
View File

@@ -17,8 +17,10 @@
package jit package jit
import ( import (
`github.com/twitchyliquid64/golang-asm/asm/arch` "unsafe"
`github.com/twitchyliquid64/golang-asm/obj`
"github.com/twitchyliquid64/golang-asm/asm/arch"
"github.com/twitchyliquid64/golang-asm/obj"
) )
var ( var (
@@ -33,6 +35,13 @@ func As(op string) obj.As {
} }
} }
func ImmPtr(imm unsafe.Pointer) obj.Addr {
return obj.Addr {
Type : obj.TYPE_CONST,
Offset : int64(uintptr(imm)),
}
}
func Imm(imm int64) obj.Addr { func Imm(imm int64) obj.Addr {
return obj.Addr { return obj.Addr {
Type : obj.TYPE_CONST, Type : obj.TYPE_CONST,

12
vendor/github.com/bytedance/sonic/internal/jit/assembler_amd64.go generated vendored
View File

@@ -72,18 +72,6 @@ func (self *BaseAssembler) NOPn(n int) {
} }
} }
func (self *BaseAssembler) StorePtr(ptr int64, to obj.Addr, tmp obj.Addr) {
if (to.Type != obj.TYPE_MEM) || (tmp.Type != obj.TYPE_REG) {
panic("must store imm to memory, tmp must be register")
}
if (ptr >> 32) != 0 {
self.Emit("MOVQ", Imm(ptr), tmp)
self.Emit("MOVQ", tmp, to)
} else {
self.Emit("MOVQ", Imm(ptr), to);
}
}
func (self *BaseAssembler) Byte(v ...byte) { func (self *BaseAssembler) Byte(v ...byte) {
for ; len(v) >= 8; v = v[8:] { self.From("QUAD", Imm(rt.Get64(v))) } for ; len(v) >= 8; v = v[8:] { self.From("QUAD", Imm(rt.Get64(v))) }
for ; len(v) >= 4; v = v[4:] { self.From("LONG", Imm(int64(rt.Get32(v)))) } for ; len(v) >= 4; v = v[4:] { self.From("LONG", Imm(int64(rt.Get32(v)))) }

7
vendor/github.com/bytedance/sonic/internal/jit/backend.go generated vendored
View File

@@ -21,6 +21,7 @@ import (
`sync` `sync`
_ `unsafe` _ `unsafe`
`github.com/bytedance/sonic/internal/rt`
`github.com/twitchyliquid64/golang-asm/asm/arch` `github.com/twitchyliquid64/golang-asm/asm/arch`
`github.com/twitchyliquid64/golang-asm/obj` `github.com/twitchyliquid64/golang-asm/obj`
`github.com/twitchyliquid64/golang-asm/objabi` `github.com/twitchyliquid64/golang-asm/objabi`
@@ -38,10 +39,6 @@ var (
_progPool sync.Pool _progPool sync.Pool
) )
//go:nosplit
//go:linkname throw runtime.throw
func throw(_ string)
func newProg() *obj.Prog { func newProg() *obj.Prog {
if val := _progPool.Get(); val == nil { if val := _progPool.Get(); val == nil {
return new(obj.Prog) return new(obj.Prog)
@@ -71,7 +68,7 @@ func newLinkContext(arch *obj.LinkArch) (ret *obj.Link) {
} }
func diagLinkContext(str string, args ...interface{}) { func diagLinkContext(str string, args ...interface{}) {
throw(fmt.Sprintf(str, args...)) rt.Throw(fmt.Sprintf(str, args...))
} }
func (self *Backend) New() (ret *obj.Prog) { func (self *Backend) New() (ret *obj.Prog) {

7
vendor/github.com/bytedance/sonic/internal/jit/runtime.go generated vendored
View File

@@ -24,11 +24,6 @@ import (
`github.com/twitchyliquid64/golang-asm/obj` `github.com/twitchyliquid64/golang-asm/obj`
) )
//go:noescape
//go:linkname getitab runtime.getitab
//goland:noinspection ALL
func getitab(inter *rt.GoType, typ *rt.GoType, canfail bool) *rt.GoItab
func Func(f interface{}) obj.Addr { func Func(f interface{}) obj.Addr {
if p := rt.UnpackEface(f); p.Type.Kind() != reflect.Func { if p := rt.UnpackEface(f); p.Type.Kind() != reflect.Func {
panic("f is not a function") panic("f is not a function")
@@ -42,7 +37,7 @@ func Type(t reflect.Type) obj.Addr {
} }
func Itab(i *rt.GoType, t reflect.Type) obj.Addr { func Itab(i *rt.GoType, t reflect.Type) obj.Addr {
return Imm(int64(uintptr(unsafe.Pointer(getitab(i, rt.UnpackType(t), false))))) return Imm(int64(uintptr(unsafe.Pointer(rt.GetItab(rt.IfaceType(i), rt.UnpackType(t), false)))))
} }
func Gitab(i *rt.GoItab) obj.Addr { func Gitab(i *rt.GoItab) obj.Addr {

191
vendor/github.com/bytedance/sonic/internal/native/avx/native_amd64.go generated vendored
View File

@@ -1,191 +0,0 @@
// Code generated by Makefile, DO NOT EDIT.
// Code generated by Makefile, DO NOT EDIT.
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package avx
import (
`unsafe`
`github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/internal/rt`
)
var (
__i64toa func(out unsafe.Pointer, val int64) (ret int)
__u64toa func(out unsafe.Pointer, val uint64) (ret int)
__f64toa func(out unsafe.Pointer, val float64) (ret int)
__f32toa func(out unsafe.Pointer, val float32) (ret int)
__lspace func(sp unsafe.Pointer, nb int, off int) (ret int)
__quote func(sp unsafe.Pointer, nb int, dp unsafe.Pointer, dn unsafe.Pointer, flags uint64) (ret int)
__html_escape func(sp unsafe.Pointer, nb int, dp unsafe.Pointer, dn unsafe.Pointer) (ret int)
__unquote func(sp unsafe.Pointer, nb int, dp unsafe.Pointer, ep unsafe.Pointer, flags uint64) (ret int)
__value func(s unsafe.Pointer, n int, p int, v unsafe.Pointer, flags uint64) (ret int)
__vstring func(s unsafe.Pointer, p unsafe.Pointer, v unsafe.Pointer, flags uint64)
__vnumber func(s unsafe.Pointer, p unsafe.Pointer, v unsafe.Pointer)
__vsigned func(s unsafe.Pointer, p unsafe.Pointer, v unsafe.Pointer)
__vunsigned func(s unsafe.Pointer, p unsafe.Pointer, v unsafe.Pointer)
__skip_one func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer, flags uint64) (ret int)
__skip_one_fast func(s unsafe.Pointer, p unsafe.Pointer) (ret int)
__skip_array func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer, flags uint64) (ret int)
__skip_object func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer, flags uint64) (ret int)
__skip_number func(s unsafe.Pointer, p unsafe.Pointer) (ret int)
__validate_one func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer) (ret int)
__get_by_path func(s unsafe.Pointer, p unsafe.Pointer, path unsafe.Pointer, m unsafe.Pointer) (ret int)
__validate_utf8 func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer) (ret int)
__validate_utf8_fast func(s unsafe.Pointer) (ret int)
__fsm_exec func(m unsafe.Pointer, s unsafe.Pointer, p unsafe.Pointer, flags uint64) (ret int)
)
//go:nosplit
func i64toa(out *byte, val int64) (ret int) {
return __i64toa(rt.NoEscape(unsafe.Pointer(out)), val)
}
//go:nosplit
func u64toa(out *byte, val uint64) (ret int) {
return __u64toa(rt.NoEscape(unsafe.Pointer(out)), val)
}
//go:nosplit
func f64toa(out *byte, val float64) (ret int) {
return __f64toa(rt.NoEscape(unsafe.Pointer(out)), val)
}
//go:nosplit
func f32toa(out *byte, val float32) (ret int) {
return __f32toa(rt.NoEscape(unsafe.Pointer(out)), val)
}
//go:nosplit
func lspace(sp unsafe.Pointer, nb int, off int) (ret int) {
return __lspace(rt.NoEscape(sp), nb, off)
}
//go:nosplit
func quote(sp unsafe.Pointer, nb int, dp unsafe.Pointer, dn *int, flags uint64) (ret int) {
return __quote(rt.NoEscape(unsafe.Pointer(sp)), nb, rt.NoEscape(unsafe.Pointer(dp)), rt.NoEscape(unsafe.Pointer(dn)), flags)
}
//go:nosplit
func html_escape(sp unsafe.Pointer, nb int, dp unsafe.Pointer, dn *int) (ret int) {
return __html_escape(rt.NoEscape(unsafe.Pointer(sp)), nb, rt.NoEscape(unsafe.Pointer(dp)), rt.NoEscape(unsafe.Pointer(dn)))
}
//go:nosplit
func unquote(sp unsafe.Pointer, nb int, dp unsafe.Pointer, ep *int, flags uint64) (ret int) {
return __unquote(rt.NoEscape(unsafe.Pointer(sp)), nb, rt.NoEscape(unsafe.Pointer(dp)), rt.NoEscape(unsafe.Pointer(ep)), flags)
}
//go:nosplit
func value(s unsafe.Pointer, n int, p int, v *types.JsonState, flags uint64) (ret int) {
return __value(rt.NoEscape(unsafe.Pointer(s)), n, p, rt.NoEscape(unsafe.Pointer(v)), flags)
}
//go:nosplit
func vstring(s *string, p *int, v *types.JsonState, flags uint64) {
__vstring(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(v)), flags)
}
//go:nosplit
func vnumber(s *string, p *int, v *types.JsonState) {
__vnumber(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(v)))
}
//go:nosplit
func vsigned(s *string, p *int, v *types.JsonState) {
__vsigned(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(v)))
}
//go:nosplit
func vunsigned(s *string, p *int, v *types.JsonState) {
__vunsigned(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(v)))
}
//go:nosplit
func skip_one(s *string, p *int, m *types.StateMachine, flags uint64) (ret int) {
return __skip_one(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(m)), flags)
}
//go:nosplit
func skip_one_fast(s *string, p *int) (ret int) {
return __skip_one_fast(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)))
}
//go:nosplit
func skip_array(s *string, p *int, m *types.StateMachine, flags uint64) (ret int) {
return __skip_array(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(m)), flags)
}
//go:nosplit
func skip_object(s *string, p *int, m *types.StateMachine, flags uint64) (ret int) {
return __skip_object(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(m)), flags)
}
//go:nosplit
func skip_number(s *string, p *int) (ret int) {
return __skip_number(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)))
}
//go:nosplit
func validate_one(s *string, p *int, m *types.StateMachine) (ret int) {
return __validate_one(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(m)))
}
//go:nosplit
func get_by_path(s *string, p *int, path *[]interface{}, m *types.StateMachine) (ret int) {
return __get_by_path(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(path)), rt.NoEscape(unsafe.Pointer(m)))
}
//go:nosplit
func validate_utf8(s *string, p *int, m *types.StateMachine) (ret int) {
return __validate_utf8(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(m)))
}
//go:nosplit
func validate_utf8_fast(s *string) (ret int) {
return __validate_utf8_fast(rt.NoEscape(unsafe.Pointer(s)))
}
//go:nosplit
func fsm_exec(m *types.StateMachine, s *string, p *int, flags uint64) (ret int) {
return __fsm_exec(rt.NoEscape(unsafe.Pointer(m)), rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), flags)
}

603
vendor/github.com/bytedance/sonic/internal/native/avx/native_subr_amd64.go generated vendored
View File

@@ -1,603 +0,0 @@
// +build !noasm !appengine
// Code generated by asm2asm, DO NOT EDIT.
package avx
import (
`github.com/bytedance/sonic/loader`
)
const (
_entry__f32toa = 31136
_entry__f64toa = 192
_entry__format_significand = 35248
_entry__format_integer = 3040
_entry__fsm_exec = 17920
_entry__advance_string = 14320
_entry__advance_string_default = 36640
_entry__do_skip_number = 20528
_entry__get_by_path = 25680
_entry__skip_one_fast = 22160
_entry__html_escape = 8912
_entry__i64toa = 3472
_entry__u64toa = 3584
_entry__lspace = 16
_entry__quote = 4864
_entry__skip_array = 17872
_entry__skip_number = 21792
_entry__skip_object = 20160
_entry__skip_one = 21936
_entry__unquote = 6576
_entry__validate_one = 21984
_entry__validate_utf8 = 29888
_entry__validate_utf8_fast = 30560
_entry__value = 12320
_entry__vnumber = 15648
_entry__atof_eisel_lemire64 = 10160
_entry__atof_native = 11712
_entry__decimal_to_f64 = 10528
_entry__right_shift = 36208
_entry__left_shift = 35712
_entry__vsigned = 17200
_entry__vstring = 14144
_entry__vunsigned = 17520
)
const (
_stack__f32toa = 48
_stack__f64toa = 80
_stack__format_significand = 24
_stack__format_integer = 16
_stack__fsm_exec = 168
_stack__advance_string = 64
_stack__advance_string_default = 64
_stack__do_skip_number = 48
_stack__get_by_path = 280
_stack__skip_one_fast = 176
_stack__html_escape = 72
_stack__i64toa = 16
_stack__u64toa = 8
_stack__lspace = 8
_stack__quote = 56
_stack__skip_array = 176
_stack__skip_number = 88
_stack__skip_object = 176
_stack__skip_one = 176
_stack__unquote = 88
_stack__validate_one = 176
_stack__validate_utf8 = 48
_stack__validate_utf8_fast = 24
_stack__value = 328
_stack__vnumber = 240
_stack__atof_eisel_lemire64 = 32
_stack__atof_native = 136
_stack__decimal_to_f64 = 80
_stack__right_shift = 8
_stack__left_shift = 24
_stack__vsigned = 16
_stack__vstring = 120
_stack__vunsigned = 8
)
const (
_size__f32toa = 3392
_size__f64toa = 2848
_size__format_significand = 464
_size__format_integer = 432
_size__fsm_exec = 1692
_size__advance_string = 1280
_size__advance_string_default = 944
_size__do_skip_number = 924
_size__get_by_path = 4208
_size__skip_one_fast = 3016
_size__html_escape = 1248
_size__i64toa = 48
_size__u64toa = 1232
_size__lspace = 144
_size__quote = 1696
_size__skip_array = 48
_size__skip_number = 144
_size__skip_object = 48
_size__skip_one = 48
_size__unquote = 2272
_size__validate_one = 48
_size__validate_utf8 = 672
_size__validate_utf8_fast = 544
_size__value = 1316
_size__vnumber = 1552
_size__atof_eisel_lemire64 = 368
_size__atof_native = 608
_size__decimal_to_f64 = 1184
_size__right_shift = 400
_size__left_shift = 496
_size__vsigned = 320
_size__vstring = 128
_size__vunsigned = 336
)
var (
_pcsp__f32toa = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{3350, 48},
{3351, 40},
{3353, 32},
{3355, 24},
{3357, 16},
{3359, 8},
{3363, 0},
{3385, 48},
}
_pcsp__f64toa = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{2788, 56},
{2792, 48},
{2793, 40},
{2795, 32},
{2797, 24},
{2799, 16},
{2801, 8},
{2805, 0},
{2843, 56},
}
_pcsp__format_significand = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{452, 24},
{453, 16},
{455, 8},
{457, 0},
}
_pcsp__format_integer = [][2]uint32{
{1, 0},
{4, 8},
{412, 16},
{413, 8},
{414, 0},
{423, 16},
{424, 8},
{426, 0},
}
_pcsp__fsm_exec = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{1342, 104},
{1346, 48},
{1347, 40},
{1349, 32},
{1351, 24},
{1353, 16},
{1355, 8},
{1356, 0},
{1692, 104},
}
_pcsp__advance_string = [][2]uint32{
{14, 0},
{18, 8},
{20, 16},
{22, 24},
{24, 32},
{26, 40},
{27, 48},
{557, 56},
{561, 48},
{562, 40},
{564, 32},
{566, 24},
{568, 16},
{570, 8},
{571, 0},
{1268, 56},
}
_pcsp__advance_string_default = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{552, 64},
{556, 48},
{557, 40},
{559, 32},
{561, 24},
{563, 16},
{565, 8},
{566, 0},
{931, 64},
}
_pcsp__do_skip_number = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{849, 48},
{850, 40},
{852, 32},
{854, 24},
{856, 16},
{858, 8},
{859, 0},
{924, 48},
}
_pcsp__get_by_path = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{4012, 104},
{4016, 48},
{4017, 40},
{4019, 32},
{4021, 24},
{4023, 16},
{4025, 8},
{4026, 0},
{4194, 104},
}
_pcsp__skip_one_fast = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{658, 176},
{659, 168},
{661, 160},
{663, 152},
{665, 144},
{667, 136},
{671, 128},
{3016, 176},
}
_pcsp__html_escape = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{1224, 72},
{1228, 48},
{1229, 40},
{1231, 32},
{1233, 24},
{1235, 16},
{1237, 8},
{1239, 0},
}
_pcsp__i64toa = [][2]uint32{
{14, 0},
{34, 8},
{36, 0},
}
_pcsp__u64toa = [][2]uint32{
{1, 0},
{161, 8},
{162, 0},
{457, 8},
{458, 0},
{756, 8},
{757, 0},
{1221, 8},
{1223, 0},
}
_pcsp__lspace = [][2]uint32{
{1, 0},
{89, 8},
{90, 0},
{103, 8},
{104, 0},
{111, 8},
{113, 0},
}
_pcsp__quote = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{1649, 56},
{1653, 48},
{1654, 40},
{1656, 32},
{1658, 24},
{1660, 16},
{1662, 8},
{1663, 0},
{1690, 56},
}
_pcsp__skip_array = [][2]uint32{
{1, 0},
{28, 8},
{34, 0},
}
_pcsp__skip_number = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{100, 40},
{101, 32},
{103, 24},
{105, 16},
{107, 8},
{108, 0},
{139, 40},
}
_pcsp__skip_object = [][2]uint32{
{1, 0},
{28, 8},
{34, 0},
}
_pcsp__skip_one = [][2]uint32{
{1, 0},
{30, 8},
{36, 0},
}
_pcsp__unquote = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{1684, 88},
{1688, 48},
{1689, 40},
{1691, 32},
{1693, 24},
{1695, 16},
{1697, 8},
{1698, 0},
{2270, 88},
}
_pcsp__validate_one = [][2]uint32{
{1, 0},
{35, 8},
{41, 0},
}
_pcsp__validate_utf8 = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{11, 40},
{623, 48},
{627, 40},
{628, 32},
{630, 24},
{632, 16},
{634, 8},
{635, 0},
{666, 48},
}
_pcsp__validate_utf8_fast = [][2]uint32{
{1, 0},
{4, 8},
{5, 16},
{247, 24},
{251, 16},
{252, 8},
{253, 0},
{527, 24},
{531, 16},
{532, 8},
{534, 0},
}
_pcsp__value = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{495, 88},
{499, 48},
{500, 40},
{502, 32},
{504, 24},
{506, 16},
{508, 8},
{509, 0},
{1316, 88},
}
_pcsp__vnumber = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{803, 104},
{807, 48},
{808, 40},
{810, 32},
{812, 24},
{814, 16},
{816, 8},
{817, 0},
{1547, 104},
}
_pcsp__atof_eisel_lemire64 = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{292, 32},
{293, 24},
{295, 16},
{297, 8},
{298, 0},
{362, 32},
}
_pcsp__atof_native = [][2]uint32{
{1, 0},
{4, 8},
{587, 56},
{591, 8},
{593, 0},
}
_pcsp__decimal_to_f64 = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{1144, 56},
{1148, 48},
{1149, 40},
{1151, 32},
{1153, 24},
{1155, 16},
{1157, 8},
{1158, 0},
{1169, 56},
}
_pcsp__right_shift = [][2]uint32{
{1, 0},
{318, 8},
{319, 0},
{387, 8},
{388, 0},
{396, 8},
{398, 0},
}
_pcsp__left_shift = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{363, 24},
{364, 16},
{366, 8},
{367, 0},
{470, 24},
{471, 16},
{473, 8},
{474, 0},
{486, 24},
}
_pcsp__vsigned = [][2]uint32{
{1, 0},
{4, 8},
{112, 16},
{113, 8},
{114, 0},
{125, 16},
{126, 8},
{127, 0},
{260, 16},
{261, 8},
{262, 0},
{266, 16},
{267, 8},
{268, 0},
{306, 16},
{307, 8},
{308, 0},
{316, 16},
{317, 8},
{319, 0},
}
_pcsp__vstring = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{11, 40},
{105, 56},
{109, 40},
{110, 32},
{112, 24},
{114, 16},
{116, 8},
{118, 0},
}
_pcsp__vunsigned = [][2]uint32{
{1, 0},
{71, 8},
{72, 0},
{83, 8},
{84, 0},
{107, 8},
{108, 0},
{273, 8},
{274, 0},
{312, 8},
{313, 0},
{320, 8},
{322, 0},
}
)
var Funcs = []loader.CFunc{
{"__native_entry__", 0, 67, 0, nil},
{"_f32toa", _entry__f32toa, _size__f32toa, _stack__f32toa, _pcsp__f32toa},
{"_f64toa", _entry__f64toa, _size__f64toa, _stack__f64toa, _pcsp__f64toa},
{"_format_significand", _entry__format_significand, _size__format_significand, _stack__format_significand, _pcsp__format_significand},
{"_format_integer", _entry__format_integer, _size__format_integer, _stack__format_integer, _pcsp__format_integer},
{"_fsm_exec", _entry__fsm_exec, _size__fsm_exec, _stack__fsm_exec, _pcsp__fsm_exec},
{"_advance_string", _entry__advance_string, _size__advance_string, _stack__advance_string, _pcsp__advance_string},
{"_advance_string_default", _entry__advance_string_default, _size__advance_string_default, _stack__advance_string_default, _pcsp__advance_string_default},
{"_do_skip_number", _entry__do_skip_number, _size__do_skip_number, _stack__do_skip_number, _pcsp__do_skip_number},
{"_get_by_path", _entry__get_by_path, _size__get_by_path, _stack__get_by_path, _pcsp__get_by_path},
{"_skip_one_fast", _entry__skip_one_fast, _size__skip_one_fast, _stack__skip_one_fast, _pcsp__skip_one_fast},
{"_html_escape", _entry__html_escape, _size__html_escape, _stack__html_escape, _pcsp__html_escape},
{"_i64toa", _entry__i64toa, _size__i64toa, _stack__i64toa, _pcsp__i64toa},
{"_u64toa", _entry__u64toa, _size__u64toa, _stack__u64toa, _pcsp__u64toa},
{"_lspace", _entry__lspace, _size__lspace, _stack__lspace, _pcsp__lspace},
{"_quote", _entry__quote, _size__quote, _stack__quote, _pcsp__quote},
{"_skip_array", _entry__skip_array, _size__skip_array, _stack__skip_array, _pcsp__skip_array},
{"_skip_number", _entry__skip_number, _size__skip_number, _stack__skip_number, _pcsp__skip_number},
{"_skip_object", _entry__skip_object, _size__skip_object, _stack__skip_object, _pcsp__skip_object},
{"_skip_one", _entry__skip_one, _size__skip_one, _stack__skip_one, _pcsp__skip_one},
{"_unquote", _entry__unquote, _size__unquote, _stack__unquote, _pcsp__unquote},
{"_validate_one", _entry__validate_one, _size__validate_one, _stack__validate_one, _pcsp__validate_one},
{"_validate_utf8", _entry__validate_utf8, _size__validate_utf8, _stack__validate_utf8, _pcsp__validate_utf8},
{"_validate_utf8_fast", _entry__validate_utf8_fast, _size__validate_utf8_fast, _stack__validate_utf8_fast, _pcsp__validate_utf8_fast},
{"_value", _entry__value, _size__value, _stack__value, _pcsp__value},
{"_vnumber", _entry__vnumber, _size__vnumber, _stack__vnumber, _pcsp__vnumber},
{"_atof_eisel_lemire64", _entry__atof_eisel_lemire64, _size__atof_eisel_lemire64, _stack__atof_eisel_lemire64, _pcsp__atof_eisel_lemire64},
{"_atof_native", _entry__atof_native, _size__atof_native, _stack__atof_native, _pcsp__atof_native},
{"_decimal_to_f64", _entry__decimal_to_f64, _size__decimal_to_f64, _stack__decimal_to_f64, _pcsp__decimal_to_f64},
{"_right_shift", _entry__right_shift, _size__right_shift, _stack__right_shift, _pcsp__right_shift},
{"_left_shift", _entry__left_shift, _size__left_shift, _stack__left_shift, _pcsp__left_shift},
{"_vsigned", _entry__vsigned, _size__vsigned, _stack__vsigned, _pcsp__vsigned},
{"_vstring", _entry__vstring, _size__vstring, _stack__vstring, _pcsp__vstring},
{"_vunsigned", _entry__vunsigned, _size__vunsigned, _stack__vunsigned, _pcsp__vunsigned},
}

191
vendor/github.com/bytedance/sonic/internal/native/avx2/native_amd64.go generated vendored
View File

@@ -1,191 +0,0 @@
// Code generated by Makefile, DO NOT EDIT.
// Code generated by Makefile, DO NOT EDIT.
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package avx2
import (
`unsafe`
`github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/internal/rt`
)
var (
__i64toa func(out unsafe.Pointer, val int64) (ret int)
__u64toa func(out unsafe.Pointer, val uint64) (ret int)
__f64toa func(out unsafe.Pointer, val float64) (ret int)
__f32toa func(out unsafe.Pointer, val float32) (ret int)
__lspace func(sp unsafe.Pointer, nb int, off int) (ret int)
__quote func(sp unsafe.Pointer, nb int, dp unsafe.Pointer, dn unsafe.Pointer, flags uint64) (ret int)
__html_escape func(sp unsafe.Pointer, nb int, dp unsafe.Pointer, dn unsafe.Pointer) (ret int)
__unquote func(sp unsafe.Pointer, nb int, dp unsafe.Pointer, ep unsafe.Pointer, flags uint64) (ret int)
__value func(s unsafe.Pointer, n int, p int, v unsafe.Pointer, flags uint64) (ret int)
__vstring func(s unsafe.Pointer, p unsafe.Pointer, v unsafe.Pointer, flags uint64)
__vnumber func(s unsafe.Pointer, p unsafe.Pointer, v unsafe.Pointer)
__vsigned func(s unsafe.Pointer, p unsafe.Pointer, v unsafe.Pointer)
__vunsigned func(s unsafe.Pointer, p unsafe.Pointer, v unsafe.Pointer)
__skip_one func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer, flags uint64) (ret int)
__skip_one_fast func(s unsafe.Pointer, p unsafe.Pointer) (ret int)
__skip_array func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer, flags uint64) (ret int)
__skip_object func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer, flags uint64) (ret int)
__skip_number func(s unsafe.Pointer, p unsafe.Pointer) (ret int)
__validate_one func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer) (ret int)
__get_by_path func(s unsafe.Pointer, p unsafe.Pointer, path unsafe.Pointer, m unsafe.Pointer) (ret int)
__validate_utf8 func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer) (ret int)
__validate_utf8_fast func(s unsafe.Pointer) (ret int)
__fsm_exec func(m unsafe.Pointer, s unsafe.Pointer, p unsafe.Pointer, flags uint64) (ret int)
)
//go:nosplit
func i64toa(out *byte, val int64) (ret int) {
return __i64toa(rt.NoEscape(unsafe.Pointer(out)), val)
}
//go:nosplit
func u64toa(out *byte, val uint64) (ret int) {
return __u64toa(rt.NoEscape(unsafe.Pointer(out)), val)
}
//go:nosplit
func f64toa(out *byte, val float64) (ret int) {
return __f64toa(rt.NoEscape(unsafe.Pointer(out)), val)
}
//go:nosplit
func f32toa(out *byte, val float32) (ret int) {
return __f32toa(rt.NoEscape(unsafe.Pointer(out)), val)
}
//go:nosplit
func lspace(sp unsafe.Pointer, nb int, off int) (ret int) {
return __lspace(rt.NoEscape(sp), nb, off)
}
//go:nosplit
func quote(sp unsafe.Pointer, nb int, dp unsafe.Pointer, dn *int, flags uint64) (ret int) {
return __quote(rt.NoEscape(unsafe.Pointer(sp)), nb, rt.NoEscape(unsafe.Pointer(dp)), rt.NoEscape(unsafe.Pointer(dn)), flags)
}
//go:nosplit
func html_escape(sp unsafe.Pointer, nb int, dp unsafe.Pointer, dn *int) (ret int) {
return __html_escape(rt.NoEscape(unsafe.Pointer(sp)), nb, rt.NoEscape(unsafe.Pointer(dp)), rt.NoEscape(unsafe.Pointer(dn)))
}
//go:nosplit
func unquote(sp unsafe.Pointer, nb int, dp unsafe.Pointer, ep *int, flags uint64) (ret int) {
return __unquote(rt.NoEscape(unsafe.Pointer(sp)), nb, rt.NoEscape(unsafe.Pointer(dp)), rt.NoEscape(unsafe.Pointer(ep)), flags)
}
//go:nosplit
func value(s unsafe.Pointer, n int, p int, v *types.JsonState, flags uint64) (ret int) {
return __value(rt.NoEscape(unsafe.Pointer(s)), n, p, rt.NoEscape(unsafe.Pointer(v)), flags)
}
//go:nosplit
func vstring(s *string, p *int, v *types.JsonState, flags uint64) {
__vstring(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(v)), flags)
}
//go:nosplit
func vnumber(s *string, p *int, v *types.JsonState) {
__vnumber(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(v)))
}
//go:nosplit
func vsigned(s *string, p *int, v *types.JsonState) {
__vsigned(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(v)))
}
//go:nosplit
func vunsigned(s *string, p *int, v *types.JsonState) {
__vunsigned(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(v)))
}
//go:nosplit
func skip_one(s *string, p *int, m *types.StateMachine, flags uint64) (ret int) {
return __skip_one(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(m)), flags)
}
//go:nosplit
func skip_one_fast(s *string, p *int) (ret int) {
return __skip_one_fast(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)))
}
//go:nosplit
func skip_array(s *string, p *int, m *types.StateMachine, flags uint64) (ret int) {
return __skip_array(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(m)), flags)
}
//go:nosplit
func skip_object(s *string, p *int, m *types.StateMachine, flags uint64) (ret int) {
return __skip_object(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(m)), flags)
}
//go:nosplit
func skip_number(s *string, p *int) (ret int) {
return __skip_number(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)))
}
//go:nosplit
func validate_one(s *string, p *int, m *types.StateMachine) (ret int) {
return __validate_one(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(m)))
}
//go:nosplit
func get_by_path(s *string, p *int, path *[]interface{}, m *types.StateMachine) (ret int) {
return __get_by_path(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(path)), rt.NoEscape(unsafe.Pointer(m)))
}
//go:nosplit
func validate_utf8(s *string, p *int, m *types.StateMachine) (ret int) {
return __validate_utf8(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(m)))
}
//go:nosplit
func validate_utf8_fast(s *string) (ret int) {
return __validate_utf8_fast(rt.NoEscape(unsafe.Pointer(s)))
}
//go:nosplit
func fsm_exec(m *types.StateMachine, s *string, p *int, flags uint64) (ret int) {
return __fsm_exec(rt.NoEscape(unsafe.Pointer(m)), rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), flags)
}

610
vendor/github.com/bytedance/sonic/internal/native/avx2/native_subr_amd64.go generated vendored
View File

@@ -1,610 +0,0 @@
// +build !noasm !appengine
// Code generated by asm2asm, DO NOT EDIT.
package avx2
import (
`github.com/bytedance/sonic/loader`
)
const (
_entry__f32toa = 34624
_entry__f64toa = 320
_entry__format_significand = 38736
_entry__format_integer = 3168
_entry__fsm_exec = 21072
_entry__advance_ns = 16928
_entry__advance_string = 17664
_entry__advance_string_default = 40160
_entry__do_skip_number = 23696
_entry__get_by_path = 28864
_entry__skip_one_fast = 25936
_entry__html_escape = 10560
_entry__i64toa = 3600
_entry__u64toa = 3712
_entry__lspace = 64
_entry__quote = 5104
_entry__skip_array = 21024
_entry__skip_number = 25392
_entry__skip_object = 23088
_entry__skip_one = 25536
_entry__unquote = 7888
_entry__validate_one = 25584
_entry__validate_utf8 = 31040
_entry__validate_utf8_fast = 31984
_entry__value = 15376
_entry__vnumber = 18800
_entry__atof_eisel_lemire64 = 12624
_entry__atof_native = 14768
_entry__decimal_to_f64 = 13056
_entry__right_shift = 39696
_entry__left_shift = 39200
_entry__vsigned = 20352
_entry__vstring = 17424
_entry__vunsigned = 20672
)
const (
_stack__f32toa = 48
_stack__f64toa = 80
_stack__format_significand = 24
_stack__format_integer = 16
_stack__fsm_exec = 144
_stack__advance_ns = 8
_stack__advance_string = 56
_stack__advance_string_default = 48
_stack__do_skip_number = 48
_stack__get_by_path = 272
_stack__skip_one_fast = 184
_stack__html_escape = 72
_stack__i64toa = 16
_stack__u64toa = 8
_stack__lspace = 8
_stack__quote = 56
_stack__skip_array = 152
_stack__skip_number = 88
_stack__skip_object = 152
_stack__skip_one = 152
_stack__unquote = 72
_stack__validate_one = 152
_stack__validate_utf8 = 48
_stack__validate_utf8_fast = 176
_stack__value = 328
_stack__vnumber = 240
_stack__atof_eisel_lemire64 = 32
_stack__atof_native = 136
_stack__decimal_to_f64 = 80
_stack__right_shift = 8
_stack__left_shift = 24
_stack__vsigned = 16
_stack__vstring = 112
_stack__vunsigned = 8
)
const (
_size__f32toa = 3392
_size__f64toa = 2848
_size__format_significand = 464
_size__format_integer = 432
_size__fsm_exec = 1468
_size__advance_ns = 496
_size__advance_string = 1088
_size__advance_string_default = 768
_size__do_skip_number = 1360
_size__get_by_path = 2176
_size__skip_one_fast = 2428
_size__html_escape = 2064
_size__i64toa = 48
_size__u64toa = 1248
_size__lspace = 224
_size__quote = 2736
_size__skip_array = 48
_size__skip_number = 144
_size__skip_object = 48
_size__skip_one = 48
_size__unquote = 2480
_size__validate_one = 48
_size__validate_utf8 = 672
_size__validate_utf8_fast = 2608
_size__value = 1004
_size__vnumber = 1552
_size__atof_eisel_lemire64 = 368
_size__atof_native = 608
_size__decimal_to_f64 = 1712
_size__right_shift = 400
_size__left_shift = 496
_size__vsigned = 320
_size__vstring = 144
_size__vunsigned = 336
)
var (
_pcsp__f32toa = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{3350, 48},
{3351, 40},
{3353, 32},
{3355, 24},
{3357, 16},
{3359, 8},
{3363, 0},
{3385, 48},
}
_pcsp__f64toa = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{2788, 56},
{2792, 48},
{2793, 40},
{2795, 32},
{2797, 24},
{2799, 16},
{2801, 8},
{2805, 0},
{2843, 56},
}
_pcsp__format_significand = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{452, 24},
{453, 16},
{455, 8},
{457, 0},
}
_pcsp__format_integer = [][2]uint32{
{1, 0},
{4, 8},
{412, 16},
{413, 8},
{414, 0},
{423, 16},
{424, 8},
{426, 0},
}
_pcsp__fsm_exec = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{1157, 88},
{1161, 48},
{1162, 40},
{1164, 32},
{1166, 24},
{1168, 16},
{1170, 8},
{1171, 0},
{1468, 88},
}
_pcsp__advance_ns = [][2]uint32{
{1, 0},
{453, 8},
{457, 0},
{481, 8},
{486, 0},
}
_pcsp__advance_string = [][2]uint32{
{14, 0},
{18, 8},
{20, 16},
{22, 24},
{24, 32},
{26, 40},
{27, 48},
{433, 56},
{437, 48},
{438, 40},
{440, 32},
{442, 24},
{444, 16},
{446, 8},
{450, 0},
{1078, 56},
}
_pcsp__advance_string_default = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{332, 48},
{333, 40},
{335, 32},
{337, 24},
{339, 16},
{341, 8},
{345, 0},
{757, 48},
}
_pcsp__do_skip_number = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{1274, 48},
{1275, 40},
{1277, 32},
{1279, 24},
{1281, 16},
{1283, 8},
{1287, 0},
{1360, 48},
}
_pcsp__get_by_path = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{2049, 88},
{2053, 48},
{2054, 40},
{2056, 32},
{2058, 24},
{2060, 16},
{2062, 8},
{2063, 0},
{2170, 88},
}
_pcsp__skip_one_fast = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{418, 176},
{419, 168},
{421, 160},
{423, 152},
{425, 144},
{427, 136},
{431, 128},
{2428, 176},
}
_pcsp__html_escape = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{2045, 72},
{2049, 48},
{2050, 40},
{2052, 32},
{2054, 24},
{2056, 16},
{2058, 8},
{2063, 0},
}
_pcsp__i64toa = [][2]uint32{
{14, 0},
{34, 8},
{36, 0},
}
_pcsp__u64toa = [][2]uint32{
{1, 0},
{161, 8},
{162, 0},
{457, 8},
{458, 0},
{758, 8},
{759, 0},
{1225, 8},
{1227, 0},
}
_pcsp__lspace = [][2]uint32{
{1, 0},
{184, 8},
{188, 0},
{204, 8},
{208, 0},
{215, 8},
{220, 0},
}
_pcsp__quote = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{2687, 56},
{2691, 48},
{2692, 40},
{2694, 32},
{2696, 24},
{2698, 16},
{2700, 8},
{2704, 0},
{2731, 56},
}
_pcsp__skip_array = [][2]uint32{
{1, 0},
{28, 8},
{34, 0},
}
_pcsp__skip_number = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{100, 40},
{101, 32},
{103, 24},
{105, 16},
{107, 8},
{108, 0},
{139, 40},
}
_pcsp__skip_object = [][2]uint32{
{1, 0},
{28, 8},
{34, 0},
}
_pcsp__skip_one = [][2]uint32{
{1, 0},
{30, 8},
{36, 0},
}
_pcsp__unquote = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{79, 72},
{83, 48},
{84, 40},
{86, 32},
{88, 24},
{90, 16},
{92, 8},
{96, 0},
{2464, 72},
}
_pcsp__validate_one = [][2]uint32{
{1, 0},
{35, 8},
{41, 0},
}
_pcsp__validate_utf8 = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{11, 40},
{623, 48},
{627, 40},
{628, 32},
{630, 24},
{632, 16},
{634, 8},
{635, 0},
{666, 48},
}
_pcsp__validate_utf8_fast = [][2]uint32{
{1, 0},
{4, 8},
{5, 16},
{1738, 176},
{1739, 168},
{1743, 160},
{2018, 176},
{2019, 168},
{2023, 160},
{2600, 176},
}
_pcsp__value = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{988, 88},
{992, 48},
{993, 40},
{995, 32},
{997, 24},
{999, 16},
{1001, 8},
{1004, 0},
}
_pcsp__vnumber = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{803, 104},
{807, 48},
{808, 40},
{810, 32},
{812, 24},
{814, 16},
{816, 8},
{817, 0},
{1547, 104},
}
_pcsp__atof_eisel_lemire64 = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{292, 32},
{293, 24},
{295, 16},
{297, 8},
{298, 0},
{362, 32},
}
_pcsp__atof_native = [][2]uint32{
{1, 0},
{4, 8},
{587, 56},
{591, 8},
{593, 0},
}
_pcsp__decimal_to_f64 = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{1673, 56},
{1677, 48},
{1678, 40},
{1680, 32},
{1682, 24},
{1684, 16},
{1686, 8},
{1690, 0},
{1702, 56},
}
_pcsp__right_shift = [][2]uint32{
{1, 0},
{318, 8},
{319, 0},
{387, 8},
{388, 0},
{396, 8},
{398, 0},
}
_pcsp__left_shift = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{363, 24},
{364, 16},
{366, 8},
{367, 0},
{470, 24},
{471, 16},
{473, 8},
{474, 0},
{486, 24},
}
_pcsp__vsigned = [][2]uint32{
{1, 0},
{4, 8},
{112, 16},
{113, 8},
{114, 0},
{125, 16},
{126, 8},
{127, 0},
{260, 16},
{261, 8},
{262, 0},
{266, 16},
{267, 8},
{268, 0},
{306, 16},
{307, 8},
{308, 0},
{316, 16},
{317, 8},
{319, 0},
}
_pcsp__vstring = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{11, 40},
{105, 56},
{109, 40},
{110, 32},
{112, 24},
{114, 16},
{116, 8},
{118, 0},
}
_pcsp__vunsigned = [][2]uint32{
{1, 0},
{71, 8},
{72, 0},
{83, 8},
{84, 0},
{107, 8},
{108, 0},
{273, 8},
{274, 0},
{312, 8},
{313, 0},
{320, 8},
{322, 0},
}
)
var Funcs = []loader.CFunc{
{"__native_entry__", 0, 67, 0, nil},
{"_f32toa", _entry__f32toa, _size__f32toa, _stack__f32toa, _pcsp__f32toa},
{"_f64toa", _entry__f64toa, _size__f64toa, _stack__f64toa, _pcsp__f64toa},
{"_format_significand", _entry__format_significand, _size__format_significand, _stack__format_significand, _pcsp__format_significand},
{"_format_integer", _entry__format_integer, _size__format_integer, _stack__format_integer, _pcsp__format_integer},
{"_fsm_exec", _entry__fsm_exec, _size__fsm_exec, _stack__fsm_exec, _pcsp__fsm_exec},
{"_advance_ns", _entry__advance_ns, _size__advance_ns, _stack__advance_ns, _pcsp__advance_ns},
{"_advance_string", _entry__advance_string, _size__advance_string, _stack__advance_string, _pcsp__advance_string},
{"_advance_string_default", _entry__advance_string_default, _size__advance_string_default, _stack__advance_string_default, _pcsp__advance_string_default},
{"_do_skip_number", _entry__do_skip_number, _size__do_skip_number, _stack__do_skip_number, _pcsp__do_skip_number},
{"_get_by_path", _entry__get_by_path, _size__get_by_path, _stack__get_by_path, _pcsp__get_by_path},
{"_skip_one_fast", _entry__skip_one_fast, _size__skip_one_fast, _stack__skip_one_fast, _pcsp__skip_one_fast},
{"_html_escape", _entry__html_escape, _size__html_escape, _stack__html_escape, _pcsp__html_escape},
{"_i64toa", _entry__i64toa, _size__i64toa, _stack__i64toa, _pcsp__i64toa},
{"_u64toa", _entry__u64toa, _size__u64toa, _stack__u64toa, _pcsp__u64toa},
{"_lspace", _entry__lspace, _size__lspace, _stack__lspace, _pcsp__lspace},
{"_quote", _entry__quote, _size__quote, _stack__quote, _pcsp__quote},
{"_skip_array", _entry__skip_array, _size__skip_array, _stack__skip_array, _pcsp__skip_array},
{"_skip_number", _entry__skip_number, _size__skip_number, _stack__skip_number, _pcsp__skip_number},
{"_skip_object", _entry__skip_object, _size__skip_object, _stack__skip_object, _pcsp__skip_object},
{"_skip_one", _entry__skip_one, _size__skip_one, _stack__skip_one, _pcsp__skip_one},
{"_unquote", _entry__unquote, _size__unquote, _stack__unquote, _pcsp__unquote},
{"_validate_one", _entry__validate_one, _size__validate_one, _stack__validate_one, _pcsp__validate_one},
{"_validate_utf8", _entry__validate_utf8, _size__validate_utf8, _stack__validate_utf8, _pcsp__validate_utf8},
{"_validate_utf8_fast", _entry__validate_utf8_fast, _size__validate_utf8_fast, _stack__validate_utf8_fast, _pcsp__validate_utf8_fast},
{"_value", _entry__value, _size__value, _stack__value, _pcsp__value},
{"_vnumber", _entry__vnumber, _size__vnumber, _stack__vnumber, _pcsp__vnumber},
{"_atof_eisel_lemire64", _entry__atof_eisel_lemire64, _size__atof_eisel_lemire64, _stack__atof_eisel_lemire64, _pcsp__atof_eisel_lemire64},
{"_atof_native", _entry__atof_native, _size__atof_native, _stack__atof_native, _pcsp__atof_native},
{"_decimal_to_f64", _entry__decimal_to_f64, _size__decimal_to_f64, _stack__decimal_to_f64, _pcsp__decimal_to_f64},
{"_right_shift", _entry__right_shift, _size__right_shift, _stack__right_shift, _pcsp__right_shift},
{"_left_shift", _entry__left_shift, _size__left_shift, _stack__left_shift, _pcsp__left_shift},
{"_vsigned", _entry__vsigned, _size__vsigned, _stack__vsigned, _pcsp__vsigned},
{"_vstring", _entry__vstring, _size__vstring, _stack__vstring, _pcsp__vstring},
{"_vunsigned", _entry__vunsigned, _size__vunsigned, _stack__vunsigned, _pcsp__vunsigned},
}

173
vendor/github.com/bytedance/sonic/internal/native/dispatch_amd64.go generated vendored
View File

@@ -20,18 +20,13 @@ import (
`unsafe` `unsafe`
`github.com/bytedance/sonic/internal/cpu` `github.com/bytedance/sonic/internal/cpu`
`github.com/bytedance/sonic/internal/native/avx`
`github.com/bytedance/sonic/internal/native/avx2` `github.com/bytedance/sonic/internal/native/avx2`
`github.com/bytedance/sonic/internal/native/sse` `github.com/bytedance/sonic/internal/native/sse`
`github.com/bytedance/sonic/internal/native/types` `github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/internal/rt` `github.com/bytedance/sonic/internal/rt`
`github.com/bytedance/sonic/loader`
) )
const ( const MaxFrameSize uintptr = 400
MaxFrameSize uintptr = 400
BufPaddingSize int = 64
)
var ( var (
S_f64toa uintptr S_f64toa uintptr
@@ -42,9 +37,8 @@ var (
) )
var ( var (
S_quote uintptr S_quote uintptr
S_unquote uintptr S_unquote uintptr
S_html_escape uintptr
) )
var ( var (
@@ -56,18 +50,12 @@ var (
) )
var ( var (
S_skip_one uintptr S_skip_one uintptr
S_skip_one_fast uintptr S_skip_one_fast uintptr
S_get_by_path uintptr S_get_by_path uintptr
S_skip_array uintptr S_skip_array uintptr
S_skip_object uintptr S_skip_object uintptr
S_skip_number uintptr S_skip_number uintptr
)
var (
S_validate_one uintptr
S_validate_utf8 uintptr
S_validate_utf8_fast uintptr
) )
var ( var (
@@ -85,7 +73,7 @@ var (
__GetByPath func(s unsafe.Pointer, p unsafe.Pointer, path unsafe.Pointer, m unsafe.Pointer) int __GetByPath func(s unsafe.Pointer, p unsafe.Pointer, path unsafe.Pointer, m unsafe.Pointer) int
__ValidateOne func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer) int __ValidateOne func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer, flags uint64) int
__I64toa func(out unsafe.Pointer, val int64) (ret int) __I64toa func(out unsafe.Pointer, val int64) (ret int)
@@ -93,9 +81,15 @@ var (
__F64toa func(out unsafe.Pointer, val float64) (ret int) __F64toa func(out unsafe.Pointer, val float64) (ret int)
__F32toa func(out unsafe.Pointer, val float32) (ret int)
__ValidateUTF8 func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer) (ret int) __ValidateUTF8 func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer) (ret int)
__ValidateUTF8Fast func(s unsafe.Pointer) (ret int) __ValidateUTF8Fast func(s unsafe.Pointer) (ret int)
__ParseWithPadding func(parser unsafe.Pointer) (ret int)
__LookupSmallKey func(key unsafe.Pointer, table unsafe.Pointer, lowerOff int) (index int)
) )
//go:nosplit //go:nosplit
@@ -134,8 +128,8 @@ func GetByPath(s *string, p *int, path *[]interface{}, m *types.StateMachine) in
} }
//go:nosplit //go:nosplit
func ValidateOne(s *string, p *int, m *types.StateMachine) int { func ValidateOne(s *string, p *int, m *types.StateMachine, flags uint64) int {
return __ValidateOne(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(m))) return __ValidateOne(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(m)), flags)
} }
//go:nosplit //go:nosplit
@@ -153,6 +147,11 @@ func F64toa(out *byte, val float64) (ret int) {
return __F64toa(rt.NoEscape(unsafe.Pointer(out)), val) return __F64toa(rt.NoEscape(unsafe.Pointer(out)), val)
} }
//go:nosplit
func F32toa(out *byte, val float32) (ret int) {
return __F32toa(rt.NoEscape(unsafe.Pointer(out)), val)
}
//go:nosplit //go:nosplit
func ValidateUTF8(s *string, p *int, m *types.StateMachine) (ret int) { func ValidateUTF8(s *string, p *int, m *types.StateMachine) (ret int) {
return __ValidateUTF8(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(m))) return __ValidateUTF8(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(m)))
@@ -163,51 +162,97 @@ func ValidateUTF8Fast(s *string) (ret int) {
return __ValidateUTF8Fast(rt.NoEscape(unsafe.Pointer(s))) return __ValidateUTF8Fast(rt.NoEscape(unsafe.Pointer(s)))
} }
var stubs = []loader.GoC{ //go:nosplit
{"_f64toa", &S_f64toa, &__F64toa}, func ParseWithPadding(parser unsafe.Pointer) (ret int) {
{"_f32toa", &S_f32toa, nil}, return __ParseWithPadding(rt.NoEscape(unsafe.Pointer(parser)))
{"_i64toa", &S_i64toa, &__I64toa},
{"_u64toa", &S_u64toa, &__U64toa},
{"_lspace", &S_lspace, nil},
{"_quote", &S_quote, &__Quote},
{"_unquote", &S_unquote, &__Unquote},
{"_html_escape", &S_html_escape, &__HTMLEscape},
{"_value", &S_value, &__Value},
{"_vstring", &S_vstring, nil},
{"_vnumber", &S_vnumber, nil},
{"_vsigned", &S_vsigned, nil},
{"_vunsigned", &S_vunsigned, nil},
{"_skip_one", &S_skip_one, &__SkipOne},
{"_skip_one_fast", &S_skip_one_fast, &__SkipOneFast},
{"_get_by_path", &S_get_by_path, &__GetByPath},
{"_skip_array", &S_skip_array, nil},
{"_skip_object", &S_skip_object, nil},
{"_skip_number", &S_skip_number, nil},
{"_validate_one", &S_validate_one, &__ValidateOne},
{"_validate_utf8", &S_validate_utf8, &__ValidateUTF8},
{"_validate_utf8_fast", &S_validate_utf8_fast, &__ValidateUTF8Fast},
} }
func useAVX() { //go:nosplit
loader.WrapGoC(avx.Text__native_entry__, avx.Funcs, stubs, "avx", "avx/native.c") func LookupSmallKey(key *string, table *[]byte, lowerOff int) (index int) {
} return __LookupSmallKey(rt.NoEscape(unsafe.Pointer(key)), rt.NoEscape(unsafe.Pointer(table)), lowerOff)
func useAVX2() {
loader.WrapGoC(avx2.Text__native_entry__, avx2.Funcs, stubs, "avx2", "avx2/native.c")
} }
func useSSE() { func useSSE() {
loader.WrapGoC(sse.Text__native_entry__, sse.Funcs, stubs, "sse", "sse/native.c") sse.Use()
S_f64toa = sse.S_f64toa
__F64toa = sse.F_f64toa
S_f32toa = sse.S_f32toa
__F32toa = sse.F_f32toa
S_i64toa = sse.S_i64toa
__I64toa = sse.F_i64toa
S_u64toa = sse.S_u64toa
__U64toa = sse.F_u64toa
S_lspace = sse.S_lspace
S_quote = sse.S_quote
__Quote = sse.F_quote
S_unquote = sse.S_unquote
__Unquote = sse.F_unquote
S_value = sse.S_value
__Value = sse.F_value
S_vstring = sse.S_vstring
S_vnumber = sse.S_vnumber
S_vsigned = sse.S_vsigned
S_vunsigned = sse.S_vunsigned
S_skip_one = sse.S_skip_one
__SkipOne = sse.F_skip_one
__SkipOneFast = sse.F_skip_one_fast
S_skip_array = sse.S_skip_array
S_skip_object = sse.S_skip_object
S_skip_number = sse.S_skip_number
S_get_by_path = sse.S_get_by_path
__GetByPath = sse.F_get_by_path
__HTMLEscape = sse.F_html_escape
__ValidateOne = sse.F_validate_one
__ValidateUTF8= sse.F_validate_utf8
__ValidateUTF8Fast = sse.F_validate_utf8_fast
__ParseWithPadding = sse.F_parse_with_padding
__LookupSmallKey = sse.F_lookup_small_key
} }
func init() { func useAVX2() {
if cpu.HasAVX2 { avx2.Use()
useAVX2() S_f64toa = avx2.S_f64toa
} else if cpu.HasAVX { __F64toa = avx2.F_f64toa
useAVX() S_f32toa = avx2.S_f32toa
} else if cpu.HasSSE { __F32toa = avx2.F_f32toa
useSSE() S_i64toa = avx2.S_i64toa
} else { __I64toa = avx2.F_i64toa
panic("Unsupported CPU, maybe it's too old to run Sonic.") S_u64toa = avx2.S_u64toa
} __U64toa = avx2.F_u64toa
S_lspace = avx2.S_lspace
S_quote = avx2.S_quote
__Quote = avx2.F_quote
S_unquote = avx2.S_unquote
__Unquote = avx2.F_unquote
S_value = avx2.S_value
__Value = avx2.F_value
S_vstring = avx2.S_vstring
S_vnumber = avx2.S_vnumber
S_vsigned = avx2.S_vsigned
S_vunsigned = avx2.S_vunsigned
S_skip_one = avx2.S_skip_one
__SkipOne = avx2.F_skip_one
__SkipOneFast = avx2.F_skip_one_fast
S_skip_array = avx2.S_skip_array
S_skip_object = avx2.S_skip_object
S_skip_number = avx2.S_skip_number
S_get_by_path = avx2.S_get_by_path
__GetByPath = avx2.F_get_by_path
__HTMLEscape = avx2.F_html_escape
__ValidateOne = avx2.F_validate_one
__ValidateUTF8= avx2.F_validate_utf8
__ValidateUTF8Fast = avx2.F_validate_utf8_fast
__ParseWithPadding = avx2.F_parse_with_padding
__LookupSmallKey = avx2.F_lookup_small_key
}
func init() {
if cpu.HasAVX2 {
useAVX2()
} else if cpu.HasSSE {
useSSE()
} else {
panic("Unsupported CPU, lacks of AVX2 or SSE CPUID Flag. maybe it's too old to run Sonic.")
}
} }

140
vendor/github.com/bytedance/sonic/internal/native/fastfloat_amd64_test.tmpl generated vendored
View File

@@ -1,140 +0,0 @@
// Code generated by Makefile, DO NOT EDIT.
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package {{PACKAGE}}
import (
`encoding/json`
`math`
`math/rand`
`strconv`
`testing`
`github.com/stretchr/testify/assert`
)
func TestFastFloat_Encode(t *testing.T) {
var buf [64]byte
assert.Equal(t, "0" , string(buf[:f64toa(&buf[0], 0)]))
assert.Equal(t, "-0" , string(buf[:f64toa(&buf[0], math.Float64frombits(0x8000000000000000))]))
assert.Equal(t, "12340000000" , string(buf[:f64toa(&buf[0], 1234e7)]))
assert.Equal(t, "12.34" , string(buf[:f64toa(&buf[0], 1234e-2)]))
assert.Equal(t, "0.001234" , string(buf[:f64toa(&buf[0], 1234e-6)]))
assert.Equal(t, "1e+30" , string(buf[:f64toa(&buf[0], 1e30)]))
assert.Equal(t, "1.234e+33" , string(buf[:f64toa(&buf[0], 1234e30)]))
assert.Equal(t, "1.234e+308" , string(buf[:f64toa(&buf[0], 1234e305)]))
assert.Equal(t, "1.234e-317" , string(buf[:f64toa(&buf[0], 1234e-320)]))
assert.Equal(t, "1.7976931348623157e+308" , string(buf[:f64toa(&buf[0], 1.7976931348623157e308)]))
assert.Equal(t, "-12340000000" , string(buf[:f64toa(&buf[0], -1234e7)]))
assert.Equal(t, "-12.34" , string(buf[:f64toa(&buf[0], -1234e-2)]))
assert.Equal(t, "-0.001234" , string(buf[:f64toa(&buf[0], -1234e-6)]))
assert.Equal(t, "-1e+30" , string(buf[:f64toa(&buf[0], -1e30)]))
assert.Equal(t, "-1.234e+33" , string(buf[:f64toa(&buf[0], -1234e30)]))
assert.Equal(t, "-1.234e+308" , string(buf[:f64toa(&buf[0], -1234e305)]))
assert.Equal(t, "-1.234e-317" , string(buf[:f64toa(&buf[0], -1234e-320)]))
assert.Equal(t, "-2.2250738585072014e-308" , string(buf[:f64toa(&buf[0], -2.2250738585072014e-308)]))
}
func TestFastFloat_Random(t *testing.T) {
var buf [64]byte
N := 10000
for i := 0; i < N; i++ {
b64 := uint64(rand.Uint32())<<32 | uint64(rand.Uint32())
f64 := math.Float64frombits(b64)
jout, jerr := json.Marshal(f64)
n := f64toa(&buf[0], f64)
if jerr == nil {
assert.Equal(t, jout, buf[:n])
} else {
assert.True(t, n == 0)
}
f32 := math.Float32frombits(rand.Uint32())
jout, jerr = json.Marshal(f32)
n = f32toa(&buf[0], f32)
if jerr == nil {
assert.Equal(t, jout, buf[:n])
} else {
assert.True(t, n == 0)
}
}
}
func BenchmarkParseFloat64(b *testing.B) {
var f64toaBenches = []struct {
name string
float float64
}{
{"Zero", 0},
{"Decimal", 33909},
{"Float", 339.7784},
{"Exp", -5.09e75},
{"NegExp", -5.11e-95},
{"LongExp", 1.234567890123456e-78},
{"Big", 123456789123456789123456789},
}
for _, c := range f64toaBenches {
f64bench := []struct {
name string
test func(*testing.B)
}{{
name: "StdLib",
test: func(b *testing.B) { var buf [64]byte; for i := 0; i < b.N; i++ { strconv.AppendFloat(buf[:0], c.float, 'g', -1, 64) }},
}, {
name: "FastFloat",
test: func(b *testing.B) { var buf [64]byte; for i := 0; i < b.N; i++ { f64toa(&buf[0], c.float) }},
}}
for _, bm := range f64bench {
name := bm.name + "_" + c.name
b.Run(name, bm.test)
}
}
}
func BenchmarkParseFloat32(b *testing.B) {
var f32toaBenches = []struct {
name string
float float32
}{
{"Zero", 0},
{"Integer", 33909},
{"ExactFraction", 3.375},
{"Point", 339.7784},
{"Exp", -5.09e25},
{"NegExp", -5.11e-25},
{"Shortest", 1.234567e-8},
}
for _, c := range f32toaBenches {
bench := []struct {
name string
test func(*testing.B)
}{{
name: "StdLib32",
test: func(b *testing.B) { var buf [64]byte; for i := 0; i < b.N; i++ { strconv.AppendFloat(buf[:0], float64(c.float), 'g', -1, 32) }},
}, {
name: "FastFloat32",
test: func(b *testing.B) { var buf [64]byte; for i := 0; i < b.N; i++ { f32toa(&buf[0], c.float) }},
}}
for _, bm := range bench {
name := bm.name + "_" + c.name
b.Run(name, bm.test)
}
}
}

153
vendor/github.com/bytedance/sonic/internal/native/fastint_amd64_test.tmpl generated vendored
View File

@@ -1,153 +0,0 @@
// Code generated by Makefile, DO NOT EDIT.
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package {{PACKAGE}}
import (
`strconv`
`testing`
`fmt`
`github.com/stretchr/testify/assert`
)
func TestFastInt_IntToString(t *testing.T) {
var buf [32]byte
assert.Equal(t, "0" , string(buf[:i64toa(&buf[0], 0)]))
assert.Equal(t, "1" , string(buf[:i64toa(&buf[0], 1)]))
assert.Equal(t, "12" , string(buf[:i64toa(&buf[0], 12)]))
assert.Equal(t, "123" , string(buf[:i64toa(&buf[0], 123)]))
assert.Equal(t, "1234" , string(buf[:i64toa(&buf[0], 1234)]))
assert.Equal(t, "12345" , string(buf[:i64toa(&buf[0], 12345)]))
assert.Equal(t, "123456" , string(buf[:i64toa(&buf[0], 123456)]))
assert.Equal(t, "1234567" , string(buf[:i64toa(&buf[0], 1234567)]))
assert.Equal(t, "12345678" , string(buf[:i64toa(&buf[0], 12345678)]))
assert.Equal(t, "123456789" , string(buf[:i64toa(&buf[0], 123456789)]))
assert.Equal(t, "1234567890" , string(buf[:i64toa(&buf[0], 1234567890)]))
assert.Equal(t, "12345678901" , string(buf[:i64toa(&buf[0], 12345678901)]))
assert.Equal(t, "123456789012" , string(buf[:i64toa(&buf[0], 123456789012)]))
assert.Equal(t, "1234567890123" , string(buf[:i64toa(&buf[0], 1234567890123)]))
assert.Equal(t, "12345678901234" , string(buf[:i64toa(&buf[0], 12345678901234)]))
assert.Equal(t, "123456789012345" , string(buf[:i64toa(&buf[0], 123456789012345)]))
assert.Equal(t, "1234567890123456" , string(buf[:i64toa(&buf[0], 1234567890123456)]))
assert.Equal(t, "12345678901234567" , string(buf[:i64toa(&buf[0], 12345678901234567)]))
assert.Equal(t, "123456789012345678" , string(buf[:i64toa(&buf[0], 123456789012345678)]))
assert.Equal(t, "1234567890123456789" , string(buf[:i64toa(&buf[0], 1234567890123456789)]))
assert.Equal(t, "9223372036854775807" , string(buf[:i64toa(&buf[0], 9223372036854775807)]))
assert.Equal(t, "-1" , string(buf[:i64toa(&buf[0], -1)]))
assert.Equal(t, "-12" , string(buf[:i64toa(&buf[0], -12)]))
assert.Equal(t, "-123" , string(buf[:i64toa(&buf[0], -123)]))
assert.Equal(t, "-1234" , string(buf[:i64toa(&buf[0], -1234)]))
assert.Equal(t, "-12345" , string(buf[:i64toa(&buf[0], -12345)]))
assert.Equal(t, "-123456" , string(buf[:i64toa(&buf[0], -123456)]))
assert.Equal(t, "-1234567" , string(buf[:i64toa(&buf[0], -1234567)]))
assert.Equal(t, "-12345678" , string(buf[:i64toa(&buf[0], -12345678)]))
assert.Equal(t, "-123456789" , string(buf[:i64toa(&buf[0], -123456789)]))
assert.Equal(t, "-1234567890" , string(buf[:i64toa(&buf[0], -1234567890)]))
assert.Equal(t, "-12345678901" , string(buf[:i64toa(&buf[0], -12345678901)]))
assert.Equal(t, "-123456789012" , string(buf[:i64toa(&buf[0], -123456789012)]))
assert.Equal(t, "-1234567890123" , string(buf[:i64toa(&buf[0], -1234567890123)]))
assert.Equal(t, "-12345678901234" , string(buf[:i64toa(&buf[0], -12345678901234)]))
assert.Equal(t, "-123456789012345" , string(buf[:i64toa(&buf[0], -123456789012345)]))
assert.Equal(t, "-1234567890123456" , string(buf[:i64toa(&buf[0], -1234567890123456)]))
assert.Equal(t, "-12345678901234567" , string(buf[:i64toa(&buf[0], -12345678901234567)]))
assert.Equal(t, "-123456789012345678" , string(buf[:i64toa(&buf[0], -123456789012345678)]))
assert.Equal(t, "-1234567890123456789" , string(buf[:i64toa(&buf[0], -1234567890123456789)]))
assert.Equal(t, "-9223372036854775808" , string(buf[:i64toa(&buf[0], -9223372036854775808)]))
}
func TestFastInt_UintToString(t *testing.T) {
var buf [32]byte
assert.Equal(t, "0" , string(buf[:u64toa(&buf[0], 0)]))
assert.Equal(t, "1" , string(buf[:u64toa(&buf[0], 1)]))
assert.Equal(t, "12" , string(buf[:u64toa(&buf[0], 12)]))
assert.Equal(t, "123" , string(buf[:u64toa(&buf[0], 123)]))
assert.Equal(t, "1234" , string(buf[:u64toa(&buf[0], 1234)]))
assert.Equal(t, "12345" , string(buf[:u64toa(&buf[0], 12345)]))
assert.Equal(t, "123456" , string(buf[:u64toa(&buf[0], 123456)]))
assert.Equal(t, "1234567" , string(buf[:u64toa(&buf[0], 1234567)]))
assert.Equal(t, "12345678" , string(buf[:u64toa(&buf[0], 12345678)]))
assert.Equal(t, "123456789" , string(buf[:u64toa(&buf[0], 123456789)]))
assert.Equal(t, "1234567890" , string(buf[:u64toa(&buf[0], 1234567890)]))
assert.Equal(t, "12345678901" , string(buf[:u64toa(&buf[0], 12345678901)]))
assert.Equal(t, "123456789012" , string(buf[:u64toa(&buf[0], 123456789012)]))
assert.Equal(t, "1234567890123" , string(buf[:u64toa(&buf[0], 1234567890123)]))
assert.Equal(t, "12345678901234" , string(buf[:u64toa(&buf[0], 12345678901234)]))
assert.Equal(t, "123456789012345" , string(buf[:u64toa(&buf[0], 123456789012345)]))
assert.Equal(t, "1234567890123456" , string(buf[:u64toa(&buf[0], 1234567890123456)]))
assert.Equal(t, "12345678901234567" , string(buf[:u64toa(&buf[0], 12345678901234567)]))
assert.Equal(t, "123456789012345678" , string(buf[:u64toa(&buf[0], 123456789012345678)]))
assert.Equal(t, "1234567890123456789" , string(buf[:u64toa(&buf[0], 1234567890123456789)]))
assert.Equal(t, "12345678901234567890" , string(buf[:u64toa(&buf[0], 12345678901234567890)]))
assert.Equal(t, "18446744073709551615" , string(buf[:u64toa(&buf[0], 18446744073709551615)]))
}
func BenchmarkFastInt_IntToString(b *testing.B) {
benchmarks := []struct {
name string
test func(*testing.B)
}{{
name: "StdLib-Positive",
test: func(b *testing.B) { var buf [32]byte; for i := 0; i < b.N; i++ { strconv.AppendInt(buf[:0], int64(i), 10) }},
}, {
name: "StdLib-Negative",
test: func(b *testing.B) { var buf [32]byte; for i := 0; i < b.N; i++ { strconv.AppendInt(buf[:0], -int64(i), 10) }},
}, {
name: "FastInt-Positive",
test: func(b *testing.B) { var buf [32]byte; for i := 0; i < b.N; i++ { i64toa(&buf[0], int64(i)) }},
}, {
name: "FastInt-Negative",
test: func(b *testing.B) { var buf [32]byte; for i := 0; i < b.N; i++ { i64toa(&buf[0], -int64(i)) }},
}}
for _, bm := range benchmarks {
b.Run(bm.name, bm.test)
}
}
type utoaBench struct {
name string
num uint64
}
func BenchmarkFastInt_UintToString(b *testing.B) {
maxUint := "18446744073709551615"
benchs := make([]utoaBench, len(maxUint) + 1)
benchs[0].name = "Zero"
benchs[0].num = 0
for i := 1; i <= len(maxUint); i++ {
benchs[i].name = strconv.FormatInt(int64(i), 10) + "-Digs"
benchs[i].num, _ = strconv.ParseUint(string(maxUint[:i]), 10, 64)
}
for _, t := range(benchs) {
benchmarks := []struct {
name string
test func(*testing.B)
}{{
name: "StdLib",
test: func(b *testing.B) { var buf [32]byte; for i := 0; i < b.N; i++ { strconv.AppendUint(buf[:0], t.num, 10) }},
}, {
name: "FastInt",
test: func(b *testing.B) { var buf [32]byte; for i := 0; i < b.N; i++ { u64toa(&buf[0], t.num) }},
}}
for _, bm := range benchmarks {
name := fmt.Sprintf("%s_%s", bm.name, t.name)
b.Run(name, bm.test)
}
}
}

189
vendor/github.com/bytedance/sonic/internal/native/native_amd64.tmpl generated vendored
View File

@@ -1,189 +0,0 @@
// Code generated by Makefile, DO NOT EDIT.
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package {{PACKAGE}}
import (
`unsafe`
`github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/internal/rt`
)
var (
__i64toa func(out unsafe.Pointer, val int64) (ret int)
__u64toa func(out unsafe.Pointer, val uint64) (ret int)
__f64toa func(out unsafe.Pointer, val float64) (ret int)
__f32toa func(out unsafe.Pointer, val float32) (ret int)
__lspace func(sp unsafe.Pointer, nb int, off int) (ret int)
__quote func(sp unsafe.Pointer, nb int, dp unsafe.Pointer, dn unsafe.Pointer, flags uint64) (ret int)
__html_escape func(sp unsafe.Pointer, nb int, dp unsafe.Pointer, dn unsafe.Pointer) (ret int)
__unquote func(sp unsafe.Pointer, nb int, dp unsafe.Pointer, ep unsafe.Pointer, flags uint64) (ret int)
__value func(s unsafe.Pointer, n int, p int, v unsafe.Pointer, flags uint64) (ret int)
__vstring func(s unsafe.Pointer, p unsafe.Pointer, v unsafe.Pointer, flags uint64)
__vnumber func(s unsafe.Pointer, p unsafe.Pointer, v unsafe.Pointer)
__vsigned func(s unsafe.Pointer, p unsafe.Pointer, v unsafe.Pointer)
__vunsigned func(s unsafe.Pointer, p unsafe.Pointer, v unsafe.Pointer)
__skip_one func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer, flags uint64) (ret int)
__skip_one_fast func(s unsafe.Pointer, p unsafe.Pointer) (ret int)
__skip_array func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer, flags uint64) (ret int)
__skip_object func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer, flags uint64) (ret int)
__skip_number func(s unsafe.Pointer, p unsafe.Pointer) (ret int)
__validate_one func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer) (ret int)
__get_by_path func(s unsafe.Pointer, p unsafe.Pointer, path unsafe.Pointer, m unsafe.Pointer) (ret int)
__validate_utf8 func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer) (ret int)
__validate_utf8_fast func(s unsafe.Pointer) (ret int)
__fsm_exec func(m unsafe.Pointer, s unsafe.Pointer, p unsafe.Pointer, flags uint64) (ret int)
)
//go:nosplit
func i64toa(out *byte, val int64) (ret int) {
return __i64toa(rt.NoEscape(unsafe.Pointer(out)), val)
}
//go:nosplit
func u64toa(out *byte, val uint64) (ret int) {
return __u64toa(rt.NoEscape(unsafe.Pointer(out)), val)
}
//go:nosplit
func f64toa(out *byte, val float64) (ret int) {
return __f64toa(rt.NoEscape(unsafe.Pointer(out)), val)
}
//go:nosplit
func f32toa(out *byte, val float32) (ret int) {
return __f32toa(rt.NoEscape(unsafe.Pointer(out)), val)
}
//go:nosplit
func lspace(sp unsafe.Pointer, nb int, off int) (ret int) {
return __lspace(rt.NoEscape(sp), nb, off)
}
//go:nosplit
func quote(sp unsafe.Pointer, nb int, dp unsafe.Pointer, dn *int, flags uint64) (ret int) {
return __quote(rt.NoEscape(unsafe.Pointer(sp)), nb, rt.NoEscape(unsafe.Pointer(dp)), rt.NoEscape(unsafe.Pointer(dn)), flags)
}
//go:nosplit
func html_escape(sp unsafe.Pointer, nb int, dp unsafe.Pointer, dn *int) (ret int) {
return __html_escape(rt.NoEscape(unsafe.Pointer(sp)), nb, rt.NoEscape(unsafe.Pointer(dp)), rt.NoEscape(unsafe.Pointer(dn)))
}
//go:nosplit
func unquote(sp unsafe.Pointer, nb int, dp unsafe.Pointer, ep *int, flags uint64) (ret int) {
return __unquote(rt.NoEscape(unsafe.Pointer(sp)), nb, rt.NoEscape(unsafe.Pointer(dp)), rt.NoEscape(unsafe.Pointer(ep)), flags)
}
//go:nosplit
func value(s unsafe.Pointer, n int, p int, v *types.JsonState, flags uint64) (ret int) {
return __value(rt.NoEscape(unsafe.Pointer(s)), n, p, rt.NoEscape(unsafe.Pointer(v)), flags)
}
//go:nosplit
func vstring(s *string, p *int, v *types.JsonState, flags uint64) {
__vstring(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(v)), flags)
}
//go:nosplit
func vnumber(s *string, p *int, v *types.JsonState) {
__vnumber(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(v)))
}
//go:nosplit
func vsigned(s *string, p *int, v *types.JsonState) {
__vsigned(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(v)))
}
//go:nosplit
func vunsigned(s *string, p *int, v *types.JsonState) {
__vunsigned(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(v)))
}
//go:nosplit
func skip_one(s *string, p *int, m *types.StateMachine, flags uint64) (ret int) {
return __skip_one(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(m)), flags)
}
//go:nosplit
func skip_one_fast(s *string, p *int) (ret int) {
return __skip_one_fast(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)))
}
//go:nosplit
func skip_array(s *string, p *int, m *types.StateMachine, flags uint64) (ret int) {
return __skip_array(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(m)), flags)
}
//go:nosplit
func skip_object(s *string, p *int, m *types.StateMachine, flags uint64) (ret int) {
return __skip_object(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(m)), flags)
}
//go:nosplit
func skip_number(s *string, p *int) (ret int) {
return __skip_number(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)))
}
//go:nosplit
func validate_one(s *string, p *int, m *types.StateMachine) (ret int) {
return __validate_one(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(m)))
}
//go:nosplit
func get_by_path(s *string, p *int, path *[]interface{}, m *types.StateMachine) (ret int) {
return __get_by_path(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(path)), rt.NoEscape(unsafe.Pointer(m)))
}
//go:nosplit
func validate_utf8(s *string, p *int, m *types.StateMachine) (ret int) {
return __validate_utf8(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(m)))
}
//go:nosplit
func validate_utf8_fast(s *string) (ret int) {
return __validate_utf8_fast(rt.NoEscape(unsafe.Pointer(s)))
}
//go:nosplit
func fsm_exec(m *types.StateMachine, s *string, p *int, flags uint64) (ret int) {
return __fsm_exec(rt.NoEscape(unsafe.Pointer(m)), rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), flags)
}

625
vendor/github.com/bytedance/sonic/internal/native/native_amd64_test.tmpl generated vendored
View File

@@ -1,625 +0,0 @@
// Code generated by Makefile, DO NOT EDIT.
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package {{PACKAGE}}
import (
`encoding/hex`
`fmt`
`math`
`strings`
`testing`
`unsafe`
`github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/internal/rt`
`github.com/davecgh/go-spew/spew`
`github.com/stretchr/testify/assert`
`github.com/stretchr/testify/require`
)
func TestNative_Value(t *testing.T) {
var v types.JsonState
s := ` -12345`
p := (*rt.GoString)(unsafe.Pointer(&s))
x := value(p.Ptr, p.Len, 0, &v, 0)
assert.Equal(t, 9, x)
assert.Equal(t, types.V_INTEGER, v.Vt)
assert.Equal(t, int64(-12345), v.Iv)
assert.Equal(t, 3, v.Ep)
}
func TestNative_Value_OutOfBound(t *testing.T) {
var v types.JsonState
mem := []byte{'"', '"'}
s := rt.Mem2Str(mem[:1])
p := (*rt.GoString)(unsafe.Pointer(&s))
x := value(p.Ptr, p.Len, 0, &v, 0)
assert.Equal(t, 1, x)
assert.Equal(t, -int(types.ERR_EOF), int(v.Vt))
}
func TestNative_Quote(t *testing.T) {
s := "hello\b\f\n\r\t\\\"\u666fworld"
d := make([]byte, 256)
dp := (*rt.GoSlice)(unsafe.Pointer(&d))
sp := (*rt.GoString)(unsafe.Pointer(&s))
rv := quote(sp.Ptr, sp.Len, dp.Ptr, &dp.Len, 0)
if rv < 0 {
require.NoError(t, types.ParsingError(-rv))
}
assert.Equal(t, len(s), rv)
assert.Equal(t, 35, len(d))
assert.Equal(t, `hello\u0008\u000c\n\r\t\\\"景world`, string(d))
}
func TestNative_QuoteNoMem(t *testing.T) {
s := "hello\b\f\n\r\t\\\"\u666fworld"
d := make([]byte, 10)
dp := (*rt.GoSlice)(unsafe.Pointer(&d))
sp := (*rt.GoString)(unsafe.Pointer(&s))
rv := quote(sp.Ptr, sp.Len, dp.Ptr, &dp.Len, 0)
assert.Equal(t, -6, rv)
assert.Equal(t, 5, len(d))
assert.Equal(t, `hello`, string(d))
}
func TestNative_DoubleQuote(t *testing.T) {
s := "hello\b\f\n\r\t\\\"\u666fworld"
d := make([]byte, 256)
dp := (*rt.GoSlice)(unsafe.Pointer(&d))
sp := (*rt.GoString)(unsafe.Pointer(&s))
rv := quote(sp.Ptr, sp.Len, dp.Ptr, &dp.Len, types.F_DOUBLE_UNQUOTE)
if rv < 0 {
require.NoError(t, types.ParsingError(-rv))
}
assert.Equal(t, len(s), rv)
assert.Equal(t, 44, len(d))
assert.Equal(t, `hello\\u0008\\u000c\\n\\r\\t\\\\\\\"景world`, string(d))
}
func TestNative_Unquote(t *testing.T) {
s := `hello\b\f\n\r\t\\\"\u2333world`
d := make([]byte, 0, len(s))
ep := -1
dp := (*rt.GoSlice)(unsafe.Pointer(&d))
sp := (*rt.GoString)(unsafe.Pointer(&s))
rv := unquote(sp.Ptr, sp.Len, dp.Ptr, &ep, 0)
if rv < 0 {
require.NoError(t, types.ParsingError(-rv))
}
dp.Len = rv
assert.Equal(t, -1, ep)
assert.Equal(t, "hello\b\f\n\r\t\\\"\u2333world", string(d))
}
func TestNative_UnquoteError(t *testing.T) {
s := `asdf\`
d := make([]byte, 0, len(s))
ep := -1
dp := (*rt.GoSlice)(unsafe.Pointer(&d))
sp := (*rt.GoString)(unsafe.Pointer(&s))
rv := unquote(sp.Ptr, sp.Len, dp.Ptr, &ep, 0)
assert.Equal(t, -int(types.ERR_EOF), rv)
assert.Equal(t, 5, ep)
s = `asdf\gqwer`
d = make([]byte, 0, len(s))
ep = -1
dp = (*rt.GoSlice)(unsafe.Pointer(&d))
sp = (*rt.GoString)(unsafe.Pointer(&s))
rv = unquote(sp.Ptr, sp.Len, dp.Ptr, &ep, 0)
assert.Equal(t, -int(types.ERR_INVALID_ESCAPE), rv)
assert.Equal(t, 5, ep)
s = `asdf\u1gggqwer`
d = make([]byte, 0, len(s))
ep = -1
dp = (*rt.GoSlice)(unsafe.Pointer(&d))
sp = (*rt.GoString)(unsafe.Pointer(&s))
rv = unquote(sp.Ptr, sp.Len, dp.Ptr, &ep, 0)
assert.Equal(t, -int(types.ERR_INVALID_CHAR), rv)
assert.Equal(t, 7, ep)
s = `asdf\ud800qwer`
d = make([]byte, 0, len(s))
ep = -1
dp = (*rt.GoSlice)(unsafe.Pointer(&d))
sp = (*rt.GoString)(unsafe.Pointer(&s))
rv = unquote(sp.Ptr, sp.Len, dp.Ptr, &ep, 0)
assert.Equal(t, -int(types.ERR_INVALID_UNICODE), rv)
assert.Equal(t, 6, ep)
s = `asdf\\ud800qwer`
d = make([]byte, 0, len(s))
ep = -1
dp = (*rt.GoSlice)(unsafe.Pointer(&d))
sp = (*rt.GoString)(unsafe.Pointer(&s))
rv = unquote(sp.Ptr, sp.Len, dp.Ptr, &ep, types.F_DOUBLE_UNQUOTE)
assert.Equal(t, -int(types.ERR_INVALID_UNICODE), rv)
assert.Equal(t, 7, ep)
s = `asdf\ud800\ud800qwer`
d = make([]byte, 0, len(s))
ep = -1
dp = (*rt.GoSlice)(unsafe.Pointer(&d))
sp = (*rt.GoString)(unsafe.Pointer(&s))
rv = unquote(sp.Ptr, sp.Len, dp.Ptr, &ep, 0)
assert.Equal(t, -int(types.ERR_INVALID_UNICODE), rv)
assert.Equal(t, 12, ep)
s = `asdf\\ud800\\ud800qwer`
d = make([]byte, 0, len(s))
ep = -1
dp = (*rt.GoSlice)(unsafe.Pointer(&d))
sp = (*rt.GoString)(unsafe.Pointer(&s))
rv = unquote(sp.Ptr, sp.Len, dp.Ptr, &ep, types.F_DOUBLE_UNQUOTE)
assert.Equal(t, -int(types.ERR_INVALID_UNICODE), rv)
assert.Equal(t, 14, ep)
}
func TestNative_DoubleUnquote(t *testing.T) {
s := `hello\\b\\f\\n\\r\\t\\\\\\\"\\u2333world`
d := make([]byte, 0, len(s))
ep := -1
dp := (*rt.GoSlice)(unsafe.Pointer(&d))
sp := (*rt.GoString)(unsafe.Pointer(&s))
rv := unquote(sp.Ptr, sp.Len, dp.Ptr, &ep, types.F_DOUBLE_UNQUOTE)
if rv < 0 {
require.NoError(t, types.ParsingError(-rv))
}
dp.Len = rv
assert.Equal(t, -1, ep)
assert.Equal(t, "hello\b\f\n\r\t\\\"\u2333world", string(d))
}
func TestNative_UnquoteUnicodeReplacement(t *testing.T) {
s := `hello\ud800world`
d := make([]byte, 0, len(s))
ep := -1
dp := (*rt.GoSlice)(unsafe.Pointer(&d))
sp := (*rt.GoString)(unsafe.Pointer(&s))
rv := unquote(sp.Ptr, sp.Len, dp.Ptr, &ep, types.F_UNICODE_REPLACE)
if rv < 0 {
require.NoError(t, types.ParsingError(-rv))
}
dp.Len = rv
assert.Equal(t, -1, ep)
assert.Equal(t, "hello\ufffdworld", string(d))
s = `hello\ud800\ud800world`
d = make([]byte, 0, len(s))
ep = -1
dp = (*rt.GoSlice)(unsafe.Pointer(&d))
sp = (*rt.GoString)(unsafe.Pointer(&s))
rv = unquote(sp.Ptr, sp.Len, dp.Ptr, &ep, types.F_UNICODE_REPLACE)
if rv < 0 {
require.NoError(t, types.ParsingError(-rv))
}
dp.Len = rv
assert.Equal(t, -1, ep)
assert.Equal(t, "hello\ufffd\ufffdworld", string(d))
}
func TestNative_HTMLEscape(t *testing.T) {
s := "hello\u2029\u2028<&>world"
d := make([]byte, 256)
dp := (*rt.GoSlice)(unsafe.Pointer(&d))
sp := (*rt.GoString)(unsafe.Pointer(&s))
rv := html_escape(sp.Ptr, sp.Len, dp.Ptr, &dp.Len)
if rv < 0 {
require.NoError(t, types.ParsingError(-rv))
}
assert.Equal(t, len(s), rv)
assert.Equal(t, 40, len(d))
assert.Equal(t, `hello\u2029\u2028\u003c\u0026\u003eworld`, string(d))
}
func TestNative_HTMLEscapeNoMem(t *testing.T) {
s := "hello\u2029\u2028<&>world"
d := make([]byte, 10)
dp := (*rt.GoSlice)(unsafe.Pointer(&d))
sp := (*rt.GoString)(unsafe.Pointer(&s))
rv := html_escape(sp.Ptr, sp.Len, dp.Ptr, &dp.Len)
assert.Equal(t, -6, rv)
assert.Equal(t, 5, len(d))
assert.Equal(t, `hello`, string(d))
}
func TestNative_Vstring(t *testing.T) {
var v types.JsonState
i := 0
s := `test"test\n2"`
vstring(&s, &i, &v, 0)
assert.Equal(t, 5, i)
assert.Equal(t, -1, v.Ep)
assert.Equal(t, int64(0), v.Iv)
vstring(&s, &i, &v, 0)
assert.Equal(t, 13, i)
assert.Equal(t, 9, v.Ep)
assert.Equal(t, int64(5), v.Iv)
}
func TestNative_Vstring_ValidUnescapedChars(t *testing.T) {
var v types.JsonState
valid := uint64(types.F_VALIDATE_STRING)
i := 0
s := "test\x1f\""
vstring(&s, &i, &v, valid)
assert.Equal(t, -int(types.ERR_INVALID_CHAR), int(v.Vt))
}
func TestNative_VstringEscapeEOF(t *testing.T) {
var v types.JsonState
i := 0
s := `xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx\"xxxxxxxxxxxxxxxxxxxxxxxxxxxxx"x`
vstring(&s, &i, &v, 0)
assert.Equal(t, 95, i)
assert.Equal(t, 63, v.Ep)
assert.Equal(t, int64(0), v.Iv)
}
func TestNative_VstringHangUpOnRandomData(t *testing.T) {
v, e := hex.DecodeString(
"228dc61efd54ef80a908fb6026b7f2d5f92a257ba8b347c995f259eb8685376a" +
"8c4500262d9c308b3f3ec2577689cf345d9f86f9b5d18d3e463bec5c22df2d2e" +
"4506010eba1dae7278",
)
assert.Nil(t, e)
p := 1
s := rt.Mem2Str(v)
var js types.JsonState
vstring(&s, &p, &js, 0)
fmt.Printf("js: %s\n", spew.Sdump(js))
}
func TestNative_Vnumber(t *testing.T) {
var v types.JsonState
i := 0
s := "1234"
vnumber(&s, &i, &v)
assert.Equal(t, 4, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, int64(1234), v.Iv)
assert.Equal(t, types.V_INTEGER, v.Vt)
i = 0
s = "1.234"
vnumber(&s, &i, &v)
assert.Equal(t, 5, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, 1.234, v.Dv)
assert.Equal(t, types.V_DOUBLE, v.Vt)
i = 0
s = "1.234e5"
vnumber(&s, &i, &v)
assert.Equal(t, 7, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, 1.234e5, v.Dv)
assert.Equal(t, types.V_DOUBLE, v.Vt)
i = 0
s = "0.0125"
vnumber(&s, &i, &v)
assert.Equal(t, 6, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, 0.0125, v.Dv)
assert.Equal(t, types.V_DOUBLE, v.Vt)
i = 0
s = "100000000000000000000"
vnumber(&s, &i, &v)
assert.Equal(t, 21, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, 100000000000000000000.0, v.Dv)
assert.Equal(t, types.V_DOUBLE, v.Vt)
i = 0
s = "999999999999999900000"
vnumber(&s, &i, &v)
assert.Equal(t, 21, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, 999999999999999900000.0, v.Dv)
assert.Equal(t, types.V_DOUBLE, v.Vt)
i = 0
s = "-1.234"
vnumber(&s, &i, &v)
assert.Equal(t, 6, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, -1.234, v.Dv)
assert.Equal(t, types.V_DOUBLE, v.Vt)
}
func TestNative_Vsigned(t *testing.T) {
var v types.JsonState
i := 0
s := "1234"
vsigned(&s, &i, &v)
assert.Equal(t, 4, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, int64(1234), v.Iv)
assert.Equal(t, types.V_INTEGER, v.Vt)
i = 0
s = "-1234"
vsigned(&s, &i, &v)
assert.Equal(t, 5, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, int64(-1234), v.Iv)
assert.Equal(t, types.V_INTEGER, v.Vt)
i = 0
s = "9223372036854775807"
vsigned(&s, &i, &v)
assert.Equal(t, 19, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, int64(math.MaxInt64), v.Iv)
assert.Equal(t, types.V_INTEGER, v.Vt)
i = 0
s = "-9223372036854775808"
vsigned(&s, &i, &v)
assert.Equal(t, 20, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, int64(math.MinInt64), v.Iv)
assert.Equal(t, types.V_INTEGER, v.Vt)
i = 0
s = "9223372036854775808"
vsigned(&s, &i, &v)
assert.Equal(t, 18, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, types.ValueType(-int(types.ERR_INTEGER_OVERFLOW)), v.Vt)
i = 0
s = "-9223372036854775809"
vsigned(&s, &i, &v)
assert.Equal(t, 19, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, types.ValueType(-int(types.ERR_INTEGER_OVERFLOW)), v.Vt)
i = 0
s = "1.234"
vsigned(&s, &i, &v)
assert.Equal(t, 1, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, types.ValueType(-int(types.ERR_INVALID_NUMBER_FMT)), v.Vt)
i = 0
s = "0.0125"
vsigned(&s, &i, &v)
assert.Equal(t, 1, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, types.ValueType(-int(types.ERR_INVALID_NUMBER_FMT)), v.Vt)
i = 0
s = "-1234e5"
vsigned(&s, &i, &v)
assert.Equal(t, 5, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, types.ValueType(-int(types.ERR_INVALID_NUMBER_FMT)), v.Vt)
i = 0
s = "-1234e-5"
vsigned(&s, &i, &v)
assert.Equal(t, 5, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, types.ValueType(-int(types.ERR_INVALID_NUMBER_FMT)), v.Vt)
}
func TestNative_Vunsigned(t *testing.T) {
var v types.JsonState
i := 0
s := "1234"
vunsigned(&s, &i, &v)
assert.Equal(t, 4, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, int64(1234), v.Iv)
assert.Equal(t, types.V_INTEGER, v.Vt)
i = 0
s = "18446744073709551615"
vunsigned(&s, &i, &v)
assert.Equal(t, 20, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, ^int64(0), v.Iv)
assert.Equal(t, types.V_INTEGER, v.Vt)
i = 0
s = "18446744073709551616"
vunsigned(&s, &i, &v)
assert.Equal(t, 19, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, types.ValueType(-int(types.ERR_INTEGER_OVERFLOW)), v.Vt)
i = 0
s = "-1234"
vunsigned(&s, &i, &v)
assert.Equal(t, 0, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, types.ValueType(-int(types.ERR_INVALID_NUMBER_FMT)), v.Vt)
i = 0
s = "1.234"
vunsigned(&s, &i, &v)
assert.Equal(t, 1, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, types.ValueType(-int(types.ERR_INVALID_NUMBER_FMT)), v.Vt)
i = 0
s = "0.0125"
vunsigned(&s, &i, &v)
assert.Equal(t, 1, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, types.ValueType(-int(types.ERR_INVALID_NUMBER_FMT)), v.Vt)
i = 0
s = "1234e5"
vunsigned(&s, &i, &v)
assert.Equal(t, 4, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, types.ValueType(-int(types.ERR_INVALID_NUMBER_FMT)), v.Vt)
i = 0
s = "-1234e5"
vunsigned(&s, &i, &v)
assert.Equal(t, 0, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, types.ValueType(-int(types.ERR_INVALID_NUMBER_FMT)), v.Vt)
i = 0
s = "-1.234e5"
vunsigned(&s, &i, &v)
assert.Equal(t, 0, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, types.ValueType(-int(types.ERR_INVALID_NUMBER_FMT)), v.Vt)
i = 0
s = "-1.234e-5"
vunsigned(&s, &i, &v)
assert.Equal(t, 0, i)
assert.Equal(t, 0, v.Ep)
assert.Equal(t, types.ValueType(-int(types.ERR_INVALID_NUMBER_FMT)), v.Vt)
}
func TestNative_SkipOne(t *testing.T) {
p := 0
s := ` {"asdf": [null, true, false, 1, 2.0, -3]}, 1234.5`
q := skip_one(&s, &p, &types.StateMachine{}, uint64(0))
assert.Equal(t, 42, p)
assert.Equal(t, 1, q)
p = 0
s = `1 2.5 -3 "asdf\nqwer" true false null {} []`
q = skip_one(&s, &p, &types.StateMachine{}, uint64(0))
assert.Equal(t, 1, p)
assert.Equal(t, 0, q)
q = skip_one(&s, &p, &types.StateMachine{}, uint64(0))
assert.Equal(t, 5, p)
assert.Equal(t, 2, q)
q = skip_one(&s, &p, &types.StateMachine{}, uint64(0))
assert.Equal(t, 8, p)
assert.Equal(t, 6, q)
q = skip_one(&s, &p, &types.StateMachine{}, uint64(0))
assert.Equal(t, 21, p)
assert.Equal(t, 9, q)
q = skip_one(&s, &p, &types.StateMachine{}, uint64(0))
assert.Equal(t, 26, p)
assert.Equal(t, 22, q)
q = skip_one(&s, &p, &types.StateMachine{}, uint64(0))
assert.Equal(t, 32, p)
assert.Equal(t, 27, q)
q = skip_one(&s, &p, &types.StateMachine{}, uint64(0))
assert.Equal(t, 37, p)
assert.Equal(t, 33, q)
q = skip_one(&s, &p, &types.StateMachine{}, uint64(0))
assert.Equal(t, 40, p)
assert.Equal(t, 38, q)
q = skip_one(&s, &p, &types.StateMachine{}, uint64(0))
assert.Equal(t, 43, p)
assert.Equal(t, 41, q)
}
func TestNative_SkipOne_Error(t *testing.T) {
for _, s := range([]string{
"-", "+", "0.", "0. ", "+1", "0.0e ", "9e+", "0e-",
"tru", "fals", "nul", "trux", "fals ",
`"asdf`, `"\\\"`,
}) {
p := 0
q := skip_one(&s, &p, &types.StateMachine{}, uint64(0))
assert.True(t, q < 0)
}
}
func TestNative_SkipArray(t *testing.T) {
p := 0
s := `null, true, false, 1, 2.0, -3, {"asdf": "wqer"}],`
skip_array(&s, &p, &types.StateMachine{}, uint64(0))
assert.Equal(t, p, 48)
}
func TestNative_SkipObject(t *testing.T) {
p := 0
s := `"asdf": "wqer"},`
skip_object(&s, &p, &types.StateMachine{}, uint64(0))
assert.Equal(t, p, 15)
}
func TestNative_SkipNumber(t *testing.T) {
p := 0
s := `-1.23e+12`
q := skip_number(&s, &p)
assert.Equal(t, 9, p)
assert.Equal(t, 0, q)
}
func TestNative_SkipOneFast(t *testing.T) {
p := 0
s := ` {"asdf": [null, true, false, 1, 2.0, -3]}, 1234.5`
q := skip_one_fast(&s, &p)
assert.Equal(t, 42, p)
assert.Equal(t, 1, q)
p = 0
s = `1, 2.5, -3, "asdf\nqwer", true, false, null, {}, [],`
q = skip_one_fast(&s, &p)
assert.Equal(t, 1, p)
assert.Equal(t, 0, q)
p += 1
q = skip_one_fast(&s, &p)
assert.Equal(t, 6, p)
assert.Equal(t, 3, q)
p += 1
q = skip_one_fast(&s, &p)
assert.Equal(t, 10, p)
assert.Equal(t, 8, q)
p += 1
q = skip_one_fast(&s, &p)
assert.Equal(t, 24, p)
assert.Equal(t, 12, q)
p += 1
q = skip_one_fast(&s, &p)
assert.Equal(t, 30, p)
assert.Equal(t, 26, q)
p += 1
q = skip_one_fast(&s, &p)
assert.Equal(t, 37, p)
assert.Equal(t, 32, q)
p += 1
q = skip_one_fast(&s, &p)
assert.Equal(t, 43, p)
assert.Equal(t, 39, q)
p += 1
q = skip_one_fast(&s, &p)
assert.Equal(t, 47, p)
assert.Equal(t, 45, q)
p += 1
q = skip_one_fast(&s, &p)
assert.Equal(t, 51, p)
assert.Equal(t, 49, q)
}
func TestNative_SkipOneFast_Error(t *testing.T) {
for _, s := range([]string{
"{{", "[{", "{{}",
`"asdf`, `"\\\"`,
}) {
p := 0
q := skip_one_fast(&s, &p)
assert.True(t, q < 0)
}
}
func TestNative_GetByPath(t *testing.T) {
s := `{"asdf": [null, true, false, 1, 2.0, -3]}, 1234.5`
p := 0
path := []interface{}{"asdf", 4}
ret := get_by_path(&s, &p, &path, types.NewStateMachine())
assert.Equal(t, strings.Index(s, "2.0"), ret)
}
func BenchmarkNative_SkipOneFast(b *testing.B) {
b.ResetTimer()
for i:=0; i<b.N; i++ {
s := `{"asdf": [null, true, false, 1, 2.0, -3]}, 1234.5`
p := 0
_ = skip_one_fast(&s, &p)
}
}
func BenchmarkNative_GetByPath(b *testing.B) {
b.ResetTimer()
for i:=0; i<b.N; i++ {
s := `{"asdf": [null, true, false, 1, 2.0, -3]}, 1234.5`
p := 0
path := []interface{}{"asdf", 3}
sm := types.NewStateMachine()
_ = get_by_path(&s, &p, &path, sm)
types.FreeStateMachine(sm)
}
}

191
vendor/github.com/bytedance/sonic/internal/native/sse/native_amd64.go generated vendored
View File

@@ -1,191 +0,0 @@
// Code generated by Makefile, DO NOT EDIT.
// Code generated by Makefile, DO NOT EDIT.
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package sse
import (
`unsafe`
`github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/internal/rt`
)
var (
__i64toa func(out unsafe.Pointer, val int64) (ret int)
__u64toa func(out unsafe.Pointer, val uint64) (ret int)
__f64toa func(out unsafe.Pointer, val float64) (ret int)
__f32toa func(out unsafe.Pointer, val float32) (ret int)
__lspace func(sp unsafe.Pointer, nb int, off int) (ret int)
__quote func(sp unsafe.Pointer, nb int, dp unsafe.Pointer, dn unsafe.Pointer, flags uint64) (ret int)
__html_escape func(sp unsafe.Pointer, nb int, dp unsafe.Pointer, dn unsafe.Pointer) (ret int)
__unquote func(sp unsafe.Pointer, nb int, dp unsafe.Pointer, ep unsafe.Pointer, flags uint64) (ret int)
__value func(s unsafe.Pointer, n int, p int, v unsafe.Pointer, flags uint64) (ret int)
__vstring func(s unsafe.Pointer, p unsafe.Pointer, v unsafe.Pointer, flags uint64)
__vnumber func(s unsafe.Pointer, p unsafe.Pointer, v unsafe.Pointer)
__vsigned func(s unsafe.Pointer, p unsafe.Pointer, v unsafe.Pointer)
__vunsigned func(s unsafe.Pointer, p unsafe.Pointer, v unsafe.Pointer)
__skip_one func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer, flags uint64) (ret int)
__skip_one_fast func(s unsafe.Pointer, p unsafe.Pointer) (ret int)
__skip_array func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer, flags uint64) (ret int)
__skip_object func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer, flags uint64) (ret int)
__skip_number func(s unsafe.Pointer, p unsafe.Pointer) (ret int)
__validate_one func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer) (ret int)
__get_by_path func(s unsafe.Pointer, p unsafe.Pointer, path unsafe.Pointer, m unsafe.Pointer) (ret int)
__validate_utf8 func(s unsafe.Pointer, p unsafe.Pointer, m unsafe.Pointer) (ret int)
__validate_utf8_fast func(s unsafe.Pointer) (ret int)
__fsm_exec func(m unsafe.Pointer, s unsafe.Pointer, p unsafe.Pointer, flags uint64) (ret int)
)
//go:nosplit
func i64toa(out *byte, val int64) (ret int) {
return __i64toa(rt.NoEscape(unsafe.Pointer(out)), val)
}
//go:nosplit
func u64toa(out *byte, val uint64) (ret int) {
return __u64toa(rt.NoEscape(unsafe.Pointer(out)), val)
}
//go:nosplit
func f64toa(out *byte, val float64) (ret int) {
return __f64toa(rt.NoEscape(unsafe.Pointer(out)), val)
}
//go:nosplit
func f32toa(out *byte, val float32) (ret int) {
return __f32toa(rt.NoEscape(unsafe.Pointer(out)), val)
}
//go:nosplit
func lspace(sp unsafe.Pointer, nb int, off int) (ret int) {
return __lspace(rt.NoEscape(sp), nb, off)
}
//go:nosplit
func quote(sp unsafe.Pointer, nb int, dp unsafe.Pointer, dn *int, flags uint64) (ret int) {
return __quote(rt.NoEscape(unsafe.Pointer(sp)), nb, rt.NoEscape(unsafe.Pointer(dp)), rt.NoEscape(unsafe.Pointer(dn)), flags)
}
//go:nosplit
func html_escape(sp unsafe.Pointer, nb int, dp unsafe.Pointer, dn *int) (ret int) {
return __html_escape(rt.NoEscape(unsafe.Pointer(sp)), nb, rt.NoEscape(unsafe.Pointer(dp)), rt.NoEscape(unsafe.Pointer(dn)))
}
//go:nosplit
func unquote(sp unsafe.Pointer, nb int, dp unsafe.Pointer, ep *int, flags uint64) (ret int) {
return __unquote(rt.NoEscape(unsafe.Pointer(sp)), nb, rt.NoEscape(unsafe.Pointer(dp)), rt.NoEscape(unsafe.Pointer(ep)), flags)
}
//go:nosplit
func value(s unsafe.Pointer, n int, p int, v *types.JsonState, flags uint64) (ret int) {
return __value(rt.NoEscape(unsafe.Pointer(s)), n, p, rt.NoEscape(unsafe.Pointer(v)), flags)
}
//go:nosplit
func vstring(s *string, p *int, v *types.JsonState, flags uint64) {
__vstring(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(v)), flags)
}
//go:nosplit
func vnumber(s *string, p *int, v *types.JsonState) {
__vnumber(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(v)))
}
//go:nosplit
func vsigned(s *string, p *int, v *types.JsonState) {
__vsigned(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(v)))
}
//go:nosplit
func vunsigned(s *string, p *int, v *types.JsonState) {
__vunsigned(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(v)))
}
//go:nosplit
func skip_one(s *string, p *int, m *types.StateMachine, flags uint64) (ret int) {
return __skip_one(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(m)), flags)
}
//go:nosplit
func skip_one_fast(s *string, p *int) (ret int) {
return __skip_one_fast(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)))
}
//go:nosplit
func skip_array(s *string, p *int, m *types.StateMachine, flags uint64) (ret int) {
return __skip_array(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(m)), flags)
}
//go:nosplit
func skip_object(s *string, p *int, m *types.StateMachine, flags uint64) (ret int) {
return __skip_object(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(m)), flags)
}
//go:nosplit
func skip_number(s *string, p *int) (ret int) {
return __skip_number(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)))
}
//go:nosplit
func validate_one(s *string, p *int, m *types.StateMachine) (ret int) {
return __validate_one(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(m)))
}
//go:nosplit
func get_by_path(s *string, p *int, path *[]interface{}, m *types.StateMachine) (ret int) {
return __get_by_path(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(path)), rt.NoEscape(unsafe.Pointer(m)))
}
//go:nosplit
func validate_utf8(s *string, p *int, m *types.StateMachine) (ret int) {
return __validate_utf8(rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), rt.NoEscape(unsafe.Pointer(m)))
}
//go:nosplit
func validate_utf8_fast(s *string) (ret int) {
return __validate_utf8_fast(rt.NoEscape(unsafe.Pointer(s)))
}
//go:nosplit
func fsm_exec(m *types.StateMachine, s *string, p *int, flags uint64) (ret int) {
return __fsm_exec(rt.NoEscape(unsafe.Pointer(m)), rt.NoEscape(unsafe.Pointer(s)), rt.NoEscape(unsafe.Pointer(p)), flags)
}

604
vendor/github.com/bytedance/sonic/internal/native/sse/native_subr_amd64.go generated vendored
View File

@@ -1,604 +0,0 @@
// +build !noasm !appengine
// Code generated by asm2asm, DO NOT EDIT.
package sse
import (
`github.com/bytedance/sonic/loader`
)
const (
_entry__f32toa = 31616
_entry__f64toa = 160
_entry__format_significand = 35888
_entry__format_integer = 2960
_entry__fsm_exec = 18016
_entry__advance_string = 14352
_entry__advance_string_default = 37280
_entry__do_skip_number = 20608
_entry__get_by_path = 26176
_entry__skip_one_fast = 22272
_entry__html_escape = 8912
_entry__i64toa = 3392
_entry__u64toa = 3520
_entry__lspace = 16
_entry__quote = 4832
_entry__skip_array = 17984
_entry__skip_number = 21904
_entry__skip_object = 20256
_entry__skip_one = 22048
_entry__unquote = 6576
_entry__validate_one = 22096
_entry__validate_utf8 = 30384
_entry__validate_utf8_fast = 31056
_entry__value = 12352
_entry__vnumber = 15744
_entry__atof_eisel_lemire64 = 10192
_entry__atof_native = 11744
_entry__decimal_to_f64 = 10560
_entry__right_shift = 36848
_entry__left_shift = 36352
_entry__vsigned = 17296
_entry__vstring = 14176
_entry__vunsigned = 17632
)
const (
_stack__f32toa = 48
_stack__f64toa = 80
_stack__format_significand = 24
_stack__format_integer = 16
_stack__fsm_exec = 168
_stack__advance_string = 64
_stack__advance_string_default = 64
_stack__do_skip_number = 48
_stack__get_by_path = 272
_stack__skip_one_fast = 136
_stack__html_escape = 72
_stack__i64toa = 16
_stack__u64toa = 8
_stack__lspace = 8
_stack__quote = 64
_stack__skip_array = 176
_stack__skip_number = 88
_stack__skip_object = 176
_stack__skip_one = 176
_stack__unquote = 88
_stack__validate_one = 176
_stack__validate_utf8 = 48
_stack__validate_utf8_fast = 24
_stack__value = 328
_stack__vnumber = 240
_stack__atof_eisel_lemire64 = 32
_stack__atof_native = 136
_stack__decimal_to_f64 = 80
_stack__right_shift = 8
_stack__left_shift = 24
_stack__vsigned = 16
_stack__vstring = 120
_stack__vunsigned = 8
)
const (
_size__f32toa = 3328
_size__f64toa = 2800
_size__format_significand = 464
_size__format_integer = 432
_size__fsm_exec = 1692
_size__advance_string = 1344
_size__advance_string_default = 960
_size__do_skip_number = 956
_size__get_by_path = 4208
_size__skip_one_fast = 3404
_size__html_escape = 1280
_size__i64toa = 48
_size__u64toa = 1264
_size__lspace = 128
_size__quote = 1728
_size__skip_array = 32
_size__skip_number = 144
_size__skip_object = 32
_size__skip_one = 48
_size__unquote = 2272
_size__validate_one = 48
_size__validate_utf8 = 672
_size__validate_utf8_fast = 544
_size__value = 1316
_size__vnumber = 1552
_size__atof_eisel_lemire64 = 368
_size__atof_native = 608
_size__decimal_to_f64 = 1184
_size__right_shift = 400
_size__left_shift = 496
_size__vsigned = 336
_size__vstring = 128
_size__vunsigned = 336
)
var (
_pcsp__f32toa = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{3286, 48},
{3287, 40},
{3289, 32},
{3291, 24},
{3293, 16},
{3295, 8},
{3296, 0},
{3318, 48},
}
_pcsp__f64toa = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{2740, 56},
{2744, 48},
{2745, 40},
{2747, 32},
{2749, 24},
{2751, 16},
{2753, 8},
{2754, 0},
{2792, 56},
}
_pcsp__format_significand = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{452, 24},
{453, 16},
{455, 8},
{457, 0},
}
_pcsp__format_integer = [][2]uint32{
{1, 0},
{4, 8},
{412, 16},
{413, 8},
{414, 0},
{423, 16},
{424, 8},
{426, 0},
}
_pcsp__fsm_exec = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{1342, 104},
{1346, 48},
{1347, 40},
{1349, 32},
{1351, 24},
{1353, 16},
{1355, 8},
{1356, 0},
{1692, 104},
}
_pcsp__advance_string = [][2]uint32{
{14, 0},
{18, 8},
{20, 16},
{22, 24},
{24, 32},
{26, 40},
{27, 48},
{614, 56},
{618, 48},
{619, 40},
{621, 32},
{623, 24},
{625, 16},
{627, 8},
{628, 0},
{1339, 56},
}
_pcsp__advance_string_default = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{576, 64},
{580, 48},
{581, 40},
{583, 32},
{585, 24},
{587, 16},
{589, 8},
{590, 0},
{955, 64},
}
_pcsp__do_skip_number = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{881, 48},
{882, 40},
{884, 32},
{886, 24},
{888, 16},
{890, 8},
{891, 0},
{956, 48},
}
_pcsp__get_by_path = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{4012, 104},
{4016, 48},
{4017, 40},
{4019, 32},
{4021, 24},
{4023, 16},
{4025, 8},
{4026, 0},
{4194, 104},
}
_pcsp__skip_one_fast = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{658, 136},
{662, 48},
{663, 40},
{665, 32},
{667, 24},
{669, 16},
{671, 8},
{672, 0},
{3404, 136},
}
_pcsp__html_escape = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{1256, 72},
{1260, 48},
{1261, 40},
{1263, 32},
{1265, 24},
{1267, 16},
{1269, 8},
{1271, 0},
}
_pcsp__i64toa = [][2]uint32{
{14, 0},
{34, 8},
{36, 0},
}
_pcsp__u64toa = [][2]uint32{
{1, 0},
{161, 8},
{162, 0},
{457, 8},
{458, 0},
{772, 8},
{773, 0},
{1249, 8},
{1251, 0},
}
_pcsp__lspace = [][2]uint32{
{1, 0},
{89, 8},
{90, 0},
{103, 8},
{104, 0},
{111, 8},
{113, 0},
}
_pcsp__quote = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{1681, 64},
{1685, 48},
{1686, 40},
{1688, 32},
{1690, 24},
{1692, 16},
{1694, 8},
{1695, 0},
{1722, 64},
}
_pcsp__skip_array = [][2]uint32{
{1, 0},
{26, 8},
{32, 0},
}
_pcsp__skip_number = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{100, 40},
{101, 32},
{103, 24},
{105, 16},
{107, 8},
{108, 0},
{139, 40},
}
_pcsp__skip_object = [][2]uint32{
{1, 0},
{26, 8},
{32, 0},
}
_pcsp__skip_one = [][2]uint32{
{1, 0},
{30, 8},
{36, 0},
}
_pcsp__unquote = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{1684, 88},
{1688, 48},
{1689, 40},
{1691, 32},
{1693, 24},
{1695, 16},
{1697, 8},
{1698, 0},
{2270, 88},
}
_pcsp__validate_one = [][2]uint32{
{1, 0},
{35, 8},
{41, 0},
}
_pcsp__validate_utf8 = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{11, 40},
{623, 48},
{627, 40},
{628, 32},
{630, 24},
{632, 16},
{634, 8},
{635, 0},
{666, 48},
}
_pcsp__validate_utf8_fast = [][2]uint32{
{1, 0},
{4, 8},
{5, 16},
{247, 24},
{251, 16},
{252, 8},
{253, 0},
{527, 24},
{531, 16},
{532, 8},
{534, 0},
}
_pcsp__value = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{495, 88},
{499, 48},
{500, 40},
{502, 32},
{504, 24},
{506, 16},
{508, 8},
{509, 0},
{1316, 88},
}
_pcsp__vnumber = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{803, 104},
{807, 48},
{808, 40},
{810, 32},
{812, 24},
{814, 16},
{816, 8},
{817, 0},
{1551, 104},
}
_pcsp__atof_eisel_lemire64 = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{292, 32},
{293, 24},
{295, 16},
{297, 8},
{298, 0},
{362, 32},
}
_pcsp__atof_native = [][2]uint32{
{1, 0},
{4, 8},
{587, 56},
{591, 8},
{593, 0},
}
_pcsp__decimal_to_f64 = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{1144, 56},
{1148, 48},
{1149, 40},
{1151, 32},
{1153, 24},
{1155, 16},
{1157, 8},
{1158, 0},
{1169, 56},
}
_pcsp__right_shift = [][2]uint32{
{1, 0},
{318, 8},
{319, 0},
{387, 8},
{388, 0},
{396, 8},
{398, 0},
}
_pcsp__left_shift = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{363, 24},
{364, 16},
{366, 8},
{367, 0},
{470, 24},
{471, 16},
{473, 8},
{474, 0},
{486, 24},
}
_pcsp__vsigned = [][2]uint32{
{1, 0},
{4, 8},
{119, 16},
{120, 8},
{121, 0},
{132, 16},
{133, 8},
{134, 0},
{276, 16},
{277, 8},
{278, 0},
{282, 16},
{283, 8},
{284, 0},
{322, 16},
{323, 8},
{324, 0},
{332, 16},
{333, 8},
{335, 0},
}
_pcsp__vstring = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{11, 40},
{105, 56},
{109, 40},
{110, 32},
{112, 24},
{114, 16},
{116, 8},
{118, 0},
}
_pcsp__vunsigned = [][2]uint32{
{1, 0},
{78, 8},
{79, 0},
{90, 8},
{91, 0},
{114, 8},
{115, 0},
{273, 8},
{274, 0},
{312, 8},
{313, 0},
{320, 8},
{322, 0},
}
)
var Funcs = []loader.CFunc{
{"__native_entry__", 0, 67, 0, nil},
{"_f32toa", _entry__f32toa, _size__f32toa, _stack__f32toa, _pcsp__f32toa},
{"_f64toa", _entry__f64toa, _size__f64toa, _stack__f64toa, _pcsp__f64toa},
{"_format_significand", _entry__format_significand, _size__format_significand, _stack__format_significand, _pcsp__format_significand},
{"_format_integer", _entry__format_integer, _size__format_integer, _stack__format_integer, _pcsp__format_integer},
{"_fsm_exec", _entry__fsm_exec, _size__fsm_exec, _stack__fsm_exec, _pcsp__fsm_exec},
{"_advance_string", _entry__advance_string, _size__advance_string, _stack__advance_string, _pcsp__advance_string},
{"_advance_string_default", _entry__advance_string_default, _size__advance_string_default, _stack__advance_string_default, _pcsp__advance_string_default},
{"_do_skip_number", _entry__do_skip_number, _size__do_skip_number, _stack__do_skip_number, _pcsp__do_skip_number},
{"_get_by_path", _entry__get_by_path, _size__get_by_path, _stack__get_by_path, _pcsp__get_by_path},
{"_skip_one_fast", _entry__skip_one_fast, _size__skip_one_fast, _stack__skip_one_fast, _pcsp__skip_one_fast},
{"_html_escape", _entry__html_escape, _size__html_escape, _stack__html_escape, _pcsp__html_escape},
{"_i64toa", _entry__i64toa, _size__i64toa, _stack__i64toa, _pcsp__i64toa},
{"_u64toa", _entry__u64toa, _size__u64toa, _stack__u64toa, _pcsp__u64toa},
{"_lspace", _entry__lspace, _size__lspace, _stack__lspace, _pcsp__lspace},
{"_quote", _entry__quote, _size__quote, _stack__quote, _pcsp__quote},
{"_skip_array", _entry__skip_array, _size__skip_array, _stack__skip_array, _pcsp__skip_array},
{"_skip_number", _entry__skip_number, _size__skip_number, _stack__skip_number, _pcsp__skip_number},
{"_skip_object", _entry__skip_object, _size__skip_object, _stack__skip_object, _pcsp__skip_object},
{"_skip_one", _entry__skip_one, _size__skip_one, _stack__skip_one, _pcsp__skip_one},
{"_unquote", _entry__unquote, _size__unquote, _stack__unquote, _pcsp__unquote},
{"_validate_one", _entry__validate_one, _size__validate_one, _stack__validate_one, _pcsp__validate_one},
{"_validate_utf8", _entry__validate_utf8, _size__validate_utf8, _stack__validate_utf8, _pcsp__validate_utf8},
{"_validate_utf8_fast", _entry__validate_utf8_fast, _size__validate_utf8_fast, _stack__validate_utf8_fast, _pcsp__validate_utf8_fast},
{"_value", _entry__value, _size__value, _stack__value, _pcsp__value},
{"_vnumber", _entry__vnumber, _size__vnumber, _stack__vnumber, _pcsp__vnumber},
{"_atof_eisel_lemire64", _entry__atof_eisel_lemire64, _size__atof_eisel_lemire64, _stack__atof_eisel_lemire64, _pcsp__atof_eisel_lemire64},
{"_atof_native", _entry__atof_native, _size__atof_native, _stack__atof_native, _pcsp__atof_native},
{"_decimal_to_f64", _entry__decimal_to_f64, _size__decimal_to_f64, _stack__decimal_to_f64, _pcsp__decimal_to_f64},
{"_right_shift", _entry__right_shift, _size__right_shift, _stack__right_shift, _pcsp__right_shift},
{"_left_shift", _entry__left_shift, _size__left_shift, _stack__left_shift, _pcsp__left_shift},
{"_vsigned", _entry__vsigned, _size__vsigned, _stack__vsigned, _pcsp__vsigned},
{"_vstring", _entry__vstring, _size__vstring, _stack__vstring, _pcsp__vstring},
{"_vunsigned", _entry__vunsigned, _size__vunsigned, _stack__vunsigned, _pcsp__vunsigned},
}

7
vendor/github.com/bytedance/sonic/internal/native/types/types.go generated vendored
View File

@@ -22,13 +22,15 @@ import (
`unsafe` `unsafe`
) )
type ValueType int type ValueType = int64
type ParsingError uint type ParsingError uint
type SearchingError uint type SearchingError uint
// NOTE: !NOT MODIFIED ONLY. // NOTE: !NOT MODIFIED ONLY.
// This definitions are followed in native/types.h. // This definitions are followed in native/types.h.
const BufPaddingSize int = 64
const ( const (
V_EOF ValueType = 1 V_EOF ValueType = 1
V_NULL ValueType = 2 V_NULL ValueType = 2
@@ -55,6 +57,9 @@ const (
B_USE_NUMBER = 1 B_USE_NUMBER = 1
B_VALIDATE_STRING = 5 B_VALIDATE_STRING = 5
B_ALLOW_CONTROL = 31 B_ALLOW_CONTROL = 31
// for native.SkipOne() flags
B_NO_VALIDATE_JSON= 6
) )
const ( const (

40
vendor/github.com/bytedance/sonic/internal/resolver/resolver.go generated vendored
View File

@@ -17,10 +17,11 @@
package resolver package resolver
import ( import (
`fmt` "fmt"
`reflect` "reflect"
`strings` "strings"
`sync` "sync"
_ "unsafe"
) )
type FieldOpts int type FieldOpts int
@@ -29,6 +30,7 @@ type OffsetType int
const ( const (
F_omitempty FieldOpts = 1 << iota F_omitempty FieldOpts = 1 << iota
F_stringize F_stringize
F_omitzero
) )
const ( const (
@@ -47,6 +49,7 @@ type FieldMeta struct {
Path []Offset Path []Offset
Opts FieldOpts Opts FieldOpts
Type reflect.Type Type reflect.Type
IsZero func(reflect.Value) bool
} }
func (self *FieldMeta) String() string { func (self *FieldMeta) String() string {
@@ -117,20 +120,26 @@ func resolveFields(vt reflect.Type) []FieldMeta {
/* convert each field */ /* convert each field */
for _, fv := range tfv.list { for _, fv := range tfv.list {
/* add to result */
ret = append(ret, FieldMeta{})
fm := &ret[len(ret)-1]
item := vt item := vt
path := []Offset(nil) path := []Offset(nil)
opts := FieldOpts(0)
/* check for "string" */ /* check for "string" */
if fv.quoted { if fv.quoted {
opts |= F_stringize fm.Opts |= F_stringize
} }
/* check for "omitempty" */ /* check for "omitempty" */
if fv.omitEmpty { if fv.omitEmpty {
opts |= F_omitempty fm.Opts |= F_omitempty
} }
/* handle the "omitzero" */
handleOmitZero(fv, fm)
/* dump the field path */ /* dump the field path */
for _, i := range fv.index { for _, i := range fv.index {
kind := F_offset kind := F_offset
@@ -161,13 +170,9 @@ func resolveFields(vt reflect.Type) []FieldMeta {
path[idx].Kind = F_offset path[idx].Kind = F_offset
} }
/* add to result */ fm.Type = fvt
ret = append(ret, FieldMeta { fm.Path = path
Type: fvt, fm.Name = fv.name
Opts: opts,
Path: path,
Name: fv.name,
})
} }
/* optimize the offsets */ /* optimize the offsets */
@@ -212,3 +217,10 @@ func ResolveStruct(vt reflect.Type) []FieldMeta {
fieldCache[vt] = fm fieldCache[vt] = fm
return fm return fm
} }
func handleOmitZero(fv StdField, fm *FieldMeta) {
if fv.omitZero {
fm.Opts |= F_omitzero
fm.IsZero = fv.isZero
}
}

42
vendor/github.com/bytedance/sonic/internal/rt/asm_amd64.s generated vendored
View File

@@ -1,4 +1,4 @@
// +build !noasm !appengine // +build !noasm,amd64 !appengine,amd64
// Code generated by asm2asm, DO NOT EDIT· // Code generated by asm2asm, DO NOT EDIT·
#include "go_asm.h" #include "go_asm.h"
@@ -18,43 +18,3 @@ _entry:
_stack_grow: _stack_grow:
CALL runtime·morestack_noctxt<>(SB) CALL runtime·morestack_noctxt<>(SB)
JMP _entry JMP _entry
TEXT ·StopProf(SB), NOSPLIT, $0-0
NO_LOCAL_POINTERS
CMPB github·com∕bytedance∕sonic∕internal∕rt·StopProfiling(SB), $0
JEQ _ret_1
MOVL $1, AX
LEAQ github·com∕bytedance∕sonic∕internal∕rt·yieldCount(SB), CX
LOCK
XADDL AX, (CX)
MOVL runtime·prof+4(SB), AX
TESTL AX, AX
JEQ _ret_1
MOVL AX, github·com∕bytedance∕sonic∕internal∕rt·oldHz(SB)
MOVL $0, runtime·prof+4(SB)
_ret_1:
RET
TEXT ·StartProf(SB), NOSPLIT, $0-0
NO_LOCAL_POINTERS
CMPB github·com∕bytedance∕sonic∕internal∕rt·StopProfiling(SB), $0
JEQ _ret_2
MOVL $-1, AX
LEAQ github·com∕bytedance∕sonic∕internal∕rt·yieldCount(SB), CX
LOCK
XADDL AX, (CX)
CMPL github·com∕bytedance∕sonic∕internal∕rt·yieldCount(SB), $0
JNE _ret_2
CMPL runtime·prof+4(SB), $0
JNE _ret_2
CMPL github·com∕bytedance∕sonic∕internal∕rt·oldHz(SB), $0
JNE _branch_1
MOVL $100, github·com∕bytedance∕sonic∕internal∕rt·oldHz(SB)
_branch_1:
MOVL github·com∕bytedance∕sonic∕internal∕rt·oldHz(SB), AX
MOVL AX, runtime·prof+4(SB)
_ret_2:
RET

10
vendor/github.com/bytedance/sonic/internal/rt/asm_arm64.s generated vendored
View File

@@ -1,10 +0,0 @@
// +build !noasm !appengine
// Code generated by asm2asm, DO NOT EDIT.
#include "go_asm.h"
#include "funcdata.h"
#include "textflag.h"
TEXT ·MoreStack(SB), NOSPLIT, $0 - 8
NO_LOCAL_POINTERS
RET

37
vendor/github.com/bytedance/sonic/internal/rt/fastmem.go generated vendored
View File

@@ -17,8 +17,10 @@
package rt package rt
import ( import (
`unsafe` "reflect"
`reflect` "unsafe"
"github.com/bytedance/sonic/option"
) )
//go:nosplit //go:nosplit
@@ -66,15 +68,16 @@ func FuncAddr(f interface{}) unsafe.Pointer {
} }
} }
//go:nocheckptr
func IndexChar(src string, index int) unsafe.Pointer { func IndexChar(src string, index int) unsafe.Pointer {
return unsafe.Pointer(uintptr((*GoString)(unsafe.Pointer(&src)).Ptr) + uintptr(index)) return unsafe.Pointer(uintptr((*GoString)(unsafe.Pointer(&src)).Ptr) + uintptr(index))
} }
//go:nocheckptr
func IndexByte(ptr []byte, index int) unsafe.Pointer { func IndexByte(ptr []byte, index int) unsafe.Pointer {
return unsafe.Pointer(uintptr((*GoSlice)(unsafe.Pointer(&ptr)).Ptr) + uintptr(index)) return unsafe.Pointer(uintptr((*GoSlice)(unsafe.Pointer(&ptr)).Ptr) + uintptr(index))
} }
//go:nosplit
func GuardSlice(buf *[]byte, n int) { func GuardSlice(buf *[]byte, n int) {
c := cap(*buf) c := cap(*buf)
l := len(*buf) l := len(*buf)
@@ -89,6 +92,21 @@ func GuardSlice(buf *[]byte, n int) {
} }
} }
func GuardSlice2(buf []byte, n int) []byte {
c := cap(buf)
l := len(buf)
if c-l < n {
c = c>>1 + n + l
if c < 32 {
c = 32
}
tmp := make([]byte, l, c)
copy(tmp, buf)
buf = tmp
}
return buf
}
//go:nosplit //go:nosplit
func Ptr2SlicePtr(s unsafe.Pointer, l int, c int) unsafe.Pointer { func Ptr2SlicePtr(s unsafe.Pointer, l int, c int) unsafe.Pointer {
slice := &GoSlice{ slice := &GoSlice{
@@ -122,3 +140,16 @@ func NoEscape(p unsafe.Pointer) unsafe.Pointer {
x := uintptr(p) x := uintptr(p)
return unsafe.Pointer(x ^ 0) return unsafe.Pointer(x ^ 0)
} }
//go:nosplit
func MoreStack(size uintptr)
//go:nosplit
func Add(ptr unsafe.Pointer, off uintptr) unsafe.Pointer {
return unsafe.Pointer(uintptr(ptr) + off)
}
// CanSizeResue
func CanSizeResue(cap int) bool {
return cap <= int(option.LimitBufferSize)
}

226
vendor/github.com/bytedance/sonic/internal/rt/fastvalue.go generated vendored
View File

@@ -17,197 +17,215 @@
package rt package rt
import ( import (
`reflect` "reflect"
`unsafe` "unsafe"
) )
var ( var (
reflectRtypeItab = findReflectRtypeItab() reflectRtypeItab = findReflectRtypeItab()
) )
// GoType.KindFlags const // GoType.KindFlags const
const ( const (
F_direct = 1 << 5 F_direct = 1 << 5
F_kind_mask = (1 << 5) - 1 F_kind_mask = (1 << 5) - 1
) )
// GoType.Flags const // GoType.Flags const
const ( const (
tflagUncommon uint8 = 1 << 0 tflagUncommon uint8 = 1 << 0
tflagExtraStar uint8 = 1 << 1 tflagExtraStar uint8 = 1 << 1
tflagNamed uint8 = 1 << 2 tflagNamed uint8 = 1 << 2
tflagRegularMemory uint8 = 1 << 3 tflagRegularMemory uint8 = 1 << 3
) )
type GoType struct { type GoType struct {
Size uintptr Size uintptr
PtrData uintptr PtrData uintptr
Hash uint32 Hash uint32
Flags uint8 Flags uint8
Align uint8 Align uint8
FieldAlign uint8 FieldAlign uint8
KindFlags uint8 KindFlags uint8
Traits unsafe.Pointer Traits unsafe.Pointer
GCData *byte GCData *byte
Str int32 Str int32
PtrToSelf int32 PtrToSelf int32
} }
func (self *GoType) IsNamed() bool { func (self *GoType) IsNamed() bool {
return (self.Flags & tflagNamed) != 0 return (self.Flags & tflagNamed) != 0
} }
func (self *GoType) Kind() reflect.Kind { func (self *GoType) Kind() reflect.Kind {
return reflect.Kind(self.KindFlags & F_kind_mask) return reflect.Kind(self.KindFlags & F_kind_mask)
} }
func (self *GoType) Pack() (t reflect.Type) { func (self *GoType) Pack() (t reflect.Type) {
(*GoIface)(unsafe.Pointer(&t)).Itab = reflectRtypeItab (*GoIface)(unsafe.Pointer(&t)).Itab = reflectRtypeItab
(*GoIface)(unsafe.Pointer(&t)).Value = unsafe.Pointer(self) (*GoIface)(unsafe.Pointer(&t)).Value = unsafe.Pointer(self)
return return
} }
func (self *GoType) String() string { func (self *GoType) String() string {
return self.Pack().String() return self.Pack().String()
} }
func (self *GoType) Indirect() bool { func (self *GoType) Indirect() bool {
return self.KindFlags & F_direct == 0 return self.KindFlags&F_direct == 0
}
type GoMap struct {
Count int
Flags uint8
B uint8
Overflow uint16
Hash0 uint32
Buckets unsafe.Pointer
OldBuckets unsafe.Pointer
Evacuate uintptr
Extra unsafe.Pointer
}
type GoMapIterator struct {
K unsafe.Pointer
V unsafe.Pointer
T *GoMapType
H *GoMap
Buckets unsafe.Pointer
Bptr *unsafe.Pointer
Overflow *[]unsafe.Pointer
OldOverflow *[]unsafe.Pointer
StartBucket uintptr
Offset uint8
Wrapped bool
B uint8
I uint8
Bucket uintptr
CheckBucket uintptr
} }
type GoItab struct { type GoItab struct {
it unsafe.Pointer it unsafe.Pointer
Vt *GoType Vt *GoType
hv uint32 hv uint32
_ [4]byte _ [4]byte
fn [1]uintptr fn [1]uintptr
} }
type GoIface struct { type GoIface struct {
Itab *GoItab Itab *GoItab
Value unsafe.Pointer Value unsafe.Pointer
} }
type GoEface struct { type GoEface struct {
Type *GoType Type *GoType
Value unsafe.Pointer Value unsafe.Pointer
} }
func (self GoEface) Pack() (v interface{}) { func (self GoEface) Pack() (v interface{}) {
*(*GoEface)(unsafe.Pointer(&v)) = self *(*GoEface)(unsafe.Pointer(&v)) = self
return return
} }
type GoPtrType struct { type GoPtrType struct {
GoType GoType
Elem *GoType Elem *GoType
} }
type GoMapType struct { type GoMapType struct {
GoType GoType
Key *GoType Key *GoType
Elem *GoType Elem *GoType
Bucket *GoType Bucket *GoType
Hasher func(unsafe.Pointer, uintptr) uintptr Hasher func(unsafe.Pointer, uintptr) uintptr
KeySize uint8 KeySize uint8
ElemSize uint8 ElemSize uint8
BucketSize uint16 BucketSize uint16
Flags uint32 Flags uint32
} }
func (self *GoMapType) IndirectElem() bool { func (self *GoMapType) IndirectElem() bool {
return self.Flags & 2 != 0 return self.Flags&2 != 0
} }
type GoStructType struct { type GoStructType struct {
GoType GoType
Pkg *byte Pkg *byte
Fields []GoStructField Fields []GoStructField
} }
type GoStructField struct { type GoStructField struct {
Name *byte Name *byte
Type *GoType Type *GoType
OffEmbed uintptr OffEmbed uintptr
} }
type GoInterfaceType struct { type GoInterfaceType struct {
GoType GoType
PkgPath *byte PkgPath *byte
Methods []GoInterfaceMethod Methods []GoInterfaceMethod
} }
type GoInterfaceMethod struct { type GoInterfaceMethod struct {
Name int32 Name int32
Type int32 Type int32
} }
type GoSlice struct { type GoSlice struct {
Ptr unsafe.Pointer Ptr unsafe.Pointer
Len int Len int
Cap int Cap int
} }
type GoString struct { type GoString struct {
Ptr unsafe.Pointer Ptr unsafe.Pointer
Len int Len int
} }
func PtrElem(t *GoType) *GoType { func PtrElem(t *GoType) *GoType {
return (*GoPtrType)(unsafe.Pointer(t)).Elem return (*GoPtrType)(unsafe.Pointer(t)).Elem
} }
func MapType(t *GoType) *GoMapType { func MapType(t *GoType) *GoMapType {
return (*GoMapType)(unsafe.Pointer(t)) return (*GoMapType)(unsafe.Pointer(t))
} }
func IfaceType(t *GoType) *GoInterfaceType { func IfaceType(t *GoType) *GoInterfaceType {
return (*GoInterfaceType)(unsafe.Pointer(t)) return (*GoInterfaceType)(unsafe.Pointer(t))
} }
func UnpackType(t reflect.Type) *GoType { func UnpackType(t reflect.Type) *GoType {
return (*GoType)((*GoIface)(unsafe.Pointer(&t)).Value) return (*GoType)((*GoIface)(unsafe.Pointer(&t)).Value)
} }
func UnpackEface(v interface{}) GoEface { func UnpackEface(v interface{}) GoEface {
return *(*GoEface)(unsafe.Pointer(&v)) return *(*GoEface)(unsafe.Pointer(&v))
} }
func UnpackIface(v interface{}) GoIface { func UnpackIface(v interface{}) GoIface {
return *(*GoIface)(unsafe.Pointer(&v)) return *(*GoIface)(unsafe.Pointer(&v))
} }
func findReflectRtypeItab() *GoItab { func findReflectRtypeItab() *GoItab {
v := reflect.TypeOf(struct{}{}) v := reflect.TypeOf(struct{}{})
return (*GoIface)(unsafe.Pointer(&v)).Itab return (*GoIface)(unsafe.Pointer(&v)).Itab
} }
func AssertI2I2(t *GoType, i GoIface) (r GoIface) {
inter := IfaceType(t)
tab := i.Itab
if tab == nil {
return
}
if (*GoInterfaceType)(tab.it) != inter {
tab = GetItab(inter, tab.Vt, true)
if tab == nil {
return
}
}
r.Itab = tab
r.Value = i.Value
return
}
func (t *GoType) IsInt64() bool {
return t.Kind() == reflect.Int64 || (t.Kind() == reflect.Int && t.Size == 8)
}
func (t *GoType) IsInt32() bool {
return t.Kind() == reflect.Int32 || (t.Kind() == reflect.Int && t.Size == 4)
}
//go:nosplit
func (t *GoType) IsUint64() bool {
isUint := t.Kind() == reflect.Uint || t.Kind() == reflect.Uintptr
return t.Kind() == reflect.Uint64 || (isUint && t.Size == 8)
}
//go:nosplit
func (t *GoType) IsUint32() bool {
isUint := t.Kind() == reflect.Uint || t.Kind() == reflect.Uintptr
return t.Kind() == reflect.Uint32 || (isUint && t.Size == 4)
}
//go:nosplit
func PtrAdd(ptr unsafe.Pointer, offset uintptr) unsafe.Pointer {
return unsafe.Pointer(uintptr(ptr) + offset)
}
//go:noescape
//go:linkname GetItab runtime.getitab
func GetItab(inter *GoInterfaceType, typ *GoType, canfail bool) *GoItab

55
vendor/github.com/bytedance/sonic/internal/rt/gcwb.go generated vendored
View File

@@ -1,3 +1,5 @@
// +build go1.21,!go1.26
/* /*
* Copyright 2021 ByteDance Inc. * Copyright 2021 ByteDance Inc.
* *
@@ -17,13 +19,18 @@
package rt package rt
import ( import (
`os`
`sync/atomic` `sync/atomic`
`unsafe` `unsafe`
`golang.org/x/arch/x86/x86asm` `golang.org/x/arch/x86/x86asm`
) )
//go:linkname GcWriteBarrier2 runtime.gcWriteBarrier2
func GcWriteBarrier2()
//go:linkname RuntimeWriteBarrier runtime.writeBarrier
var RuntimeWriteBarrier uintptr
const ( const (
_MaxInstr = 15 _MaxInstr = 15
) )
@@ -76,49 +83,3 @@ func GcwbAddr() uintptr {
} }
} }
// StopProfiling is used to stop traceback introduced by SIGPROF while native code is running.
// WARN: this option is only a workaround for traceback issue (https://github.com/bytedance/sonic/issues/310),
// and will be dropped when the issue is fixed.
var StopProfiling = os.Getenv("SONIC_STOP_PROFILING") != ""
// WARN: must be aligned with runtime.Prof
// type Prof struct {
// signalLock uint32
// hz int32
// }
var (
// // go:linkname runtimeProf runtime.prof
// runtimeProf Prof
// count of native-C calls
yieldCount uint32
// previous value of runtimeProf.hz
oldHz int32
)
//go:nosplit
func MoreStack(size uintptr)
func StopProf()
// func StopProf() {
// atomic.AddUint32(&yieldCount, 1)
// if runtimeProf.hz != 0 {
// oldHz = runtimeProf.hz
// runtimeProf.hz = 0
// }
// }
func StartProf()
// func StartProf() {
// atomic.AddUint32(&yieldCount, ^uint32(0))
// if yieldCount == 0 && runtimeProf.hz == 0 {
// if oldHz == 0 {
// oldHz = 100
// }
// runtimeProf.hz = oldHz
// }
// }

6
vendor/github.com/bytedance/sonic/internal/rt/int48.go generated vendored
View File

@@ -17,12 +17,12 @@
package rt package rt
const ( const (
MinInt48 = -(1 << 47) MinInt48 int64 = -(1 << 47)
MaxInt48 = +(1 << 47) - 1 MaxInt48 int64 = +(1 << 47) - 1
) )
func PackInt(v int) uint64 { func PackInt(v int) uint64 {
if u := uint64(v); v < MinInt48 || v > MaxInt48 { if u := uint64(v); int64(v) < MinInt48 || int64(v) > MaxInt48 {
panic("int48 out of range") panic("int48 out of range")
} else { } else {
return ((u >> 63) << 47) | (u & 0x00007fffffffffff) return ((u >> 63) << 47) | (u & 0x00007fffffffffff)

181
vendor/github.com/bytedance/sonic/internal/rt/stackmap.go generated vendored
View File

@@ -1,181 +0,0 @@
/**
* Copyright 2023 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package rt
import (
`fmt`
`strings`
`unsafe`
)
type Bitmap struct {
N int
B []byte
}
func (self *Bitmap) grow() {
if self.N >= len(self.B) * 8 {
self.B = append(self.B, 0)
}
}
func (self *Bitmap) mark(i int, bv int) {
if bv != 0 {
self.B[i / 8] |= 1 << (i % 8)
} else {
self.B[i / 8] &^= 1 << (i % 8)
}
}
func (self *Bitmap) Set(i int, bv int) {
if i >= self.N {
panic("bitmap: invalid bit position")
} else {
self.mark(i, bv)
}
}
func (self *Bitmap) Append(bv int) {
self.grow()
self.mark(self.N, bv)
self.N++
}
func (self *Bitmap) AppendMany(n int, bv int) {
for i := 0; i < n; i++ {
self.Append(bv)
}
}
// var (
// _stackMapLock = sync.Mutex{}
// _stackMapCache = make(map[*StackMap]struct{})
// )
type BitVec struct {
N uintptr
B unsafe.Pointer
}
func (self BitVec) Bit(i uintptr) byte {
return (*(*byte)(unsafe.Pointer(uintptr(self.B) + i / 8)) >> (i % 8)) & 1
}
func (self BitVec) String() string {
var i uintptr
var v []string
/* add each bit */
for i = 0; i < self.N; i++ {
v = append(v, fmt.Sprintf("%d", self.Bit(i)))
}
/* join them together */
return fmt.Sprintf(
"BitVec { %s }",
strings.Join(v, ", "),
)
}
type StackMap struct {
N int32
L int32
B [1]byte
}
// func (self *StackMap) add() {
// _stackMapLock.Lock()
// _stackMapCache[self] = struct{}{}
// _stackMapLock.Unlock()
// }
func (self *StackMap) Pin() uintptr {
// self.add()
return uintptr(unsafe.Pointer(self))
}
func (self *StackMap) Get(i int32) BitVec {
return BitVec {
N: uintptr(self.L),
B: unsafe.Pointer(uintptr(unsafe.Pointer(&self.B)) + uintptr(i * ((self.L + 7) >> 3))),
}
}
func (self *StackMap) String() string {
sb := strings.Builder{}
sb.WriteString("StackMap {")
/* dump every stack map */
for i := int32(0); i < self.N; i++ {
sb.WriteRune('\n')
sb.WriteString(" " + self.Get(i).String())
}
/* close the stackmap */
sb.WriteString("\n}")
return sb.String()
}
func (self *StackMap) MarshalBinary() ([]byte, error) {
size := int(self.N) * int(self.L) + int(unsafe.Sizeof(self.L)) + int(unsafe.Sizeof(self.N))
return BytesFrom(unsafe.Pointer(self), size, size), nil
}
var (
byteType = UnpackEface(byte(0)).Type
)
const (
_StackMapSize = unsafe.Sizeof(StackMap{})
)
//go:linkname mallocgc runtime.mallocgc
//goland:noinspection GoUnusedParameter
func mallocgc(nb uintptr, vt *GoType, zero bool) unsafe.Pointer
type StackMapBuilder struct {
b Bitmap
}
//go:nocheckptr
func (self *StackMapBuilder) Build() (p *StackMap) {
nb := len(self.b.B)
bm := mallocgc(_StackMapSize + uintptr(nb) - 1, byteType, false)
/* initialize as 1 bitmap of N bits */
p = (*StackMap)(bm)
p.N, p.L = 1, int32(self.b.N)
copy(BytesFrom(unsafe.Pointer(&p.B), nb, nb), self.b.B)
return
}
func (self *StackMapBuilder) AddField(ptr bool) {
if ptr {
self.b.Append(1)
} else {
self.b.Append(0)
}
}
func (self *StackMapBuilder) AddFields(n int, ptr bool) {
if ptr {
self.b.AppendMany(n, 1)
} else {
self.b.AppendMany(n, 0)
}
}

461
vendor/github.com/bytedance/sonic/loader/funcdata_go116.go generated vendored
View File

@@ -1,461 +0,0 @@
//go:build go1.16 && !go1.18
// +build go1.16,!go1.18
/*
* Copyright 2021 ByteDance Inc.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package loader
import (
`os`
`unsafe`
`sort`
`github.com/bytedance/sonic/internal/rt`
)
const (
_Magic uint32 = 0xfffffffa
)
type pcHeader struct {
magic uint32 // 0xFFFFFFF0
pad1, pad2 uint8 // 0,0
minLC uint8 // min instruction size
ptrSize uint8 // size of a ptr in bytes
nfunc int // number of functions in the module
nfiles uint // number of entries in the file tab
funcnameOffset uintptr // offset to the funcnametab variable from pcHeader
cuOffset uintptr // offset to the cutab variable from pcHeader
filetabOffset uintptr // offset to the filetab variable from pcHeader
pctabOffset uintptr // offset to the pctab variable from pcHeader
pclnOffset uintptr // offset to the pclntab variable from pcHeader
}
type moduledata struct {
pcHeader *pcHeader
funcnametab []byte
cutab []uint32
filetab []byte
pctab []byte
pclntable []byte
ftab []funcTab
findfunctab uintptr
minpc, maxpc uintptr // first func address, last func address + last func size
text, etext uintptr // start/end of text, (etext-text) must be greater than MIN_FUNC
noptrdata, enoptrdata uintptr
data, edata uintptr
bss, ebss uintptr
noptrbss, enoptrbss uintptr
end, gcdata, gcbss uintptr
types, etypes uintptr
textsectmap []textSection // see runtime/symtab.go: textAddr()
typelinks []int32 // offsets from types
itablinks []*rt.GoItab
ptab []ptabEntry
pluginpath string
pkghashes []modulehash
modulename string
modulehashes []modulehash
hasmain uint8 // 1 if module contains the main function, 0 otherwise
gcdatamask, gcbssmask bitVector
typemap map[int32]*rt.GoType // offset to *_rtype in previous module
bad bool // module failed to load and should be ignored
next *moduledata
}
type _func struct {
entry uintptr // start pc, as offset from moduledata.text/pcHeader.textStart
nameOff int32 // function name, as index into moduledata.funcnametab.
args int32 // in/out args size
deferreturn uint32 // offset of start of a deferreturn call instruction from entry, if any.
pcsp uint32
pcfile uint32
pcln uint32
npcdata uint32
cuOffset uint32 // runtime.cutab offset of this function's CU
funcID uint8 // set for certain special runtime functions
_ [2]byte // pad
nfuncdata uint8 //
// The end of the struct is followed immediately by two variable-length
// arrays that reference the pcdata and funcdata locations for this
// function.
// pcdata contains the offset into moduledata.pctab for the start of
// that index's table. e.g.,
// &moduledata.pctab[_func.pcdata[_PCDATA_UnsafePoint]] is the start of
// the unsafe point table.
//
// An offset of 0 indicates that there is no table.
//
// pcdata [npcdata]uint32
// funcdata contains the offset past moduledata.gofunc which contains a
// pointer to that index's funcdata. e.g.,
// *(moduledata.gofunc + _func.funcdata[_FUNCDATA_ArgsPointerMaps]) is
// the argument pointer map.
//
// An offset of ^uint32(0) indicates that there is no entry.
//
// funcdata [nfuncdata]uint32
}
type funcTab struct {
entry uintptr
funcoff uintptr
}
type bitVector struct {
n int32 // # of bits
bytedata *uint8
}
type ptabEntry struct {
name int32
typ int32
}
type textSection struct {
vaddr uintptr // prelinked section vaddr
end uintptr // vaddr + section length
baseaddr uintptr // relocated section address
}
type modulehash struct {
modulename string
linktimehash string
runtimehash *string
}
// findfuncbucket is an array of these structures.
// Each bucket represents 4096 bytes of the text segment.
// Each subbucket represents 256 bytes of the text segment.
// To find a function given a pc, locate the bucket and subbucket for
// that pc. Add together the idx and subbucket value to obtain a
// function index. Then scan the functab array starting at that
// index to find the target function.
// This table uses 20 bytes for every 4096 bytes of code, or ~0.5% overhead.
type findfuncbucket struct {
idx uint32
_SUBBUCKETS [16]byte
}
type compilationUnit struct {
fileNames []string
}
func makeFtab(funcs []_func, maxpc uintptr) (ftab []funcTab, pclntabSize int64, startLocations []uint32) {
// Allocate space for the pc->func table. This structure consists of a pc offset
// and an offset to the func structure. After that, we have a single pc
// value that marks the end of the last function in the binary.
pclntabSize = int64(len(funcs)*2*int(_PtrSize) + int(_PtrSize))
startLocations = make([]uint32, len(funcs))
for i, f := range funcs {
pclntabSize = rnd(pclntabSize, int64(_PtrSize))
//writePCToFunc
startLocations[i] = uint32(pclntabSize)
pclntabSize += int64(uint8(_FUNC_SIZE) + f.nfuncdata*_PtrSize + uint8(f.npcdata)*4)
}
ftab = make([]funcTab, 0, len(funcs)+1)
// write a map of pc->func info offsets
for i, f := range funcs {
ftab = append(ftab, funcTab{uintptr(f.entry), uintptr(startLocations[i])})
}
// Final entry of table is just end pc offset.
ftab = append(ftab, funcTab{maxpc, 0})
return
}
// Pcln table format: [...]funcTab + [...]_Func
func makePclntable(size int64, startLocations []uint32, funcs []_func, maxpc uintptr, pcdataOffs [][]uint32, funcdataAddr uintptr, funcdataOffs [][]uint32) (pclntab []byte) {
pclntab = make([]byte, size, size)
// write a map of pc->func info offsets
offs := 0
for i, f := range funcs {
byteOrder.PutUint64(pclntab[offs:offs+8], uint64(f.entry))
byteOrder.PutUint64(pclntab[offs+8:offs+16], uint64(startLocations[i]))
offs += 16
}
// Final entry of table is just end pc offset.
byteOrder.PutUint64(pclntab[offs:offs+8], uint64(maxpc))
offs += 8
// write func info table
for i, f := range funcs {
off := startLocations[i]
// write _func structure to pclntab
byteOrder.PutUint64(pclntab[off:off+8], uint64(f.entry))
off += 8
byteOrder.PutUint32(pclntab[off:off+4], uint32(f.nameOff))
off += 4
byteOrder.PutUint32(pclntab[off:off+4], uint32(f.args))
off += 4
byteOrder.PutUint32(pclntab[off:off+4], uint32(f.deferreturn))
off += 4
byteOrder.PutUint32(pclntab[off:off+4], uint32(f.pcsp))
off += 4
byteOrder.PutUint32(pclntab[off:off+4], uint32(f.pcfile))
off += 4
byteOrder.PutUint32(pclntab[off:off+4], uint32(f.pcln))
off += 4
byteOrder.PutUint32(pclntab[off:off+4], uint32(f.npcdata))
off += 4
byteOrder.PutUint32(pclntab[off:off+4], uint32(f.cuOffset))
off += 4
pclntab[off] = f.funcID
// NOTICE: _[2]byte alignment
off += 3
pclntab[off] = f.nfuncdata
off += 1
// NOTICE: _func.pcdata always starts from PcUnsafePoint, which is index 3
for j := 3; j < len(pcdataOffs[i]); j++ {
byteOrder.PutUint32(pclntab[off:off+4], uint32(pcdataOffs[i][j]))
off += 4
}
off = uint32(rnd(int64(off), int64(_PtrSize)))
// funcdata refs as offsets from gofunc
for _, funcdata := range funcdataOffs[i] {
if funcdata == _INVALID_FUNCDATA_OFFSET {
byteOrder.PutUint64(pclntab[off:off+8], 0)
} else {
byteOrder.PutUint64(pclntab[off:off+8], uint64(funcdataAddr)+uint64(funcdata))
}
off += 8
}
}
return
}
// findfunc table used to map pc to belonging func,
// returns the index in the func table.
//
// All text section are divided into buckets sized _BUCKETSIZE(4K):
// every bucket is divided into _SUBBUCKETS sized _SUB_BUCKETSIZE(64),
// and it has a base idx to plus the offset stored in jth subbucket.
// see findfunc() in runtime/symtab.go
func writeFindfunctab(out *[]byte, ftab []funcTab) (start int) {
start = len(*out)
max := ftab[len(ftab)-1].entry
min := ftab[0].entry
nbuckets := (max - min + _BUCKETSIZE - 1) / _BUCKETSIZE
n := (max - min + _SUB_BUCKETSIZE - 1) / _SUB_BUCKETSIZE
tab := make([]findfuncbucket, 0, nbuckets)
var s, e = 0, 0
for i := 0; i<int(nbuckets); i++ {
// store the start func of the bucket
var fb = findfuncbucket{idx: uint32(s)}
// find the last e-th func of the bucket
var pc = min + uintptr((i+1)*_BUCKETSIZE)
for ; e < len(ftab)-1 && ftab[e+1].entry <= pc; e++ {}
for j := 0; j<_SUBBUCKETS && (i*_SUBBUCKETS+j)<int(n); j++ {
// store the start func of the subbucket
fb._SUBBUCKETS[j] = byte(uint32(s) - fb.idx)
// find the s-th end func of the subbucket
pc = min + uintptr(i*_BUCKETSIZE) + uintptr((j+1)*_SUB_BUCKETSIZE)
for ; s < len(ftab)-1 && ftab[s+1].entry <= pc; s++ {}
}
s = e
tab = append(tab, fb)
}
// write findfuncbucket
if len(tab) > 0 {
size := int(unsafe.Sizeof(findfuncbucket{}))*len(tab)
*out = append(*out, rt.BytesFrom(unsafe.Pointer(&tab[0]), size, size)...)
}
return
}
func makeModuledata(name string, filenames []string, funcsp *[]Func, text []byte) (mod *moduledata) {
mod = new(moduledata)
mod.modulename = name
// sort funcs by entry
funcs := *funcsp
sort.Slice(funcs, func(i, j int) bool {
return funcs[i].EntryOff < funcs[j].EntryOff
})
*funcsp = funcs
// make filename table
cu := make([]string, 0, len(filenames))
cu = append(cu, filenames...)
cutab, filetab, cuOffs := makeFilenametab([]compilationUnit{{cu}})
mod.cutab = cutab
mod.filetab = filetab
// make funcname table
funcnametab, nameOffs := makeFuncnameTab(funcs)
mod.funcnametab = funcnametab
// mmap() text and funcdata segements
p := os.Getpagesize()
size := int(rnd(int64(len(text)), int64(p)))
addr := mmap(size)
// copy the machine code
s := rt.BytesFrom(unsafe.Pointer(addr), len(text), size)
copy(s, text)
// make it executable
mprotect(addr, size)
// assign addresses
mod.text = addr
mod.etext = addr + uintptr(size)
mod.minpc = addr
mod.maxpc = addr + uintptr(len(text))
// make pcdata table
// NOTICE: _func only use offset to index pcdata, thus no need mmap() pcdata
cuOff := cuOffs[0]
pctab, pcdataOffs, _funcs := makePctab(funcs, addr, cuOff, nameOffs)
mod.pctab = pctab
// write func data
// NOTICE: _func use mod.gofunc+offset to directly point funcdata, thus need cache funcdata
// TODO: estimate accurate capacity
cache := make([]byte, 0, len(funcs)*int(_PtrSize))
fstart, funcdataOffs := writeFuncdata(&cache, funcs)
// make pc->func (binary search) func table
ftab, pclntSize, startLocations := makeFtab(_funcs, mod.maxpc)
mod.ftab = ftab
// write pc->func (modmap) findfunc table
ffstart := writeFindfunctab(&cache, ftab)
// cache funcdata and findfuncbucket
moduleCache.Lock()
moduleCache.m[mod] = cache
moduleCache.Unlock()
mod.findfunctab = uintptr(rt.IndexByte(cache, ffstart))
funcdataAddr := uintptr(rt.IndexByte(cache, fstart))
// make pclnt table
pclntab := makePclntable(pclntSize, startLocations, _funcs, mod.maxpc, pcdataOffs, funcdataAddr, funcdataOffs)
mod.pclntable = pclntab
// make pc header
mod.pcHeader = &pcHeader {
magic : _Magic,
minLC : _MinLC,
ptrSize : _PtrSize,
nfunc : len(funcs),
nfiles: uint(len(cu)),
funcnameOffset: getOffsetOf(moduledata{}, "funcnametab"),
cuOffset: getOffsetOf(moduledata{}, "cutab"),
filetabOffset: getOffsetOf(moduledata{}, "filetab"),
pctabOffset: getOffsetOf(moduledata{}, "pctab"),
pclnOffset: getOffsetOf(moduledata{}, "pclntable"),
}
// sepecial case: gcdata and gcbss must by non-empty
mod.gcdata = uintptr(unsafe.Pointer(&emptyByte))
mod.gcbss = uintptr(unsafe.Pointer(&emptyByte))
return
}
// makePctab generates pcdelta->valuedelta tables for functions,
// and returns the table and the entry offset of every kind pcdata in the table.
func makePctab(funcs []Func, addr uintptr, cuOffset uint32, nameOffset []int32) (pctab []byte, pcdataOffs [][]uint32, _funcs []_func) {
_funcs = make([]_func, len(funcs))
// Pctab offsets of 0 are considered invalid in the runtime. We respect
// that by just padding a single byte at the beginning of runtime.pctab,
// that way no real offsets can be zero.
pctab = make([]byte, 1, 12*len(funcs)+1)
pcdataOffs = make([][]uint32, len(funcs))
for i, f := range funcs {
_f := &_funcs[i]
var writer = func(pc *Pcdata) {
var ab []byte
var err error
if pc != nil {
ab, err = pc.MarshalBinary()
if err != nil {
panic(err)
}
pcdataOffs[i] = append(pcdataOffs[i], uint32(len(pctab)))
} else {
ab = []byte{0}
pcdataOffs[i] = append(pcdataOffs[i], _PCDATA_INVALID_OFFSET)
}
pctab = append(pctab, ab...)
}
if f.Pcsp != nil {
_f.pcsp = uint32(len(pctab))
}
writer(f.Pcsp)
if f.Pcfile != nil {
_f.pcfile = uint32(len(pctab))
}
writer(f.Pcfile)
if f.Pcline != nil {
_f.pcln = uint32(len(pctab))
}
writer(f.Pcline)
writer(f.PcUnsafePoint)
writer(f.PcStackMapIndex)
writer(f.PcInlTreeIndex)
writer(f.PcArgLiveIndex)
_f.entry = addr + uintptr(f.EntryOff)
_f.nameOff = nameOffset[i]
_f.args = f.ArgsSize
_f.deferreturn = f.DeferReturn
// NOTICE: _func.pcdata is always as [PCDATA_UnsafePoint(0) : PCDATA_ArgLiveIndex(3)]
_f.npcdata = uint32(_N_PCDATA)
_f.cuOffset = cuOffset
_f.funcID = f.ID
_f.nfuncdata = uint8(_N_FUNCDATA)
}
return
}
func registerFunction(name string, pc uintptr, textSize uintptr, fp int, args int, size uintptr, argptrs uintptr, localptrs uintptr) {}

2
vendor/github.com/bytedance/sonic/loader/funcdata_go118.go generated vendored
View File

@@ -21,7 +21,7 @@
package loader package loader
import ( import (
`github.com/bytedance/sonic/internal/rt` `github.com/bytedance/sonic/loader/internal/rt`
) )
const ( const (

2
vendor/github.com/bytedance/sonic/loader/funcdata_go120.go generated vendored
View File

@@ -20,7 +20,7 @@
package loader package loader
import ( import (
`github.com/bytedance/sonic/internal/rt` `github.com/bytedance/sonic/loader/internal/rt`
) )
const ( const (

4
vendor/github.com/bytedance/sonic/loader/mmap_unix.go generated vendored
View File

@@ -1,5 +1,5 @@
//go:build darwin || linux //go:build !windows
// +build darwin linux // +build !windows
/** /**
* Copyright 2023 ByteDance Inc. * Copyright 2023 ByteDance Inc.

12
vendor/github.com/bytedance/sonic/loader/stubs.go generated vendored
View File

@@ -17,7 +17,6 @@
package loader package loader
import ( import (
`sync`
_ `unsafe` _ `unsafe`
) )
@@ -25,16 +24,5 @@ import (
//goland:noinspection GoUnusedGlobalVariable //goland:noinspection GoUnusedGlobalVariable
var lastmoduledatap *moduledata var lastmoduledatap *moduledata
var moduledataMux sync.Mutex
func registerModule(mod *moduledata) {
moduledataMux.Lock()
lastmoduledatap.next = mod
lastmoduledatap = mod
moduledataMux.Unlock()
}
//go:linkname moduledataverify1 runtime.moduledataverify1 //go:linkname moduledataverify1 runtime.moduledataverify1
func moduledataverify1(_ *moduledata) func moduledataverify1(_ *moduledata)

14
vendor/github.com/bytedance/sonic/option/option.go generated vendored
View File

@@ -18,10 +18,17 @@ package option
var ( var (
// DefaultDecoderBufferSize is the initial buffer size of StreamDecoder // DefaultDecoderBufferSize is the initial buffer size of StreamDecoder
DefaultDecoderBufferSize uint = 128 * 1024 DefaultDecoderBufferSize uint = 4 * 1024
// DefaultEncoderBufferSize is the initial buffer size of Encoder // DefaultEncoderBufferSize is the initial buffer size of Encoder
DefaultEncoderBufferSize uint = 128 * 1024 DefaultEncoderBufferSize uint = 4 * 1024
// DefaultAstBufferSize is the initial buffer size of ast.Node.MarshalJSON()
DefaultAstBufferSize uint = 4 * 1024
// LimitBufferSize indicates the max pool buffer size, in case of OOM.
// See issue https://github.com/bytedance/sonic/issues/614
LimitBufferSize uint = 1024 * 1024
) )
// CompileOptions includes all options for encoder or decoder compiler. // CompileOptions includes all options for encoder or decoder compiler.
@@ -61,7 +68,7 @@ type CompileOption func(o *CompileOptions)
// //
// For deep nested struct (depth exceeds MaxInlineDepth), // For deep nested struct (depth exceeds MaxInlineDepth),
// try to set more loops to completely compile, // try to set more loops to completely compile,
// thus reduce JIT unstability in the first hit. // thus reduce JIT instability in the first hit.
func WithCompileRecursiveDepth(loop int) CompileOption { func WithCompileRecursiveDepth(loop int) CompileOption {
return func(o *CompileOptions) { return func(o *CompileOptions) {
if loop < 0 { if loop < 0 {
@@ -83,4 +90,3 @@ func WithCompileMaxInlineDepth(depth int) CompileOption {
o.MaxInlineDepth = depth o.MaxInlineDepth = depth
} }
} }

59
vendor/github.com/bytedance/sonic/sonic.go generated vendored
View File

@@ -1,4 +1,5 @@
// +build amd64,go1.16,!go1.22 //go:build (amd64 && go1.17 && !go1.26) || (arm64 && go1.20 && !go1.26)
// +build amd64,go1.17,!go1.26 arm64,go1.20,!go1.26
/* /*
* Copyright 2021 ByteDance Inc. * Copyright 2021 ByteDance Inc.
@@ -16,7 +17,6 @@
* limitations under the License. * limitations under the License.
*/ */
//go:generate make
package sonic package sonic
import ( import (
@@ -29,6 +29,8 @@ import (
`github.com/bytedance/sonic/internal/rt` `github.com/bytedance/sonic/internal/rt`
) )
const apiKind = UseSonicJSON
type frozenConfig struct { type frozenConfig struct {
Config Config
encoderOpts encoder.Options encoderOpts encoder.Options
@@ -58,8 +60,20 @@ func (cfg Config) Froze() API {
if cfg.ValidateString { if cfg.ValidateString {
api.encoderOpts |= encoder.ValidateString api.encoderOpts |= encoder.ValidateString
} }
if cfg.NoValidateJSONMarshaler {
api.encoderOpts |= encoder.NoValidateJSONMarshaler
}
if cfg.NoEncoderNewline {
api.encoderOpts |= encoder.NoEncoderNewline
}
if cfg.EncodeNullForInfOrNan {
api.encoderOpts |= encoder.EncodeNullForInfOrNan
}
// configure decoder options: // configure decoder options:
if cfg.NoValidateJSONSkip {
api.decoderOpts |= decoder.OptionNoValidateJSON
}
if cfg.UseInt64 { if cfg.UseInt64 {
api.decoderOpts |= decoder.OptionUseInt64 api.decoderOpts |= decoder.OptionUseInt64
} }
@@ -75,6 +89,9 @@ func (cfg Config) Froze() API {
if cfg.ValidateString { if cfg.ValidateString {
api.decoderOpts |= decoder.OptionValidateString api.decoderOpts |= decoder.OptionValidateString
} }
if cfg.CaseSensitive {
api.decoderOpts |= decoder.OptionCaseSensitive
}
return api return api
} }
@@ -139,23 +156,23 @@ func (cfg frozenConfig) Valid(data []byte) bool {
// Opts are the compile options, for example, "option.WithCompileRecursiveDepth" is // Opts are the compile options, for example, "option.WithCompileRecursiveDepth" is
// a compile option to set the depth of recursive compile for the nested struct type. // a compile option to set the depth of recursive compile for the nested struct type.
func Pretouch(vt reflect.Type, opts ...option.CompileOption) error { func Pretouch(vt reflect.Type, opts ...option.CompileOption) error {
if err := encoder.Pretouch(vt, opts...); err != nil { if err := encoder.Pretouch(vt, opts...); err != nil {
return err return err
} }
if err := decoder.Pretouch(vt, opts...); err != nil { if err := decoder.Pretouch(vt, opts...); err != nil {
return err return err
} }
// to pretouch the corresponding pointer type as well // to pretouch the corresponding pointer type as well
if vt.Kind() == reflect.Ptr { if vt.Kind() == reflect.Ptr {
vt = vt.Elem() vt = vt.Elem()
} else { } else {
vt = reflect.PtrTo(vt) vt = reflect.PtrTo(vt)
} }
if err := encoder.Pretouch(vt, opts...); err != nil { if err := encoder.Pretouch(vt, opts...); err != nil {
return err return err
} }
if err := decoder.Pretouch(vt, opts...); err != nil { if err := decoder.Pretouch(vt, opts...); err != nil {
return err return err
} }
return nil return nil
} }

35
vendor/github.com/bytedance/sonic/unquote/unquote.go generated vendored
View File

@@ -1,3 +1,7 @@
//go:build (amd64 && go1.17 && !go1.26) || (arm64 && go1.20 && !go1.26)
// +build amd64,go1.17,!go1.26 arm64,go1.20,!go1.26
/* /*
* Copyright 2021 ByteDance Inc. * Copyright 2021 ByteDance Inc.
* *
@@ -25,27 +29,45 @@ import (
`github.com/bytedance/sonic/internal/rt` `github.com/bytedance/sonic/internal/rt`
) )
// String unescapes an escaped string (not including `"` at beginning and end)
// It validates invalid UTF8 and replace with `\ufffd`
func String(s string) (ret string, err types.ParsingError) { func String(s string) (ret string, err types.ParsingError) {
mm := make([]byte, 0, len(s)) mm := make([]byte, 0, len(s))
err = intoBytesUnsafe(s, &mm) err = intoBytesUnsafe(s, &mm, true)
ret = rt.Mem2Str(mm) ret = rt.Mem2Str(mm)
return return
} }
// IntoBytes is same with String besides it output result into a buffer m
func IntoBytes(s string, m *[]byte) types.ParsingError { func IntoBytes(s string, m *[]byte) types.ParsingError {
if cap(*m) < len(s) { if cap(*m) < len(s) {
return types.ERR_EOF return types.ERR_EOF
} else { } else {
return intoBytesUnsafe(s, m) return intoBytesUnsafe(s, m, true)
} }
} }
func intoBytesUnsafe(s string, m *[]byte) types.ParsingError { // String unescapes an escaped string (not including `"` at beginning and end)
// - replace enables replacing invalid utf8 escaped char with `\uffd`
func _String(s string, replace bool) (ret string, err error) {
mm := make([]byte, 0, len(s))
err = intoBytesUnsafe(s, &mm, replace)
ret = rt.Mem2Str(mm)
return
}
func intoBytesUnsafe(s string, m *[]byte, replace bool) types.ParsingError {
pos := -1 pos := -1
slv := (*rt.GoSlice)(unsafe.Pointer(m)) slv := (*rt.GoSlice)(unsafe.Pointer(m))
str := (*rt.GoString)(unsafe.Pointer(&s)) str := (*rt.GoString)(unsafe.Pointer(&s))
/* unquote as the default configuration, replace invalid unicode with \ufffd */
ret := native.Unquote(str.Ptr, str.Len, slv.Ptr, &pos, types.F_UNICODE_REPLACE) flags := uint64(0)
if replace {
/* unquote as the default configuration, replace invalid unicode with \ufffd */
flags |= types.F_UNICODE_REPLACE
}
ret := native.Unquote(str.Ptr, str.Len, slv.Ptr, &pos, flags)
/* check for errors */ /* check for errors */
if ret < 0 { if ret < 0 {
@@ -57,3 +79,6 @@ func intoBytesUnsafe(s string, m *[]byte) types.ParsingError {
runtime.KeepAlive(s) runtime.KeepAlive(s)
return 0 return 0
} }

19
vendor/github.com/bytedance/sonic/utf8/utf8.go generated vendored
View File

@@ -1,3 +1,6 @@
//go:build (amd64 && go1.17 && !go1.26) || (arm64 && go1.20 && !go1.26)
// +build amd64,go1.17,!go1.26 arm64,go1.20,!go1.26
/* /*
* Copyright 2022 ByteDance Inc. * Copyright 2022 ByteDance Inc.
* *
@@ -17,6 +20,8 @@
package utf8 package utf8
import ( import (
`runtime`
`github.com/bytedance/sonic/internal/rt` `github.com/bytedance/sonic/internal/rt`
`github.com/bytedance/sonic/internal/native/types` `github.com/bytedance/sonic/internal/native/types`
`github.com/bytedance/sonic/internal/native` `github.com/bytedance/sonic/internal/native`
@@ -27,7 +32,7 @@ func CorrectWith(dst []byte, src []byte, repl string) []byte {
sstr := rt.Mem2Str(src) sstr := rt.Mem2Str(src)
sidx := 0 sidx := 0
/* state machine records the invalid postions */ /* state machine records the invalid positions */
m := types.NewStateMachine() m := types.NewStateMachine()
m.Sp = 0 // invalid utf8 numbers m.Sp = 0 // invalid utf8 numbers
@@ -60,12 +65,20 @@ func CorrectWith(dst []byte, src []byte, repl string) []byte {
return dst return dst
} }
// Validate is a simd-accelereated drop-in replacement for the standard library's utf8.Valid. // Validate is a simd-accelerated drop-in replacement for the standard library's utf8.Valid.
func Validate(src []byte) bool { func Validate(src []byte) bool {
if src == nil {
return true
}
return ValidateString(rt.Mem2Str(src)) return ValidateString(rt.Mem2Str(src))
} }
// ValidateString as Validate, but for string. // ValidateString as Validate, but for string.
func ValidateString(src string) bool { func ValidateString(src string) bool {
return native.ValidateUTF8Fast(&src) == 0 if src == "" {
return true
}
ret := native.ValidateUTF8Fast(&src) == 0
runtime.KeepAlive(src)
return ret
} }

43
vendor/github.com/chenzhuoyu/base64x/.gitignore generated vendored
View File

@@ -1,43 +0,0 @@
*.o
*.swp
*.swm
*.swn
*.a
*.so
_obj
_test
*.[568vq]
[568vq].out
*.cgo1.go
*.cgo2.c
_cgo_defun.c
_cgo_gotypes.go
_cgo_export.*
_testmain.go
*.exe
*.exe~
*.test
*.prof
*.rar
*.zip
*.gz
*.psd
*.bmd
*.cfg
*.pptx
*.log
*nohup.out
*settings.pyc
*.sublime-project
*.sublime-workspace
.DS_Store
/.idea/
/.vscode/
/output/
/vendor/
/Gopkg.lock
/Gopkg.toml
coverage.html
coverage.out
coverage.xml
junit.xml

3
vendor/github.com/chenzhuoyu/base64x/.gitmodules generated vendored
View File

@@ -1,3 +0,0 @@
[submodule "tools/asm2asm"]
path = tools/asm2asm
url = https://github.com/chenzhuoyu/asm2asm

201
vendor/github.com/chenzhuoyu/base64x/LICENSE generated vendored
View File

@@ -1,201 +0,0 @@
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

29
vendor/github.com/chenzhuoyu/base64x/Makefile generated vendored
View File

@@ -1,29 +0,0 @@
.PHONY: all clean
CFLAGS := -mavx
CFLAGS += -mavx2
CFLAGS += -mno-bmi
CFLAGS += -mno-red-zone
CFLAGS += -fno-asynchronous-unwind-tables
CFLAGS += -fno-stack-protector
CFLAGS += -fno-exceptions
CFLAGS += -fno-builtin
CFLAGS += -fno-rtti
CFLAGS += -nostdlib
CFLAGS += -O3
NATIVE_ASM := $(wildcard native/*.S)
NATIVE_SRC := $(wildcard native/*.h)
NATIVE_SRC += $(wildcard native/*.c)
all: native_amd64.s
clean:
rm -vf native_text_amd64.go native_subr_amd64.go output/*.s
native_amd64.s: ${NATIVE_SRC} ${NATIVE_ASM} native_amd64.go
mkdir -p output
clang ${CFLAGS} -S -o output/native.s native/native.c
python3 tools/asm2asm/asm2asm.py -r native_amd64.go output/native.s ${NATIVE_ASM}
awk '{gsub(/Text__native_entry__/, "text__native_entry__")}1' native_text_amd64.go > native_text_amd64.go.tmp && mv native_text_amd64.go.tmp native_text_amd64.go
awk '{gsub(/Funcs/, "funcs")}1' native_subr_amd64.go > native_subr_amd64.go.tmp && mv native_subr_amd64.go.tmp native_subr_amd64.go

4
vendor/github.com/chenzhuoyu/base64x/README.md generated vendored
View File

@@ -1,4 +0,0 @@
# base64x
High performance drop-in replacement of the `encoding/base64` library.

151
vendor/github.com/chenzhuoyu/base64x/base64x.go generated vendored
View File

@@ -1,151 +0,0 @@
package base64x
import (
`encoding/base64`
)
// An Encoding is a radix 64 encoding/decoding scheme, defined by a
// 64-character alphabet. The most common encoding is the "base64"
// encoding defined in RFC 4648 and used in MIME (RFC 2045) and PEM
// (RFC 1421). RFC 4648 also defines an alternate encoding, which is
// the standard encoding with - and _ substituted for + and /.
type Encoding int
const (
_MODE_URL = 1 << 0
_MODE_RAW = 1 << 1
_MODE_AVX2 = 1 << 2
_MODE_JSON = 1 << 3
)
// StdEncoding is the standard base64 encoding, as defined in
// RFC 4648.
const StdEncoding Encoding = 0
// URLEncoding is the alternate base64 encoding defined in RFC 4648.
// It is typically used in URLs and file names.
const URLEncoding Encoding = _MODE_URL
// RawStdEncoding is the standard raw, unpadded base64 encoding,
// as defined in RFC 4648 section 3.2.
//
// This is the same as StdEncoding but omits padding characters.
const RawStdEncoding Encoding = _MODE_RAW
// RawURLEncoding is the unpadded alternate base64 encoding defined in RFC 4648.
// It is typically used in URLs and file names.
//
// This is the same as URLEncoding but omits padding characters.
const RawURLEncoding Encoding = _MODE_RAW | _MODE_URL
// JSONStdEncoding is the StdEncoding and encoded as JSON string as RFC 8259.
const JSONStdEncoding Encoding = _MODE_JSON;
var (
archFlags = 0
)
/** Encoder Functions **/
// Encode encodes src using the specified encoding, writing
// EncodedLen(len(src)) bytes to out.
//
// The encoding pads the output to a multiple of 4 bytes,
// so Encode is not appropriate for use on individual blocks
// of a large data stream.
//
// If out is not large enough to contain the encoded result,
// it will panic.
func (self Encoding) Encode(out []byte, src []byte) {
if len(src) != 0 {
if buf := out[:0:len(out)]; self.EncodedLen(len(src)) <= len(out) {
self.EncodeUnsafe(&buf, src)
} else {
panic("encoder output buffer is too small")
}
}
}
// EncodeUnsafe behaves like Encode, except it does NOT check if
// out is large enough to contain the encoded result.
//
// It will also update the length of out.
func (self Encoding) EncodeUnsafe(out *[]byte, src []byte) {
b64encode(out, &src, int(self) | archFlags)
}
// EncodeToString returns the base64 encoding of src.
func (self Encoding) EncodeToString(src []byte) string {
nbs := len(src)
ret := make([]byte, 0, self.EncodedLen(nbs))
/* encode in native code */
self.EncodeUnsafe(&ret, src)
return mem2str(ret)
}
// EncodedLen returns the length in bytes of the base64 encoding
// of an input buffer of length n.
func (self Encoding) EncodedLen(n int) int {
if (self & _MODE_RAW) == 0 {
return (n + 2) / 3 * 4
} else {
return (n * 8 + 5) / 6
}
}
/** Decoder Functions **/
// Decode decodes src using the encoding enc. It writes at most
// DecodedLen(len(src)) bytes to out and returns the number of bytes
// written. If src contains invalid base64 data, it will return the
// number of bytes successfully written and base64.CorruptInputError.
//
// New line characters (\r and \n) are ignored.
//
// If out is not large enough to contain the encoded result,
// it will panic.
func (self Encoding) Decode(out []byte, src []byte) (int, error) {
if len(src) == 0 {
return 0, nil
} else if buf := out[:0:len(out)]; self.DecodedLen(len(src)) <= len(out) {
return self.DecodeUnsafe(&buf, src)
} else {
panic("decoder output buffer is too small")
}
}
// DecodeUnsafe behaves like Decode, except it does NOT check if
// out is large enough to contain the decoded result.
//
// It will also update the length of out.
func (self Encoding) DecodeUnsafe(out *[]byte, src []byte) (int, error) {
if n := b64decode(out, mem2addr(src), len(src), int(self) | archFlags); n >= 0 {
return n, nil
} else {
return 0, base64.CorruptInputError(-n - 1)
}
}
// DecodeString returns the bytes represented by the base64 string s.
func (self Encoding) DecodeString(s string) ([]byte, error) {
src := str2mem(s)
ret := make([]byte, 0, self.DecodedLen(len(s)))
/* decode into the allocated buffer */
if _, err := self.DecodeUnsafe(&ret, src); err != nil {
return nil, err
} else {
return ret, nil
}
}
// DecodedLen returns the maximum length in bytes of the decoded data
// corresponding to n bytes of base64-encoded data.
func (self Encoding) DecodedLen(n int) int {
if (self & _MODE_RAW) == 0 {
return n / 4 * 3
} else {
return n * 6 / 8
}
}

17
vendor/github.com/chenzhuoyu/base64x/cpuid.go generated vendored
View File

@@ -1,17 +0,0 @@
package base64x
import (
`fmt`
`os`
`github.com/klauspost/cpuid/v2`
)
func hasAVX2() bool {
switch v := os.Getenv("B64X_MODE"); v {
case "" : fallthrough
case "auto" : return cpuid.CPU.Has(cpuid.AVX2)
case "noavx2" : return false
default : panic(fmt.Sprintf("invalid mode: '%s', should be one of 'auto', 'noavx2'", v))
}
}

35
vendor/github.com/chenzhuoyu/base64x/faststr.go generated vendored
View File

@@ -1,35 +0,0 @@
package base64x
import (
`reflect`
`unsafe`
)
func mem2str(v []byte) (s string) {
(*reflect.StringHeader)(unsafe.Pointer(&s)).Len = (*reflect.SliceHeader)(unsafe.Pointer(&v)).Len
(*reflect.StringHeader)(unsafe.Pointer(&s)).Data = (*reflect.SliceHeader)(unsafe.Pointer(&v)).Data
return
}
func str2mem(s string) (v []byte) {
(*reflect.SliceHeader)(unsafe.Pointer(&v)).Cap = (*reflect.StringHeader)(unsafe.Pointer(&s)).Len
(*reflect.SliceHeader)(unsafe.Pointer(&v)).Len = (*reflect.StringHeader)(unsafe.Pointer(&s)).Len
(*reflect.SliceHeader)(unsafe.Pointer(&v)).Data = (*reflect.StringHeader)(unsafe.Pointer(&s)).Data
return
}
func mem2addr(v []byte) unsafe.Pointer {
return *(*unsafe.Pointer)(unsafe.Pointer(&v))
}
// NoEscape hides a pointer from escape analysis. NoEscape is
// the identity function but escape analysis doesn't think the
// output depends on the input. NoEscape is inlined and currently
// compiles down to zero instructions.
// USE CAREFULLY!
//go:nosplit
//goland:noinspection GoVetUnsafePointer
func noEscape(p unsafe.Pointer) unsafe.Pointer {
x := uintptr(p)
return unsafe.Pointer(x ^ 0)
}

42
vendor/github.com/chenzhuoyu/base64x/native_amd64.go generated vendored
View File

@@ -1,42 +0,0 @@
//go:generate make
package base64x
import (
`unsafe`
`github.com/bytedance/sonic/loader`
)
//go:nosplit
func b64encode(out *[]byte, src *[]byte, mode int) {
__b64encode(noEscape(unsafe.Pointer(out)), noEscape(unsafe.Pointer(src)), mode)
}
//go:nosplit
func b64decode(out *[]byte, src unsafe.Pointer, len int, mode int) (ret int) {
return __b64decode(noEscape(unsafe.Pointer(out)), noEscape(unsafe.Pointer(src)), len, mode)
}
// asm2asm templates
var (
__b64encode func(out unsafe.Pointer, src unsafe.Pointer, mod int)
__b64decode func(out unsafe.Pointer, src unsafe.Pointer, len int, mod int) (ret int)
)
// directly jump PCs
var (
_subr__b64encode uintptr
_subr__b64decode uintptr
)
var stubs = []loader.GoC{
{"_b64encode", &_subr__b64encode, &__b64encode},
{"_b64decode", &_subr__b64decode, &__b64decode},
}
func init() {
if hasAVX2() {
archFlags = _MODE_AVX2
}
loader.WrapGoC(text__native_entry__, funcs, stubs, "base64x", "base64x/native.c")
}

63
vendor/github.com/chenzhuoyu/base64x/native_subr_amd64.go generated vendored
View File

@@ -1,63 +0,0 @@
// +build !noasm !appengine
// Code generated by asm2asm, DO NOT EDIT.
package base64x
import (
`github.com/bytedance/sonic/loader`
)
const (
_entry__b64decode = 1328
_entry__b64encode = 256
)
const (
_stack__b64decode = 152
_stack__b64encode = 40
)
const (
_size__b64decode = 17616
_size__b64encode = 864
)
var (
_pcsp__b64decode = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{12, 40},
{13, 48},
{17560, 152},
{17564, 48},
{17565, 40},
{17567, 32},
{17569, 24},
{17571, 16},
{17573, 8},
{17577, 0},
{17608, 152},
}
_pcsp__b64encode = [][2]uint32{
{1, 0},
{4, 8},
{6, 16},
{8, 24},
{10, 32},
{852, 40},
{853, 32},
{855, 24},
{857, 16},
{859, 8},
{864, 0},
}
)
var funcs = []loader.CFunc{
{"__native_entry__", 0, 67, 0, nil},
{"_b64decode", _entry__b64decode, _size__b64decode, _stack__b64decode, _pcsp__b64decode},
{"_b64encode", _entry__b64encode, _size__b64encode, _stack__b64encode, _pcsp__b64encode},
}

76
vendor/github.com/gabriel-vasile/mimetype/CODE_OF_CONDUCT.md generated vendored
View File

@@ -1,76 +0,0 @@
# Contributor Covenant Code of Conduct
## Our Pledge
In the interest of fostering an open and welcoming environment, we as
contributors and maintainers pledge to making participation in our project and
our community a harassment-free experience for everyone, regardless of age, body
size, disability, ethnicity, sex characteristics, gender identity and expression,
level of experience, education, socio-economic status, nationality, personal
appearance, race, religion, or sexual identity and orientation.
## Our Standards
Examples of behavior that contributes to creating a positive environment
include:
* Using welcoming and inclusive language
* Being respectful of differing viewpoints and experiences
* Gracefully accepting constructive criticism
* Focusing on what is best for the community
* Showing empathy towards other community members
Examples of unacceptable behavior by participants include:
* The use of sexualized language or imagery and unwelcome sexual attention or
advances
* Trolling, insulting/derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or electronic
address, without explicit permission
* Other conduct which could reasonably be considered inappropriate in a
professional setting
## Our Responsibilities
Project maintainers are responsible for clarifying the standards of acceptable
behavior and are expected to take appropriate and fair corrective action in
response to any instances of unacceptable behavior.
Project maintainers have the right and responsibility to remove, edit, or
reject comments, commits, code, wiki edits, issues, and other contributions
that are not aligned to this Code of Conduct, or to ban temporarily or
permanently any contributor for other behaviors that they deem inappropriate,
threatening, offensive, or harmful.
## Scope
This Code of Conduct applies both within project spaces and in public spaces
when an individual is representing the project or its community. Examples of
representing a project or community include using an official project e-mail
address, posting via an official social media account, or acting as an appointed
representative at an online or offline event. Representation of a project may be
further defined and clarified by project maintainers.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be
reported by contacting the project team at vasile.gabriel@email.com. All
complaints will be reviewed and investigated and will result in a response that
is deemed necessary and appropriate to the circumstances. The project team is
obligated to maintain confidentiality with regard to the reporter of an incident.
Further details of specific enforcement policies may be posted separately.
Project maintainers who do not follow or enforce the Code of Conduct in good
faith may face temporary or permanent repercussions as determined by other
members of the project's leadership.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4,
available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html
[homepage]: https://www.contributor-covenant.org
For answers to common questions about this code of conduct, see
https://www.contributor-covenant.org/faq

12
vendor/github.com/gabriel-vasile/mimetype/CONTRIBUTING.md generated vendored
View File

@@ -1,12 +0,0 @@
## Contribute
Contributions to **mimetype** are welcome. If you find an issue and you consider
contributing, you can use the [Github issues tracker](https://github.com/gabriel-vasile/mimetype/issues)
in order to report it, or better yet, open a pull request.
Code contributions must respect these rules:
- code must be test covered
- code must be formatted using gofmt tool
- exported names must be documented
**Important**: By submitting a pull request, you agree to allow the project
owner to license your work under the same license as that used by the project.

2
vendor/github.com/gabriel-vasile/mimetype/LICENSE generated vendored
View File

@@ -1,6 +1,6 @@
MIT License MIT License
Copyright (c) 2018-2020 Gabriel Vasile Copyright (c) 2018 Gabriel Vasile
Permission is hereby granted, free of charge, to any person obtaining a copy Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal of this software and associated documentation files (the "Software"), to deal

43
vendor/github.com/gabriel-vasile/mimetype/README.md generated vendored
View File

@@ -10,18 +10,12 @@
</h6> </h6>
<p align="center"> <p align="center">
<a href="https://travis-ci.org/gabriel-vasile/mimetype">
<img alt="Build Status" src="https://travis-ci.org/gabriel-vasile/mimetype.svg?branch=master">
</a>
<a href="https://pkg.go.dev/github.com/gabriel-vasile/mimetype"> <a href="https://pkg.go.dev/github.com/gabriel-vasile/mimetype">
<img alt="Go Reference" src="https://pkg.go.dev/badge/github.com/gabriel-vasile/mimetype.svg"> <img alt="Go Reference" src="https://pkg.go.dev/badge/github.com/gabriel-vasile/mimetype.svg">
</a> </a>
<a href="https://goreportcard.com/report/github.com/gabriel-vasile/mimetype"> <a href="https://goreportcard.com/report/github.com/gabriel-vasile/mimetype">
<img alt="Go report card" src="https://goreportcard.com/badge/github.com/gabriel-vasile/mimetype"> <img alt="Go report card" src="https://goreportcard.com/badge/github.com/gabriel-vasile/mimetype">
</a> </a>
<a href="https://codecov.io/gh/gabriel-vasile/mimetype">
<img alt="Code coverage" src="https://codecov.io/gh/gabriel-vasile/mimetype/branch/master/graph/badge.svg?token=qcfJF1kkl2"/>
</a>
<a href="LICENSE"> <a href="LICENSE">
<img alt="License" src="https://img.shields.io/badge/License-MIT-green.svg"> <img alt="License" src="https://img.shields.io/badge/License-MIT-green.svg">
</a> </a>
@@ -33,6 +27,7 @@
- possibility to [extend](https://pkg.go.dev/github.com/gabriel-vasile/mimetype#example-package-Extend) with other file formats - possibility to [extend](https://pkg.go.dev/github.com/gabriel-vasile/mimetype#example-package-Extend) with other file formats
- common file formats are prioritized - common file formats are prioritized
- [text vs. binary files differentiation](https://pkg.go.dev/github.com/gabriel-vasile/mimetype#example-package-TextVsBinary) - [text vs. binary files differentiation](https://pkg.go.dev/github.com/gabriel-vasile/mimetype#example-package-TextVsBinary)
- no external dependencies
- safe for concurrent usage - safe for concurrent usage
## Install ## Install
@@ -51,8 +46,7 @@ fmt.Println(mtype.String(), mtype.Extension())
``` ```
See the [runnable Go Playground examples](https://pkg.go.dev/github.com/gabriel-vasile/mimetype#pkg-overview). See the [runnable Go Playground examples](https://pkg.go.dev/github.com/gabriel-vasile/mimetype#pkg-overview).
## Usage' Caution: only use libraries like **mimetype** as a last resort. Content type detection
Only use libraries like **mimetype** as a last resort. Content type detection
using magic numbers is slow, inaccurate, and non-standard. Most of the times using magic numbers is slow, inaccurate, and non-standard. Most of the times
protocols have methods for specifying such metadata; e.g., `Content-Type` header protocols have methods for specifying such metadata; e.g., `Content-Type` header
in HTTP and SMTP. in HTTP and SMTP.
@@ -73,6 +67,18 @@ mimetype.DetectFile("file.doc")
If increasing the limit does not help, please If increasing the limit does not help, please
[open an issue](https://github.com/gabriel-vasile/mimetype/issues/new?assignees=&labels=&template=mismatched-mime-type-detected.md&title=). [open an issue](https://github.com/gabriel-vasile/mimetype/issues/new?assignees=&labels=&template=mismatched-mime-type-detected.md&title=).
## Tests
In addition to unit tests,
[mimetype_tests](https://github.com/gabriel-vasile/mimetype_tests) compares the
library with the [Unix file utility](https://en.wikipedia.org/wiki/File_(command))
for around 50 000 sample files. Check the latest comparison results
[here](https://github.com/gabriel-vasile/mimetype_tests/actions).
## Benchmarks
Benchmarks for each file format are performed when a PR is open. The results can
be seen on the [workflows page](https://github.com/gabriel-vasile/mimetype/actions/workflows/benchmark.yml).
Performance improvements are welcome but correctness is prioritized.
## Structure ## Structure
**mimetype** uses a hierarchical structure to keep the MIME type detection logic. **mimetype** uses a hierarchical structure to keep the MIME type detection logic.
This reduces the number of calls needed for detecting the file type. The reason This reduces the number of calls needed for detecting the file type. The reason
@@ -87,22 +93,11 @@ To prevent loading entire files into memory, when detecting from a
or from a [file](https://pkg.go.dev/github.com/gabriel-vasile/mimetype#DetectFile) or from a [file](https://pkg.go.dev/github.com/gabriel-vasile/mimetype#DetectFile)
**mimetype** limits itself to reading only the header of the input. **mimetype** limits itself to reading only the header of the input.
<div align="center"> <div align="center">
<img alt="structure" src="https://github.com/gabriel-vasile/mimetype/blob/420a05228c6a6efbb6e6f080168a25663414ff36/mimetype.gif?raw=true" width="88%"> <img alt="how project is structured" src="https://raw.githubusercontent.com/gabriel-vasile/mimetype/master/testdata/gif.gif" width="88%">
</div> </div>
## Performance
Thanks to the hierarchical structure, searching for common formats first,
and limiting itself to file headers, **mimetype** matches the performance of
stdlib `http.DetectContentType` while outperforming the alternative package.
```bash
mimetype http.DetectContentType filetype
BenchmarkMatchTar-24 250 ns/op 400 ns/op 3778 ns/op
BenchmarkMatchZip-24 524 ns/op 351 ns/op 4884 ns/op
BenchmarkMatchJpeg-24 103 ns/op 228 ns/op 839 ns/op
BenchmarkMatchGif-24 139 ns/op 202 ns/op 751 ns/op
BenchmarkMatchPng-24 165 ns/op 221 ns/op 1176 ns/op
```
## Contributing ## Contributing
See [CONTRIBUTING.md](CONTRIBUTING.md). Contributions are unexpected but welcome. When submitting a PR for detection of
a new file format, please make sure to add a record to the list of testcases
from [mimetype_test.go](mimetype_test.go). For complex files a record can be added
in the [testdata](testdata) directory.

Some files were not shown because too many files have changed in this diff Show More