Commit Graph
53 Commits
Author SHA1 Message Date
Denis Gregor e222189360 Check the whole RFC 8259 number grammar in strict mode
Strict mode rejected leading zeros but nothing else about the shape of a
number, so several malformed numbers parsed and were then written back out
verbatim, producing JSON that other parsers reject:

    $ echo '[2.e3]' | ./json_parse -s -
    [ 2.e3 ]

RFC 8259 requires a mandatory integer part and at least one digit in both the
fraction and the exponent:

    number = [ minus ] int [ frac ] [ exp ]
    int    = zero / ( digit1-9 *DIGIT )
    frac   = decimal-point 1*DIGIT
    exp    = e [ minus / plus ] 1*DIGIT

so "1.", "-2.", "2.e3", "2.e+3", "0.e1", "-.123", "1e" and "1e+" are all
invalid. Walk the accumulated text against that grammar instead. The leading
zero rule from the previous check is part of the same walk rather than a
separate test, and its cases keep their coverage.

Only JSON_TOKENER_STRICT is affected; the default tokener stays as lenient as
it was. Measured against JSONTestSuite (318 files): strict mode went from 44
to 37 files accepted that the suite says must be rejected, default mode stayed
at 71, and nothing that must be accepted regressed in either mode.
2026-08-16 20:26:15 +03:00
Javid Khan ff18c75286 reject leading zeros in strict-mode number parsing 2026-07-28 13:27:25 +05:30
Shane F. Carr dbefb48948 Add tests with current behavior 2025-07-30 17:39:47 -07:00
Eric Hawicz 7cee5237dc Issue #867 - also disallow control characters in keys 2025-04-03 21:16:29 -04:00
Eric Hawicz 565f181f65 Fix issue #875: cast to unsigned char so bytes above 0x7f aren't interpreted as negative, which was causing the strict-mode control characters check to incorrectly trigger. 2024-11-08 22:20:40 -05:00
Eric Hawicz 6bfab90c87 Issue #867: disallow control characters in strict mode. 2024-09-15 10:37:45 -04:00
Eric Haszlakiewicz d6f46ae104 Explicitly check for integer overflow/underflow when parsing integers with JSON_TOKENER_STRICT. 2022-10-30 19:39:30 +00:00
Khem Raj 6eca65617a Fix build with clang-15+
Fixes
json_util.c:63:35: error: a function declaration without a prototype is deprecated in all versions of C [-We
rror,-Wstrict-prototypes]
const char *json_util_get_last_err()
                                  ^
                                   void

Signed-off-by: Khem Raj <raj.khem@gmail.com>
2022-08-13 20:47:20 -07:00
Eric Haszlakiewicz bdd5e03d6e Apply some of the fixes from PR #740, although by using size_t instead of castings. 2022-07-31 19:28:48 +00:00
Eric Haszlakiewicz 49c3721a5a Unset NDEBUG so assert() is enabled in all tests. One more fix there when building with -DCMAKE_BUILD_TYPE=release. 2022-05-30 15:39:54 +00:00
Eric Haszlakiewicz 6ba1adf8ef Include the tests and apps directories at the _end_, since order in the
cmake rules matters, and fix compile warnings now that we're building
those sources with all the regular flags.
2022-05-30 14:33:16 +00:00
Eric Haszlakiewicz a4e3700972 Fix code formatting 2020-06-29 02:31:18 +00:00
Eric Haszlakiewicz 5f3bf70f03 Make sure TEST_PARSE_CHUNKSIZE is valid if it's set. 2020-06-29 02:18:34 +00:00
Eric Haszlakiewicz f23486a321 In the json_tokener_state_number case, explicitly adjust what "number" characters are allowed based on the exact micro-state that we're in, and check for invalid following characters in a different way, to allow a valid json_type_number object to be returned at the top level.
This causes previously failing strings like "123-456" to return a valid json_object with the appropriate value.  If you care about the trailing content, call json_tokener_parse_ex() and check the parse end point with json_tokener_get_parse_end().
2020-06-29 02:14:26 +00:00
Eric Haszlakiewicz 6eac6986c9 Fix incremental parsing of invalid numbers with exponents, such as "0e+-" and "12.3E12E12", while still allowing "0e+" in non-strict mode.
Deprecate the json_parse_double() function from json_util.h
2020-06-27 15:35:04 +00:00
Eric Haszlakiewicz 7a72805e34 Fix memory leak in test_parse's single_incremental_parse(). 2020-06-22 01:12:03 +00:00
Eric Haszlakiewicz a68566bf6a Issue #616: Change the parsing of surrogate pairs in unicode escapes so it uses a couple of additional states instead of assuming the low surrogate is already present, to ensure that we correctly handle various cases of incremental parsing. 2020-06-21 18:29:57 +00:00
Eric Haszlakiewicz 197e372464 In test_parse, fix lengths passed during a couple of incremental tests.
Add an optional way to use an incremental chunk size ($TEST_PARSE_CHUNKSIZE)
for all of the single_basic_parse tests, in additional to the regular way.
2020-06-21 17:36:38 +00:00
dota17 c1b872d817 fix issue 616: support the surrogate pair in split file. 2020-06-08 17:19:32 +08:00
David McCann add7b13a9a Improved support for IBM operating systems
Fix compiler errors and warnings when building on IBM operating systems such as AIX and IBM i.
2020-05-14 15:39:35 +01:00
Eric Haszlakiewicz f6f76f9430 Add a JSON_TOKENER_ALLOW_TRAILING_CHARS flag for json_tokener_set_flags() to allow multiple objects to be parsed from input even when JSON_TOKENER_STRICT is set. 2020-04-21 03:53:44 +00:00
Eric Haszlakiewicz 05623b3a2e Add an tok_flags field to explicitly specify tokener flags in test_parse and eliminate the previous bogus calls to json_tokener_set_flags() 2020-04-21 03:53:44 +00:00
dota17 8b162c4b89 clang-format the files 2020-04-03 11:39:30 +08:00
dota17 c117d8a8a8 add the disabling formatting coments and adjust the partial code manuly 2020-04-03 11:28:04 +08:00
dota17 3c3b5920f7 add uint64 data to json-c 2020-02-25 14:51:35 +08:00
Eric Haszlakiewicz 0ffdbb2395 Merge pull request #531 from dota17/utf8test
validate utf-8 string
2020-02-05 21:56:06 -05:00
dota17 787a8b3f1c update code 2020-01-20 10:41:24 +08:00
Po-Chuan Hsieh bb5971ba2a Fix make check
cc -DHAVE_CONFIG_H -I. -I..  -I.. -I../tests    -O2 -pipe  -fstack-protector-strong -fno-strict-aliasing  -Wall -Werror -Wcast-qual -Wno-error=deprecated-declarations -Wextra -Wno-unused-parameter -D_GNU_SOURCE -D_REENTRANT -MT test_parse.o -MD -MP -MF .deps/test_parse.Tpo -c -o test_parse.o test_parse.c
test_parse.c:256:14: error: adding 'int' to a string does not append to the string [-Werror,-Wstring-plus-int]
        { "null123" + 4,       4, 3, json_tokener_success, 1 },
          ~~~~~~~~~~^~~
test_parse.c:256:14: note: use array indexing to silence this warning
        { "null123" + 4,       4, 3, json_tokener_success, 1 },
                    ^
          &         [  ]
test_parse.c:258:12: error: adding 'int' to a string does not append to the string [-Werror,-Wstring-plus-int]
        { "nullx" + 4,         2, 0, json_tokener_error_parse_unexpected, 1 },
          ~~~~~~~~^~~
test_parse.c:258:12: note: use array indexing to silence this warning
        { "nullx" + 4,         2, 0, json_tokener_error_parse_unexpected, 1 },
                  ^
          &       [  ]
test_parse.c:260:25: error: adding 'int' to a string does not append to the string [-Werror,-Wstring-plus-int]
        { "{\"a\":1}{\"b\":2}" + 7,
          ~~~~~~~~~~~~~~~~~~~~~^~~
test_parse.c:260:25: note: use array indexing to silence this warning
        { "{\"a\":1}{\"b\":2}" + 7,
                               ^
          &                    [  ]
3 errors generated.
*** Error code 1

Stop.
2020-01-11 07:20:44 +00:00
dota17 eca74dcccf test utf8 2020-01-10 18:33:14 +08:00
dota17 5fe4448541 update tsetcase for tokener_c 2020-01-02 19:19:12 +08:00
chenguoping 1446572997 add testcases of object and token 2019-12-27 15:07:00 +08:00
Eric Haszlakiewicz 25aedddcdf Fix memory leaks in test_double_serializer, and make sure all tests return 0 at the end of main(). 2019-11-23 12:05:28 -05:00
Eric Haszlakiewicz 05b41b159e Add a json_tokener_get_parse_end() function to replace direct access of tok->char_offset. 2019-09-08 21:35:37 -04:00
Eric Haszlakiewicz da4b34355d Add an parse test for content starting with a UTF8 BOM, which is _not_ a valid start to a JSON message. 2018-03-25 18:23:42 -04:00
Eric Haszlakiewicz 11ab365324 Deprecated a few typedefs, add docs on json_object_iter, move json_number_chars and json_hex-chars to json_object_private.h. 2017-12-05 09:20:59 -05:00
Eric Haszlakiewicz 31a6c2fac8 Add a few more tests to the partial parse to cover bytes after "Infinity". 2017-11-27 00:18:55 -05:00
Eric Haszlakiewicz 48f7e389fb Fix parsing of "-Infinity" when the "-" and "Infinity" are split across multiple calls.
Add several additional test cases for partial parsing of infinity values.
2017-11-27 00:04:33 -05:00
Eric Haszlakiewicz 4d1e0b4409 Issue #371: fix parsing of "-Infinity" (although in a somewhat different location than PR#372 used), and add a case to test_parse to check for this. 2017-11-26 22:44:39 -05:00
Eric Haszlakiewicz 2a22858fe7 Add a few extra cases to test_parse to show how trailing bytes are handled, especially for parsing "Infinity". 2017-11-26 22:34:43 -05:00
Eric Haszlakiewicz b8738dd623 Rewrite test_basic_parse() to factor out a single_basic_parse() to avoid lots of duplicate boiler plate code.
Also, emit the input string in the output so it's easier to see what's going on.
2017-11-26 22:33:36 -05:00
Eric Haszlakiewicz 8c86207258 Add a few more cases to test_parse to provide some examples of how parsing works; should help address the questions raised in Issue #302. 2017-02-26 16:22:45 +00:00
Eric Haszlakiewicz 33339ae595 Use json_visit() to clear the serializer data set when parsing so the output from test_parse reflects the actual values parsed. 2016-10-29 15:13:16 -04:00
Eric Haszlakiewicz 36a28fcb0c Issue #275: fix out of bounds read when handling unicode surrogate pairs. 2016-10-05 23:15:51 -04:00
Nicola Spanti (RyDroid) 8215c0ac0e Very minor changes to some tests 2016-08-13 00:53:33 +02:00
Eric Haszlakiewicz 316da85818 Fix issue #201: add a JSON_C_TO_STRING_NOSLASHESCAPE flag to turn off escaping of forward slashes. 2015-11-28 20:00:30 -06:00
chl 99d8fc975e Tightening the number parsing algorithm
Some badly formated "numbers" could get partly parsed,
resulting in truncated results instead of raising an
error.

Examples :
 '1.2.3'      -> (double)1.2
 '2015-01-15' -> (int)2015

This patch is not perfect (ex: input can still end with a 'E', which
is forbidden by json.org doc) but should avoid non-sensically
formated input.

Tests added.
2015-02-05 01:50:37 +01:00
Eric Haszlakiewicz 05da316b9c Issue #103: allow Infinity and -Infinity to be parsed. 2014-03-22 17:28:40 -04:00
Eric Haszlakiewicz 0eedf3802f Issue#102 - add support for parsing "NaN". 2014-03-09 16:41:33 -04:00
Eric Haszlakiewicz e8161a11bb Issue #15: add a way to set a JSON_TOKENER_STRICT flag to forbid commas at the end of arrays and objects. 2013-03-31 20:05:36 -05:00
Eric Haszlakiewicz ca8b27d183 Enable -Werror and fix a number of minor warnings that existed. 2013-02-09 16:35:24 -06:00