still waiting for more data, and also after read() failures.
Finalize the tokener at EOF with the terminating NUL so valid scalar
values can complete, while rejecting genuinely truncated input.
Also propagate read errors to the caller and add CLI regression tests.
