Comment on Loading Pydantic models from JSON without running out of memoryparentComments−aitchnyu1yCan it do incremental parsing? Cant tell from a brief look.−jtmcivor1yIIUC:* You still need to load all the bytes into memory before passing to msgspec decoding* You can decode a subset of fields, which is really helpful* Reusing msgspec decoders saves some cpu cycles https://jcristharif.com/msgspec/perf-tips.html#reuse-encoder...Slides 17, 18, 19 have an example of the first two points https://pythonspeed.com/pycon2025/slides/#17
Comments
Can it do incremental parsing? Cant tell from a brief look.
IIUC:
* You still need to load all the bytes into memory before passing to msgspec decoding
* You can decode a subset of fields, which is really helpful
* Reusing msgspec decoders saves some cpu cycles https://jcristharif.com/msgspec/perf-tips.html#reuse-encoder...
Slides 17, 18, 19 have an example of the first two points https://pythonspeed.com/pycon2025/slides/#17