A great feature of pydantic are the validation hooks that let you intercept serialization/deserialization of specific fields and augment behavior.
For example if you are querying a DB that returns a column as a JSON string, trivial with Pydantic to json parse the column are part of deser with an annotation.
Pydantic is definitely slower and not a 'zero cost abstraction', but you do get a lot for it.
Comments
Or just dump pydantic and use msgspec instead: https://jcristharif.com/msgspec/
A great feature of pydantic are the validation hooks that let you intercept serialization/deserialization of specific fields and augment behavior.
For example if you are querying a DB that returns a column as a JSON string, trivial with Pydantic to json parse the column are part of deser with an annotation.
Pydantic is definitely slower and not a 'zero cost abstraction', but you do get a lot for it.
One approach to do that in msgspec is described here https://github.com/jcrist/msgspec/issues/375#issuecomment-15...
msgspec is much more memory efficient out of the box, yes. Also quite fast.
Can it do incremental parsing? Cant tell from a brief look.
IIUC:
* You still need to load all the bytes into memory before passing to msgspec decoding
* You can decode a subset of fields, which is really helpful
* Reusing msgspec decoders saves some cpu cycles https://jcristharif.com/msgspec/perf-tips.html#reuse-encoder...
Slides 17, 18, 19 have an example of the first two points https://pythonspeed.com/pycon2025/slides/#17