This is superb. Thank you for making it and licensing it MIT. I think this is a contender to replace the lexer within jank. I'll do some benchmarking next year and we'll see!
It looks like the key missing part which would be needed for a lexer is source information (bare minimum: byte offset and size). I don't think edn.c can be used as a lexer without that, since error reporting requires accurate source information.
As a side note, I'm curious how much AI was used in the creation of edn.c. These days, I like to get a measure of that for every library I use.
It should be easy to add source info for every token, some of them already keep both (size and offset) I can create a branch for that.
I'm curious how much AI was used in the creation of edn.c
A fair amount. This is my first big public project written in pure C. I did consult LLM about best practices for code organisation, memory management, difference in SIMD instructions between platforms, etc. All the things Clojure developer typically don't think about (luxury of a hosted language). Ultimately, the goal was to learn some part of C programming, working reader is a side effect of that.
These days, I like to get a measure of that for every library I use.
Btw, I'm curious, what kind of measuring you are looking for?
Comments
This is superb. Thank you for making it and licensing it MIT. I think this is a contender to replace the lexer within jank. I'll do some benchmarking next year and we'll see!
Wow, that is a greate news!) Thanks for looking at it from this perspective! There are some benchmarks already available in the project - https://github.com/DotFox/edn.c/blob/main/bench/bench_integr...
you can run it locally with `make bench bench-clj bench-wasm`
Let me know if I can do anything to help you with support in jank.
It looks like the key missing part which would be needed for a lexer is source information (bare minimum: byte offset and size). I don't think edn.c can be used as a lexer without that, since error reporting requires accurate source information.
As a side note, I'm curious how much AI was used in the creation of edn.c. These days, I like to get a measure of that for every library I use.
It should be easy to add source info for every token, some of them already keep both (size and offset) I can create a branch for that.
A fair amount. This is my first big public project written in pure C. I did consult LLM about best practices for code organisation, memory management, difference in SIMD instructions between platforms, etc. All the things Clojure developer typically don't think about (luxury of a hosted language). Ultimately, the goal was to learn some part of C programming, working reader is a side effect of that.
Btw, I'm curious, what kind of measuring you are looking for?
Oooo that’d be nice.