I don't think using a language model to generate inputs directly is ever going to be as efficient as writing a little bit of code to do the generation; it's really hard to beat an input generator that can craft thousands of inputs/second.
For one, it'd be really hard for an LLM to get the CRC32 right, especially when it's in a header before the data it covers.
Then again, this whole approach to fuzzing comes across as kinda naive, at the very least you'd want to use an API of a coverage-guided fuzzer for generating the randomness (and then almost always fixing up CRC32 on top of that, like a human-written wrapper function would).
Exactly. If I actually wanted to fuzz this I'd use libfuzzer and manually fix the crc32. An LLM would be useful in helping me write the libfuzzer glue code.
Comments
I don't understand why we are getting LLMs to generate code to create fuzzing data as a 'thing'
Logically LLMs should be quite good at creating the fuzzing data.
To state the obvious why, it's too expensive to use LLMs directly and this way works since they found "4 memory safety bugs and one hang"
But the future we are heading to should be LLMs will directly pentest/test the code. This is where it's interesting and new.
I don't think using a language model to generate inputs directly is ever going to be as efficient as writing a little bit of code to do the generation; it's really hard to beat an input generator that can craft thousands of inputs/second.
For one, it'd be really hard for an LLM to get the CRC32 right, especially when it's in a header before the data it covers.
Then again, this whole approach to fuzzing comes across as kinda naive, at the very least you'd want to use an API of a coverage-guided fuzzer for generating the randomness (and then almost always fixing up CRC32 on top of that, like a human-written wrapper function would).
Exactly. If I actually wanted to fuzz this I'd use libfuzzer and manually fix the crc32. An LLM would be useful in helping me write the libfuzzer glue code.