I really do think it has to do with grep swapping out regex implementations based on features needed.
That wouldn't explain why one regex engine is 5x faster than another. Only looking at the regex engines themselves would tell you that.
Complaining that Perl uses a regex and it isn't Perl
I'm not complaining that it uses a regex, I'm complaining that it doesn't do anything else.
A representative Perl program would use regexes and contain some logic that processes the results of those regexes.
I think it's more related to the fact that the actions of the regex parsing implementation when optimized sufficiently is very close in implementation to C code that steps through a char array
I used to think that, but it is really not true unless your regex engine contains a JIT compiler.
Specialized machine code for a text parser (which is what you would get from writing C) is significantly faster than generic NFA/DFA code. In these tests, an average of 65% of runtime was saved when the regex engine included a JIT (ie. the specialized code was over twice as fast): http://sljit.sourceforge.net/pcre.html
That wouldn't explain why one regex engine is 5x faster than another. Only looking at the regex engines themselves would tell you that.
It could definitely explain it, but it may not be the best explanation given the facts. I'll definitely concede that it's pure conjecture.
A representative Perl program would use regexes and contain some logic that processes the results of those regexes.
Sure, depending on what you want to show. Nobody is trying to say Perl is as fast or faster than C, just that relatively, it's fast for the development cost it requires.
> I think it's more related to the fact that the actions of the regex parsing implementation when optimized sufficiently is very close in implementation to C code that steps through a char array
> I used to think that, but it is really not true unless your regex engine contains a JIT compiler.
I think we're referring to different things, which is mostly my fault for being loose with my terminology. I really only meant close to C in a conceptual manner, which yields some performance benefit by keeping a large chunk of the looping and work storing specific chunks of text low level and in the interpreter. I wasn't trying to imply the regex engine's cost was negligible or the actual machine operations we comparable in a large way.
> regex parsing implementation when optimized sufficiently is very close in implementation to C code
I used to think that, but it is really not true unless your regex engine contains a JIT compiler.
The P6 Rules engine is written in NQP so it gets JIT'd on the JVM and MoarVM backends.
Of course it'll be many years before the engine is seriously optimized but it's a good start.
It'll be interesting to see if the code gen of this next gen regexen engine gets good enough in 2015 to make its advantages (most notably the grapheme-by-default design) actually pay dividends.
Comments
That wouldn't explain why one regex engine is 5x faster than another. Only looking at the regex engines themselves would tell you that.
I'm not complaining that it uses a regex, I'm complaining that it doesn't do anything else.
A representative Perl program would use regexes and contain some logic that processes the results of those regexes.
I used to think that, but it is really not true unless your regex engine contains a JIT compiler.
Specialized machine code for a text parser (which is what you would get from writing C) is significantly faster than generic NFA/DFA code. In these tests, an average of 65% of runtime was saved when the regex engine included a JIT (ie. the specialized code was over twice as fast): http://sljit.sourceforge.net/pcre.html
It could definitely explain it, but it may not be the best explanation given the facts. I'll definitely concede that it's pure conjecture.
Sure, depending on what you want to show. Nobody is trying to say Perl is as fast or faster than C, just that relatively, it's fast for the development cost it requires.
I think we're referring to different things, which is mostly my fault for being loose with my terminology. I really only meant close to C in a conceptual manner, which yields some performance benefit by keeping a large chunk of the looping and work storing specific chunks of text low level and in the interpreter. I wasn't trying to imply the regex engine's cost was negligible or the actual machine operations we comparable in a large way.
The P6 Rules engine is written in NQP so it gets JIT'd on the JVM and MoarVM backends.
Of course it'll be many years before the engine is seriously optimized but it's a good start.
It'll be interesting to see if the code gen of this next gen regexen engine gets good enough in 2015 to make its advantages (most notably the grapheme-by-default design) actually pay dividends.