if you're performance driven you're already not using virtual functions for places where that overhead is unacceptable
Well, that can only be true for some very localized computations. Virtual method calls are the basis for polymorphism (virtual method calls can even be used when implementing functional languages' pattern matching -- the functional form of polymorphism), and polymorphism is the basis for most programming abstractions. If your code is interesting enough, it will have lots of virtual calls. If it doesn't, then you're probably doing something very specific, and C/C++ might be a better option indeed.
Can you expand on how GC would make my life easier in general?
Sure (I guess you mean re concurrent data structures; otherwise it simply saves you the pain of manual memory management). The basic principle of most (all?) lock-free data structures is that multiple threads might be reading a single node of the DS (which may contain stale data) and then try to CAS a new node into the DS. Without GC, it's very hard to determine when all threads have stopped examining a given node so that it can be safely deallocated.
See the following discussions on the difficulty of implementing lock-free data structures in C++ (b/c of lack of GC):
I don't find myself using a lot of virtual functions in C++ mostly because between trying to prefer composition to inheritance and generic programming you can avoid a lot of typical use cases for polymorphism. So we don't need everything to derive from Object in order to have vector<Object>. There are definitely some areas where polymorphism is the most elegant solution/abstraction and the only place you have to be a little bit careful is around your performance bottlenecks. What I mean is that if you're looping through a bunch of objects and calling o->render() the actual virtual call will usually be minor vs. the work of rendering said object. Also all these objects have a different implementation of render - right? So I'm not sure what's to inline (but I will read up your references). If you're AOT optimizing a scenario like this you'd probably want to separate out those objects and process them all in a group based on their type. It'll make your code suckier and less maintainable, you give up your nice abstraction, but faster. This is something a compiler can't do for you upfront because you would be essentially changing your design to get speed.
So honestly the performance of virtual functions in C++ isn't something I've ever seen as a performance issue (which probably means my code isn't very interesting ;-)
I will second your observation. I have often observed (purely anecdotal) that runtime polymorphism gets used gratuitously even when compile time polymorphism would have sufficed. In my C++ code I seem to be reaching for CRTP fairly often and it does its job. In this case there would be little benefit to be had by inlining virtuals. In fact modern compilers do take a stab at devirtualization, but of course they surely cannot be as informed as a runtime system. Without adequate support to detect exhaustiveness, compile time polymorphism does have the drawback that one could forget to account for all the different cases. This is an instance where Ocaml does better. Sure exhaustiveness can fail, but it requires far less discipline to code in a way that it doesnt. Sure, in future I might need to add more cases, but the beauty is that the type system will tell me all the places where I need to handle the newly inserted case. This takes care of majority of the use case of virtual functions. Exhaustiveness checked pattern matching is a killer feature !
Side note, compile time polymorphism ought to be easier to do in D than C++.
I dont have a problem with JIT. In fact the later the system emits actual code the more information it has at its disposal which it can presumably use to emit better code.
What I disagree with, however, is the widespread defense of the practice of emitting crappy byte code with the justification that we will fix the mess at runtime when and if poor codegen becomes a problem. This self inflicted pessimization of moving that starting block behind is what bugs me. For a scripting language I can buy this, a script could potentially start faster. But Java (the current mainstream poster boy of a JITed language) is no scripting language.
When I am compiling the code with optimization I have more time at my disposal, make use of it. I will even help you (the compiler) with profiles collected from previous runs if I think they would be helpful. If that means spending effort optimizing parts of the code that never get run in future, that is fine, I can tolerate that because I have time now. If some of that effort is wasted, fine I can take that hit, sky wont fall. On the other hand when my system is running, I am indeed short of time and very little budget for optimization and analysis, so cannot run very deep optimizations. Not all tasks can afford JIT burn in.
For me a superior way would be to optimize hard AOT, clear all the low hanging fruits. Have JIT hooks that can change code to adapt to runtime characteristics to mop up whatever further benefits that remain.
On the other hand when my system is running, I am indeed short of time and very little budget for optimization and analysis, so cannot run very deep optimizations. Not all tasks can afford JIT burn in.
Well, server side Java apps are meant to run for hours, days or months. They certainly have the time for a few seconds of compilation (the JIT is profile-guided, so it knows where it should spend effort optimizing, so the whole process takes no more than a few seconds, amortized over the application's first couple of minutes). For long-running server side apps, the JIT is just as negligible as AOT compilation (even if you count the time when the application is running uncompiled/unoptimized, which, for Java, may take 2-120 seconds or so). AOT only improves startup time. But AOT does have an impact on short-lived apps (for which Java currently isn't the best choice), and on mobile apps (where you'd rather not spend battery on compilation). Which is why Oracle is now testing a cached JIT for Java 9.
For me a superior way would be to optimize hard AOT, clear all the low hanging fruits. Have JIT hooks that can change code to adapt to runtime characteristics to mop up whatever further benefits that remain.
A cached JIT which would achieve pretty much the same goal.
Comments
Virtual method inlining is the "mother of all optimizations". See:
* http://parleys.com/play/514892260364bc17fc56be1d/chapter6/ab...
* (more advanced) http://www.azulsystems.com/blog/cliff/2011-04-04-fixing-the-...
Well, that can only be true for some very localized computations. Virtual method calls are the basis for polymorphism (virtual method calls can even be used when implementing functional languages' pattern matching -- the functional form of polymorphism), and polymorphism is the basis for most programming abstractions. If your code is interesting enough, it will have lots of virtual calls. If it doesn't, then you're probably doing something very specific, and C/C++ might be a better option indeed.
Sure (I guess you mean re concurrent data structures; otherwise it simply saves you the pain of manual memory management). The basic principle of most (all?) lock-free data structures is that multiple threads might be reading a single node of the DS (which may contain stale data) and then try to CAS a new node into the DS. Without GC, it's very hard to determine when all threads have stopped examining a given node so that it can be safely deallocated.
See the following discussions on the difficulty of implementing lock-free data structures in C++ (b/c of lack of GC):
* http://www.drdobbs.com/lock-free-data-structures/184401865
* https://software.intel.com/en-us/forums/topic/295279
Without a GC, you'd need hazard pointers[1] or RCU[2], both are rife with issues, and are either not general enough or perform worse than a GC.
True.
Right, but if your code isn't concurrent/parallel there's a hard (and low) limit on how fast it can run.
[1]: http://en.wikipedia.org/wiki/Hazard_pointer
[2]: http://en.wikipedia.org/wiki/Read-copy-update
Thanks!
I don't find myself using a lot of virtual functions in C++ mostly because between trying to prefer composition to inheritance and generic programming you can avoid a lot of typical use cases for polymorphism. So we don't need everything to derive from Object in order to have vector<Object>. There are definitely some areas where polymorphism is the most elegant solution/abstraction and the only place you have to be a little bit careful is around your performance bottlenecks. What I mean is that if you're looping through a bunch of objects and calling o->render() the actual virtual call will usually be minor vs. the work of rendering said object. Also all these objects have a different implementation of render - right? So I'm not sure what's to inline (but I will read up your references). If you're AOT optimizing a scenario like this you'd probably want to separate out those objects and process them all in a group based on their type. It'll make your code suckier and less maintainable, you give up your nice abstraction, but faster. This is something a compiler can't do for you upfront because you would be essentially changing your design to get speed.
So honestly the performance of virtual functions in C++ isn't something I've ever seen as a performance issue (which probably means my code isn't very interesting ;-)
I will second your observation. I have often observed (purely anecdotal) that runtime polymorphism gets used gratuitously even when compile time polymorphism would have sufficed. In my C++ code I seem to be reaching for CRTP fairly often and it does its job. In this case there would be little benefit to be had by inlining virtuals. In fact modern compilers do take a stab at devirtualization, but of course they surely cannot be as informed as a runtime system. Without adequate support to detect exhaustiveness, compile time polymorphism does have the drawback that one could forget to account for all the different cases. This is an instance where Ocaml does better. Sure exhaustiveness can fail, but it requires far less discipline to code in a way that it doesnt. Sure, in future I might need to add more cases, but the beauty is that the type system will tell me all the places where I need to handle the newly inserted case. This takes care of majority of the use case of virtual functions. Exhaustiveness checked pattern matching is a killer feature !
Side note, compile time polymorphism ought to be easier to do in D than C++.
I dont have a problem with JIT. In fact the later the system emits actual code the more information it has at its disposal which it can presumably use to emit better code.
What I disagree with, however, is the widespread defense of the practice of emitting crappy byte code with the justification that we will fix the mess at runtime when and if poor codegen becomes a problem. This self inflicted pessimization of moving that starting block behind is what bugs me. For a scripting language I can buy this, a script could potentially start faster. But Java (the current mainstream poster boy of a JITed language) is no scripting language.
When I am compiling the code with optimization I have more time at my disposal, make use of it. I will even help you (the compiler) with profiles collected from previous runs if I think they would be helpful. If that means spending effort optimizing parts of the code that never get run in future, that is fine, I can tolerate that because I have time now. If some of that effort is wasted, fine I can take that hit, sky wont fall. On the other hand when my system is running, I am indeed short of time and very little budget for optimization and analysis, so cannot run very deep optimizations. Not all tasks can afford JIT burn in.
For me a superior way would be to optimize hard AOT, clear all the low hanging fruits. Have JIT hooks that can change code to adapt to runtime characteristics to mop up whatever further benefits that remain.
Well, server side Java apps are meant to run for hours, days or months. They certainly have the time for a few seconds of compilation (the JIT is profile-guided, so it knows where it should spend effort optimizing, so the whole process takes no more than a few seconds, amortized over the application's first couple of minutes). For long-running server side apps, the JIT is just as negligible as AOT compilation (even if you count the time when the application is running uncompiled/unoptimized, which, for Java, may take 2-120 seconds or so). AOT only improves startup time. But AOT does have an impact on short-lived apps (for which Java currently isn't the best choice), and on mobile apps (where you'd rather not spend battery on compilation). Which is why Oracle is now testing a cached JIT for Java 9.
A cached JIT which would achieve pretty much the same goal.