You realize your criticism of D has nothing to do with the language itself, and everything to do with network effects, and first-to-market effect? Of course D lacks the widespread adoptions of C++, it's a newer language! The same goes for your criticism of Rust.
If I were moved by these objections, I would conclude that we shouldn't even try to overthrow established programming languages. Heck why not embrace Status-quo Bias altogether?
About Java & co, you need to realized that most C++ code out there didn't need C++ (or D, or Rust) in the first place. The projects that are (i) very demanding to the CPU and memory, (ii) need a complex solution, and (iii) can't possibly separate a few bottlenecks in C from a high level language with a good FFI (Python, OCaml, Lua, Haskell, Lisp…), are uncommon. Some of those projects have incredible impact, but they are still uncommon.
Java & co are for projects where it was a mistake to use C++ in the first place. They don't compete with C++ on its own turf, they just take away territory C++ should never have controlled.
No, D's lack of adoption does involve the language itself to some degree. While the core language itself is better than C++ in some ways, these benefits just aren't compelling enough to exceed the downsides of D, nor do they outweigh the benefits that C++ offers.
And D isn't a new language, by any means. It has been out there for well over a decade now. That should be more than enough time for it to achieve relatively widespread usage, were it to actually offer sufficient benefit over C++ and its other competitors.
I don't know D, so I wouldn't know about it's downsides, besides the one you have spoken of already: lack of adoption. What are the downsides of D that wouldn't go away if it was widely adopted?
You say D is sufficiently old, that it's lack of adoption is significant evidence that we should never adopt it. The problem is, most people think like you do: they wouldn't adopt a language before it has some visible traction. The corollary is, new languages never get widely adopted, unless of course some Big Corporation does massive publicity around it (Java, C#…). The real reason D isn't widely adopted, is not because it's not better than C++. It's because it was not widely adopted in the first place. Textbook chicken and egg.
You speak of benefits and downsides of a language, but what you are really speaking of is the whole ecosystem as it is now. You don't even look at the long term potential.
Also, put yourself in the shoes of a language designer. Say you have a brilliant programming language. Experimental quantitative research has shown with no ambiguity that your language is better than some established, old language in its own niche (20% more productivity, 10% less post-release bugs, same observed runtime speed and memory consumption…). Just one little problem: your language have zero users, and only a little standard library. How would you get people to adopt your new language?
Hint: people won't believe your peer-reviewed experiments. People will refuse to learn a new syntax. People will want to reuse their old code. Even if you come up with an automatic translation tool, people will not trust it. People will say the benefits of the language are outmatched by the drawbacks of its standard library (too small), and of its community (none to speak of). People will be afraid of the required training (even if your research shows productivity benefits after a couple weeks).
How do you fight Status-quo Bias?
Personally, I found only one answer, which unfortunately isn't applicable everywhere: making the new language's implementation so cheap that all it's costs are paid off in a single project. DSLs may do that: with the right tools, a small external DSL can take no more than a couple hundred lines to implement.
You realize your criticism of D has nothing to do with the language itself, and everything to do with network effects, and first-to-market effect? Of course D lacks the widespread adoptions of C++, it's a newer language! The same goes for your criticism of Rust.
Yeah, it's basically a catch-22: lang X is not a viable substitute because it isn't mature enough/not mature enough ecosystem, and continually and relentlessly stating and reinforcing this point will just keep discouraging people from adopting (and indirectly improving) these languages, since clearly... there haven't been any languages that have been able to displace C/++, so what's the point.
And, according to Pacabel, even if you manage to break through some niche with these languages, most underlying software (OSs, VMs, etc.) will still be written in C/++. So if you reimplement the whole bottom of the stack with other systems languages, only then can you talk about displacement, apparently. A tall order.
I have an easy counter to Pacabel's argument: Apple used to use PowerPC. Now they're on X86. People used to use 16bits, then 32bits CPUs. Now they use 64bits CPUs. Operating systems have followed, and soon, programs will as well.
My point is, if you can abstract the underlying architecture, you can replace it. It's difficult because most complex abstractions leak, but it has been done in the past. That Javascript virtual machine written in C++ could be re-written in Rust.
So, even if displacement means "you are not even allowed to use C/C++ as a target language", and "you are not even allowed to use OS services written in C/C++", we can still "displace" C and C++. It won't happen any time soon however. I believe we need to wait for something like mainstream FPGAs.
The problem with current processors and C/C++ is, they tend to be optimized for each other. If instead we had something like an FPGA running at desktop speed, programs would be compiled to logic gates instead of being compiled to assembly. This would dig a grave for C++.
Comments
You realize your criticism of D has nothing to do with the language itself, and everything to do with network effects, and first-to-market effect? Of course D lacks the widespread adoptions of C++, it's a newer language! The same goes for your criticism of Rust.
If I were moved by these objections, I would conclude that we shouldn't even try to overthrow established programming languages. Heck why not embrace Status-quo Bias altogether?
About Java & co, you need to realized that most C++ code out there didn't need C++ (or D, or Rust) in the first place. The projects that are (i) very demanding to the CPU and memory, (ii) need a complex solution, and (iii) can't possibly separate a few bottlenecks in C from a high level language with a good FFI (Python, OCaml, Lua, Haskell, Lisp…), are uncommon. Some of those projects have incredible impact, but they are still uncommon.
Java & co are for projects where it was a mistake to use C++ in the first place. They don't compete with C++ on its own turf, they just take away territory C++ should never have controlled.
No, D's lack of adoption does involve the language itself to some degree. While the core language itself is better than C++ in some ways, these benefits just aren't compelling enough to exceed the downsides of D, nor do they outweigh the benefits that C++ offers.
And D isn't a new language, by any means. It has been out there for well over a decade now. That should be more than enough time for it to achieve relatively widespread usage, were it to actually offer sufficient benefit over C++ and its other competitors.
I don't know D, so I wouldn't know about it's downsides, besides the one you have spoken of already: lack of adoption. What are the downsides of D that wouldn't go away if it was widely adopted?
You say D is sufficiently old, that it's lack of adoption is significant evidence that we should never adopt it. The problem is, most people think like you do: they wouldn't adopt a language before it has some visible traction. The corollary is, new languages never get widely adopted, unless of course some Big Corporation does massive publicity around it (Java, C#…). The real reason D isn't widely adopted, is not because it's not better than C++. It's because it was not widely adopted in the first place. Textbook chicken and egg.
You speak of benefits and downsides of a language, but what you are really speaking of is the whole ecosystem as it is now. You don't even look at the long term potential.
Also, put yourself in the shoes of a language designer. Say you have a brilliant programming language. Experimental quantitative research has shown with no ambiguity that your language is better than some established, old language in its own niche (20% more productivity, 10% less post-release bugs, same observed runtime speed and memory consumption…). Just one little problem: your language have zero users, and only a little standard library. How would you get people to adopt your new language?
Hint: people won't believe your peer-reviewed experiments. People will refuse to learn a new syntax. People will want to reuse their old code. Even if you come up with an automatic translation tool, people will not trust it. People will say the benefits of the language are outmatched by the drawbacks of its standard library (too small), and of its community (none to speak of). People will be afraid of the required training (even if your research shows productivity benefits after a couple weeks).
How do you fight Status-quo Bias?
Personally, I found only one answer, which unfortunately isn't applicable everywhere: making the new language's implementation so cheap that all it's costs are paid off in a single project. DSLs may do that: with the right tools, a small external DSL can take no more than a couple hundred lines to implement.
Yeah, it's basically a catch-22: lang X is not a viable substitute because it isn't mature enough/not mature enough ecosystem, and continually and relentlessly stating and reinforcing this point will just keep discouraging people from adopting (and indirectly improving) these languages, since clearly... there haven't been any languages that have been able to displace C/++, so what's the point.
And, according to Pacabel, even if you manage to break through some niche with these languages, most underlying software (OSs, VMs, etc.) will still be written in C/++. So if you reimplement the whole bottom of the stack with other systems languages, only then can you talk about displacement, apparently. A tall order.
I have an easy counter to Pacabel's argument: Apple used to use PowerPC. Now they're on X86. People used to use 16bits, then 32bits CPUs. Now they use 64bits CPUs. Operating systems have followed, and soon, programs will as well.
My point is, if you can abstract the underlying architecture, you can replace it. It's difficult because most complex abstractions leak, but it has been done in the past. That Javascript virtual machine written in C++ could be re-written in Rust.
So, even if displacement means "you are not even allowed to use C/C++ as a target language", and "you are not even allowed to use OS services written in C/C++", we can still "displace" C and C++. It won't happen any time soon however. I believe we need to wait for something like mainstream FPGAs.
The problem with current processors and C/C++ is, they tend to be optimized for each other. If instead we had something like an FPGA running at desktop speed, programs would be compiled to logic gates instead of being compiled to assembly. This would dig a grave for C++.