I mildly disagree - depending on your definition of "typical app". Most applications have much greater use of multi-processing and concurrent cross-thread (or cross-process) communication. Compilers, aside from high-level parallelism across modules, tend to be quite single-threaded applications.
If you're solely interested in single-core performance, then I would agree that they are a good stress test, but I think for a processor that is being sold on it's parallelism, they are not a great benchmark.
In the history of the SPEC benchmarks, the compiler benchmarks, like gcc, have been the best predictor of CPU performance for the applications that cannot benefit from array operations, so they cannot use the vector or matrix instruction set extensions.
The reason is that for the other benchmarks the CPU vendors have always succeeded sooner or later, to tweak their compilers and compiling options, or even the hardware of the CPUs, in order to get improved benchmark results that nonetheless are not indicative of the improvements in other applications.
On the other hand, the compiling benchmarks, like with gcc, and now also with clang, are too diverse in CPU resource usage and no special feature of the CPU has a significantly greater weight than others, so special tricks to enhance the benchmark results have never been found.
When looking at the past SPECint results, the values of the gcc benchmark remain the most reliable relative performance estimator.
I doubt that this will change in the near future.
Moreover, the multi-threaded compiling benchmark is also very useful, because it matches exactly a real-world workload that is extremely frequently encountered. Due to the great clock frequency difference between running a benchmark on a single thread and running it on all available threads, the single-threaded results have a very poor correlation with the multi-threaded results.
Comments
I mildly disagree - depending on your definition of "typical app". Most applications have much greater use of multi-processing and concurrent cross-thread (or cross-process) communication. Compilers, aside from high-level parallelism across modules, tend to be quite single-threaded applications.
If you're solely interested in single-core performance, then I would agree that they are a good stress test, but I think for a processor that is being sold on it's parallelism, they are not a great benchmark.
In the history of the SPEC benchmarks, the compiler benchmarks, like gcc, have been the best predictor of CPU performance for the applications that cannot benefit from array operations, so they cannot use the vector or matrix instruction set extensions.
The reason is that for the other benchmarks the CPU vendors have always succeeded sooner or later, to tweak their compilers and compiling options, or even the hardware of the CPUs, in order to get improved benchmark results that nonetheless are not indicative of the improvements in other applications.
On the other hand, the compiling benchmarks, like with gcc, and now also with clang, are too diverse in CPU resource usage and no special feature of the CPU has a significantly greater weight than others, so special tricks to enhance the benchmark results have never been found.
When looking at the past SPECint results, the values of the gcc benchmark remain the most reliable relative performance estimator.
I doubt that this will change in the near future.
Moreover, the multi-threaded compiling benchmark is also very useful, because it matches exactly a real-world workload that is extremely frequently encountered. Due to the great clock frequency difference between running a benchmark on a single thread and running it on all available threads, the single-threaded results have a very poor correlation with the multi-threaded results.