Exactly. You really should avoid doing performance optimizations without measurements that show improvement as well as provide some coverage against performance regressions.
Here's an example, Thrift (as used in Hector, a Cassandra client), had someone make a performance improvement:
The discussion has a lot of "shoulds", and one measurement of latency distributions, but no measurement of typical workloads or bulk inserts. Turns out, that caused at least a 30% performance regression:
Comments
I cam here to say this. It's weird to talk about performance without actually showing any performance figures.
I'm all for removing old code, but if you're going to claim performance gains then why not measure those?
Exactly. You really should avoid doing performance optimizations without measurements that show improvement as well as provide some coverage against performance regressions.
Here's an example, Thrift (as used in Hector, a Cassandra client), had someone make a performance improvement:
https://issues.apache.org/jira/browse/THRIFT-959
The discussion has a lot of "shoulds", and one measurement of latency distributions, but no measurement of typical workloads or bulk inserts. Turns out, that caused at least a 30% performance regression:
https://issues.apache.org/jira/browse/THRIFT-1121