They do seem to talk about writes per second. That means you must flush to disk, not read from cache.

[Edit, more info] They seem to run with multiple clients which also stresses the system. From their explanation * The clients simulate 320,000 concurrent sessions