Test protocolAs we said in our article on SSDs, TRIM and IOMeter, testing SSDs correctly in an environment compatible with TRIM is not necessarily an easy exercise. Various parameters need to be taken into account and the fact that SandForce controllers include data compression algorithms only complicates things further.
Some very popular tools such as CrystalDiskMark give turnkey solutions, but these results are unfortunately not entirely conclusive. For example, the test file is only 4 GB in size, which is fine for a quick test but gives pretty variable results. Also the random tests only address the SSD across a restricted area.
Other tools such as h2Bench, HD Tune or HD Tach were designed for testing hard drives and sometimes operate on drives that contain no data. They can then get mixed up during read measurements, as random reads are turned into sequential reads by some controllers when an SSD is empty!
The marvellous box of tools that is IOmeter is what’s known as a synthetic benchmark. We used it to measure performance in several cases:
- Sequential reads by blocks of 2 MB
- Sequential reads by blocks of 4 KB
- Random reads by blocks of 4 KB
- Sequential writes by blocks of 2 MB
- Sequential writes by blocks of 4 KB
- Random writes by blocks of 4 KB
This gives us a vision of the speed of any given storage device in terms of throughput, but also in terms of I/0s. The tests on 4 KB blocks were carried out with 1, 2, 4, 8, 16 and 32 simultaneous commands so as to highlight the controller’s ability to parallelise accesses.
Next we ran the practical tests, first of all with reads and writes of various groups of files. These groups were composed as follows:
- Extra large: 731.17 MB on average
- Large files: 5.2 MB on average
- Medium sized files: 800.88 KB on average
- Small files: 48.78 KB on average
We used a RamDisk as the source or the target for reads or writes on the SSD. In view of the high performance of recent SSDs and so as to obtain results that are subject to as little variation as possible, we used Robocopy with an in-house piece of software which enables us to carry the tests out repeatedly.
To finish with we carried out some purely practical tests, namely timed operations after installation of Windows 7 (64 bit) on each of the SSDs:
- Boot Windows 7
- Installation of Photoshop CS 5
- Start up 3D Studio Max
- Start up 3D Studio Max + Photoshop + Word + Excel
- Launch a game in Civilization V
- Launch a game in Crysis 2
We tried to add another test halfway through, a test that involved processing groups of images in Photoshop. The problem is that even with a simple processing task such as JPEG recompression, the performance of the various devices don’t have much of an impact.
We left out benchmarks such as PC Mark Vantage, which are halfway between synthetic and practical, or any of the other pieces of software that repeat recorded accesses, known as traces. We don’t consider such results to be pertinent as they consist in carrying out accesses as fast as possible on the storage devices. They tell you that device X is the fastest but if at the end of the day practical performance levels are limited by the machine’s processing capabilities, what’s the point?
If we had recorded the accesses carried out by Photoshop in processing groups of images and used a trace benchmark, we would typically have observed differences of a magnitude of around 200% between SSDs, while in practical usage there’s no difference. Also, these traces simply record the type of access but don’t take into account the content, which can show SandForce controllers in a more positive light depending on whether compressible data or data that has already been compressed is used.
The test configuration consisted of a Core i7-2600K mounted on an Intel DP67BG with 16 GB of DDR3-1333 and a Radeon HD 5870. The synthetic and file copying tests were carried out in Windows 7 (64 bit) with the storage system tested as the secondary drive and the boot drive a Crucial M225 64 GB, all with Intel RST 10 drivers in AHCI mode. The other tests were carried out with the test storage device as the primary drive.
For comparison we also included the star SSDs from the previous generation, namely the:
- Corsair Force 120 GB (SandForce SF-1200 + 34nm IMFT flash)
- Crucial C300 128 GB (Marvell 88SS9174 + 34nm IMFT flash)
- Crucial C300 256 GB (Marvell 88SS9174 + 34nm IMFT flash)
- G.Skill Falcon II 128 GB (Indilinx Barefoot + 34nm IMFT flash)