Supreme Commander Benchmark - BeHardware
>> Processors

Written by Marc Prieur

Published on March 5, 2007

URL: http://www.behardware.com/art/lire/660/


Page 1

The game



The release of a new game, which requires the high performance of hardware components is always of great interest to us for potential integration to our test protocol. This is the case of Supreme Commander, Chris Taylor’s latest RTS. Because of its results in tests we decided to write this article.

The game
Supreme Commander is a real time strategy game, which functions on another scale than most current games, as Total Annihilation did a few years ago,.

The game features a zoom function called "Strategic", which gives the user the possibility to choose a different height other than that of the battlefield level. The implementation of such a function compared to standard views has of course some consequences on performances.


The other important characteristic of Supreme Commander is, of course, like many RTS the great number of calculations dependant on the CPU, whether they are AI, unit movements, management of projectiles, etc. In order to take advantage of the potential of the latest CPUs, engineers of GPG have made Supreme Commander a multithread game. This means that it is capable of using processors with 1, 2 and even 4 cores. To do so, they have chosen a "relatively" simple support of threading in that each major part of the game is a thread. There is one thread for the graphic part, another for the simulation, and less heavy ones for sound amongst other things. GPG hasn't answered our questions concerning AI, mainly if it’s included to the simulation thread or if it is on another thread of its own.

What is the impact of this implementation on performances? Is the processor really dominant? This is what we will see in this article. An interesting point is that Supcom isn't Quake-like. The view type combined with lateral scrolling mostly replaced by the zoom means that the framerate required to enjoy the game doesn't have to be much higher than 25 fps and even at 15 fps it is still possible to play comfortably. Between 10 to 15 fps, it’s a bit bothersome and below 10 fps it’s dismal.


Page 2
Graphic cards

3D graphic cards
Before taking a closer look at the CPU, let's see the graphic card’s impact on Supreme Commander’s performances. Our tests showed that when there were many units on the screen, the graphic card’s influence was quite significant because of the geometry, a limitation that we don’t often encounter. This part of the game is indeed quite large because of the strategic zoom and LOD (level of detail) implemented by developers at the various zoom levels.


This aspect is visible when many units are on screen, and also with some of the maps resulting in a consequent performance reduction. For example, here is the framerate of an Athlon 64 X2 6000+ and a Radeon X1900 GT with a resolution of 1024*768, level of detail set on « high » , without AA, and with several 5x5 km maps. They are quite small and with the zoom out set to the maximum, geometry doesn't disappear completely. The first figure corresponds to the framerate with the zoom out max and the second after the activation of the mini map, which is in fact a second 3D view:
Ambush Pass : 50 fps / 28 fps
Blasted Rock : 100fps / 65 fps
Canis River : 50 fps / 27 fps
Four-Corners : 80 fps / 50 fps
Sentry Point : 27 fps / 16 fps
Sludge : 100 fps / 80 fps
Theta Passage : 100 fps / 100 fps
Winter Duel : 45 fps / 37 fps
As you can see, some maps require high performances, especially the Canis River or Sentry Point. The activation of the mini map also has a great impact on performances and it’s unfortunate that the LOD used for this view isn't automatically set to a lower level.

What is the difference between these maps? On the first one, there is a permanent sand storm whereas with the second the vegetation is quite significant. We noted that the LOD can be reduced via a command in the console, however the adjustment isn't saved and this increases the speed of the appearance of strategic icons above units (not to everyone's taste). Changing the cam_setlod WorldCamera with Sentry Point value from 1 to 1.5 increases the framerate from 27 to 100 fps. The reason for this is that vegetation is no longer calculated in the strategic view. Unfortunately, this same adjustment doesn't have an impact on the sand storm.

Vue 1
Vue 2

For our test, we decided first to measure the framerate obtained when zooming in on one unit, the commander,(View 1) and then with an intermediate zoom to see our base (View 2). We also chose the initial setting, level of detail "High", except we deactivated AA. One interesting point is that between the resolutions of 4/3, 5/4 and 16/10, the game doesn't display the same thing on the battlefield when we focus on specific points. This explains the low performance cost of increasing from 1600*1200 to 1920*1200. These figures were obtained with a Core 2 E6700 :



If with the first framerate measurement, the impact of modification of resolution is quite significant on performances, this isn't the case at all with the second. Indeed, if the limitation of performances is generally on the graphic card level, this time it’s due to geometry. The result is that even in 1024*768 using the intermediate zoom it isn't really possible to enjoy the game with a 7600 GT. A 7900 GTO is the minimum required and the GeForce 8 is the only one to really reach satisfactory result. The 8800 GTS in this test featured 32 MB of memory. Under those circumstances, the activation of 4x anti aliasing and anisotropic filtering doesn't have a large negative impact. For example, with a 8800 GTX, the performance reduction is 9% and 8% in 1920*1200 with both tests, instead of 15 and 10% with 2560*1600. In 1920*1200, with a 7900 GTO, the reductions are 5 and 7%, instead of 7% and 8% with the X1900 GT.

Now the question is, is it really possible to play Supreme Commander with a mid-level graphic card? The answer is, “Yes!”. The reason is that it is possible to modify graphic settings. Choosing the medium setting doesn't have a great impact and the improvement is 1 fps with the 7600 GT and X1650 XT in 1280*1024. However, choosing the "low" level of detail seriously increases performances, resulting in 46 fps and 28 fps in View 1 and 2 with 7600 GT and 53 and 34 fps with X1650 XT.

High
Medium
Low
High sans ombres

Unfortunately, the compromise on graphic quality is quite significant and the level of detail reminds us of games from the 20th century. To have a good level of detail and a nice boost in performance, the only parameter to modify is the deactivation of shadows. If we haven't really noted differences for the different level of shadows, deactivating them increases performances by 55% with View 2 with the two maps.


Page 3
Processors

The processors
To evaluate CPU performances, we recorded a game against three AI and on a 10x10 km map. This features the destruction of the three AI by a Scathis combined with land and aerial troops and, of course, a small nuclear bomb. For one minute of "Supcom" time, we placed ourselves on our base and record the average framerate in this view. In terms of processor use, the percentages are 98% with a Core 2 Duo E6700 and 55% with a Core 2 Duo QX6700.


The framerate doesn't mean anything by itself with a game of this type. Indeed, if processor power isn't sufficient, the simulation time will increase. This means that 1 minute of Supreme commander simulation time will last 1 minute 30 seconds or more. The analysis of the combination of framerate, obtained with an 8800 GTX, and simulation time is important to have meaningful results.

Also, we noted that performances could be subject to great variation in scenes requiring heavy CPU resources. Therefore we give you results from 5 tests.
Core 2
To start with, here are performances obtained by increasing the number of cores with a Core 2 Quad QX6700 :


As you can see, the gap isn't that great between 3 or 4 cores and the simulation isn't made in real time even under these circumstances. Reducing to 2 cores seriously cuts the framerate but has a lesser impact on the real time of the simulation. This is also true when reducing to 1 core as you can see in this test. With this view, performances are catastrophic. Is it possible to do something about it? Yes! By reducing the duration of the simulation, some processor resources are freed up for rendering and this helps increase the framerate. Here are the parameters we selected for increases over 10 fps with 2 and 1 cores:


Here again, some compromises will be necessary. What is the impact of overclocking? We verified this by increasing processor frequency to 3.5 GHz:


With 4 cores, the simulation is close to 60 seconds and the framerate increases noticeably. With two cores, results are really surprising and performance is more than doubled! In fact, this is an extreme variation but with our tests, we noted very significant variations at times regardless of the graphic card (ATI or Nvidia), the game version (original or after patches), or sound management (without or with an Audigy). The same type of variation, a little less consequent, can be observed with 2 cores at 2.67 GHz and at normal speed.
Athlon 64 X2
Now, here are the results with an Athlon 64:


As you can see, the framerate variation isn't too great with 1 or 2 cores with the A64 X2 6000+ and is low in both cases. Simulation time, however, literally explodes with only one core. With the A64 X2 3800, it’s identical to one core at 3 GHz. The framerate, however, is better and two cores at 2 GHz are better than one at 3 GHz!

How much does the simulation speed need to be decreased to reach over 10 fps?


Here again, the point of a dual core even with a low core frequency is obvious. Performances can also be strongly subject to variations between the two cases especially with the A64 X2 6000+ (from 11 to 14 fps !).

A64 X2 6000+ vs the Core2 E6700
What about the AMD/Intel rivalry? To measure the performance difference, we compared the simulation speed.


With normal speed, the Core 2 and A64 X2 generally provide quite equal simulation times. The framerate, however, is noticeably higher with the Intel processor. Strangely though, when game speed is decreased, the situation reverses. Indeed, to have similar simulation time, the time selected must be -3 for the A64 X2 and -2 for the Core 2. With these parameters, the A64 X2 performs best at 11 to 14 fps instead of a little less than 10 fps with the Core 2.


Page 4
Conclusion

Conclusion
Supremer Commander requires extremely high performances from the graphic card and the processor unlike what some may have thought. This partly results from the choices made by GPG to set up the strategic zoom making the hardware requirements correspond to high end computers today and middle line products of tomorrow.

Of course, we knowingly used a heavy scene and the game on Internet will unload the processor from AI management. Nevertheless, it is inconceivable to really enjoy the game with less than a dual core unless you stick to 1vs1 games that will last less than half an hour. In our test conditions, it is of course the quad core that is best for play even if the multi-threading of Supreme Commander is quite simple and the 4 cores are only partially used. In fact, 3 are enough.

As for the graphic card, if it is possible to play with less powerful cards on the condition that the level of detail is significantly reduced, we recommend at least the ex high end of DirectX 9 cards, the X1900/1950 and 7900/7950. Like the quad core, the GeForce 8800 brings some real improvements and we have heard about a DirectX 10 patch for the game. We have to hope that it will reduce the CPU load due to the API and that, in consequence, will increase overall performances. Knowing that the performance improvement is more on the geometrical level, we will be able to increase the resolution without much consequence or add effects such as anti-aliasing.


A word about SLI…Whether for the GeForce 7 or GeForce 8, it doesn’t change much. Having two cards for a dual monitor configuration isn't really useful either because both monitors are managed by a single graphic card. It’s also interesting to note that the CPU and motherboard aren't the only restricting factors. Supreme Commander is quite hungry in terms of memory and with average games it can easily requires up to 1 GB. With very big games (7 AI on a big map) we reached 2GB of virtual memory attributed to the game. Increasing this limit leads to a game crash. To go around this problem, modify the boot.ini of a Windows 32 bit to reach 3 GB for one application and also the game executable (to have more information, take a look at this topic ). This corresponds to an extreme case, but in practice, 2 GB of RAM memory are recommended.

Are we going to integrate Supreme Commander to our test protocol for processors and graphic cards? This was the question we tried to answer in this article. The answer is, no. The most interesting aspect of Supreme Commander is that this game is multithreaded and it isn't really usable as a bench. Having a combination of simulation time and framerate that varies according to the processor power isn't a problem by itself. However, it is a problem to have significant performance variations between two executions without any real explanation. In consequence and unfortunately, this prevents any use for a serious test protocol and product analysis.


Copyright © 1997-2008 BeHardware. All rights reserved.