Dynamic Power Management: A Quantitative Approach

The Hardware

One of the latest low power Xeons is the Intel Xeon L3426. It is the Xeon version of the "Lynnfield" architecture. What really makes this CPU special is the ability to boost its clock speed from 1.86GHz (default) to 3.2GHz without dissipating more power than 45W. The Turbo Boosted clock speed of 3.2GHz can only be reached if only one core is under load. Combine this with the fact that the CPU can cope with eight threads at once (four cores + Hyper-Threading) and you will see why this CPU deserves special attention. It should offer very low power when the load on the server is low as in that case the CPU steps back to a 1.2GHz clock. At the same time the CPU can scale up to 3.2GHz to provide excellent single threaded performance at 3.2GHz. And it also holds the promise that the CPU will never consume more than 45W, even when under heavy load. Performance will be "midrange" in those situations as the CPU cannot clock higher than 1.86GHz, but eight threads will still be running simultaneously.

At $284 this CPU looks like the best Intel offering ever in the entry level server space... but it does have drawbacks of course. In the desktop space, the affordable Core i7 860 "Lynnfield" CPU relegated the expensive Core i7 900 series "Bloomfield" CPUs to the shrinking high-end desktop CPU market. This is not going to happen in the server world: the "Lynnfield" CPU has no QPI links, so it cannot be used in multi-socket servers. The triple channel Xeon X55xx series ("Gulftown") will continue to be Intel's dual socket Xeons until they are followed up by quad- and six-core 32nm Westmere CPUs. So the first drawback is that you will be limited to four physical cores per server (and four logical SMT cores). For those of us running non-virtualized workloads, this is probably more than enough CPU power.

Talking about virtualization, that is the second drawback: each of the memory channels supports three DIMMs. As a result the CPU does not support more than 24GB. This is another reason why the cheap "Lynnfield" Xeon is not going to threaten the Xeon 5500 series anytime soon: a dual Xeon "Gulftown" server supports up to 144GB.

We also have the Xeon X3470 in the lab. When running close to idle, this CPU can also throttle back to 1.2GHz and save a lot of power. For the occasional single threaded task, the X3470 offers the best single threaded performance in the whole server CPU industry: one core can speed up to 3.6GHz. Yes, this is the "Xeonized" Core i7 860. Why does this CPU interest us? Well, the Xeon 3470 (2.93GHz) is only one speed bin faster than the X3460 (2.83GHz). The X3460 costs around the same price ($316) as the L3426 and can Turbo Boost up to 3.46GHz. And the X3460 brings up an interesting question: is a L3426 really the most interesting choice if you want a decent balance between performance and power? The L3426 has the advantage that you are sure you will never break a certain power limit. The X3460 however offers low power at low load too, and has more headroom to handle rare peaks. Below you can find the specs of our Intel server:

Intel SC5650UP
Single Xeon X3470 2.93GHz or Xeon L3426 1.86GHz
S3420GPLC, BIOS version August 24, 2009
Intel 3420 chipset
8GB (4 x 2GB) 1066MHz DDR-3
PSU: 400W HIPro HP-D4001E0

While we did not have a comparable AMD based server yet, this article would not be complete without a look at an AMD based system. AMD promised to send us a low power server, but after some back and forth correspondence it became clear the system would not be able to meet our deadline. Rest assured that we will update you once we get the new low power system from AMD. At the moment AMD looks a bit weak in the low cost server arena as its honor is defended by the 2.5GHz - 2.9GHz Opteron "Suzuka" CPU. That is a single CPU solution based on "Shanghai": four K10 cores and a 6MB L3 cache. This platform is almost EOL: we expect the San Marino and Adelaide platform around CeBIT 2010. Servers based on these new AMD platforms will save quite a bit of power compared to their older siblings. The six-core Lisbon that will find a home in these servers will be a slightly improved version of the current six-core AMD Opteron. Below are the specs of our AMD server:

Supermicro A+ Server 1021M-UR+V
Dual Opteron 2435 "Istanbul" 2.6GHz or Opteron 2389 2.9GHz
Supermicro H8DMU+, BIOS version June 18, 2009
8GB (4 x 2GB) 800MHz
PSU: 650W Cold Watt HE Power Solutions CWA2-0650-10-SM01-1

AMD picked the components in this server for our previous low power comparison of servers. We went with the Opteron 2435 as the six-core Opteron offers a very decent performance/watt ratio on paper. We will update the numbers once the Opteron 2419 EE arrives.

Making the comparison reasonably fair

So we had to work with two different servers. While the AMD versus Intel side of things is not our main focus, how can we make a reasonably fair comparison? The difference in power supplies is hardly a problem: both AMD and Intel feel that these power supplies are among the best available as they were chosen for their low power platforms. Both power supplies are 90% efficient over a very wide range of power load. The problem is the fans.

The fans in the AMD machine are small and fast, with speeds up to 11900 rpm! We disabled fan speed control to keep the power consumption of the fans constant. There are four fans and we measured the fan power consumption by taking out the fans that blow over the memory while keeping the two fans that cool the CPUs. This way we were sure that our CPU would not overheat and leak more power. We carefully measured the temperature of the CPU and jotted down the power measurements in all of our tests. We found out that each fan consumes about 8W. We did the same thing for the Intel machine: the power consumption of each fan was measured at the electrical outlet. The memory DIMMs were also checked: there was no significant difference between DDR2-800 and DDR3-1066, both in idle as well as under load. By taking the fans out of the equation, we can get a very reasonable comparison of both platforms. So how well do the current CPUs manage power?