Expensive Quad Sockets vs. Ubiquitous Dual Sockets

Power Consumption

We have already done fairly comprehensive power consumption comparisons; note however the numbers of the dual Xeon X5570 are higher than in that article. The reason is that we use the 1U ASUS RS700-E6 and not one of the nodes in the ASUS twin server. The latter server has been sent back to ASUS, and the 1U model is of course a better choice if you want to compare with the other servers in this review, which are all single node servers. We notice that our ASUS RS700-E6 consumes about 12W more than the node of the ASUS twin, which has a smaller motherboard with slightly less features.

As the Dual Xeon X5570 had only 24GB, we determined how much one DIMM adds to the idle and full load power, and we added that power to the numbers measured. So all dual socket servers have 32GB of RAM, while the quad socket servers have 64GB of RAM. The reason is that we like to know what will happen if we replace two dual socket servers by one quad socket.

A software bug in ESX is the reason why most servers - except Intel's - are not able to use EIST when running ESX 4.0 (VMware vSphere 4). This bug will be solved in ESX 4.0 SP1, which will be out at the end of this year. ASUS did not want to wait and circumvented the problem by adapting their BIOS. Thumbs up to ASUS for addressing this problem at such short notice. All the servers were able to use power saving features on ESX, as the AMD platform does not suffer from the ESX bug. To our surprise, enabling EIST did not decrease the power significantly: we went from 165W to 162W. So far as we know, there are no tools that can read out CPU clock and voltage data, so we have no way of verifying what is going on. Enabling AMD's PowerNow! causes a drop of about 10%. A possible explanation is that EIST does work, but that the PCU (Power Control Unit) of the Xeon x55xx is already shutting down so many parts of the die that the CPU sips very little power in idle and there is little room for any improvements. Our measurements were confirmed when we measured with and without EIST on Intel's low power optimized "Willowbrook/Chenbro" server.

The problem that the quad socket servers have is immediately apparent: they consume close to twice as much as the dual socket platform. That is rather bad news for the quad socket servers: it means that buying one large server instead of two dual does not result in any tangible power saving. One of the reasons is that for example our dual hex-core Opterons work together with very efficient 650W power supply, while our quad socket platform is using a relatively heavy 1200W PSU. Quad socket platforms still have a lot of room for improvement when it comes to power efficiency. What is worse is that the performance/watt of the dual servers is clearly better. Let us check the power numbers are full load.

Again, the quad machines do not really convince us that they may save us a lot of power, especially from the performance/watt point of view. If you are not performance limited but memory limited, the quad machines might still make some sense.