Expensive Quad Sockets vs. Ubiquitous Dual Sockets
by Johan De Gelas on October 6, 2009 1:00 AM EST- Posted in
- IT Computing
Power Consumption
We have already done fairly comprehensive power consumption comparisons; note however the numbers of the dual Xeon X5570 are higher than in that article. The reason is that we use the 1U ASUS RS700-E6 and not one of the nodes in the ASUS twin server. The latter server has been sent back to ASUS, and the 1U model is of course a better choice if you want to compare with the other servers in this review, which are all single node servers. We notice that our ASUS RS700-E6 consumes about 12W more than the node of the ASUS twin, which has a smaller motherboard with slightly less features.
As the Dual Xeon X5570 had only 24GB, we determined how much one DIMM adds to the idle and full load power, and we added that power to the numbers measured. So all dual socket servers have 32GB of RAM, while the quad socket servers have 64GB of RAM. The reason is that we like to know what will happen if we replace two dual socket servers by one quad socket.
A software bug in ESX is the reason why most servers - except Intel's - are not able to use EIST when running ESX 4.0 (VMware vSphere 4). This bug will be solved in ESX 4.0 SP1, which will be out at the end of this year. ASUS did not want to wait and circumvented the problem by adapting their BIOS. Thumbs up to ASUS for addressing this problem at such short notice. All the servers were able to use power saving features on ESX, as the AMD platform does not suffer from the ESX bug. To our surprise, enabling EIST did not decrease the power significantly: we went from 165W to 162W. So far as we know, there are no tools that can read out CPU clock and voltage data, so we have no way of verifying what is going on. Enabling AMD's PowerNow! causes a drop of about 10%. A possible explanation is that EIST does work, but that the PCU (Power Control Unit) of the Xeon x55xx is already shutting down so many parts of the die that the CPU sips very little power in idle and there is little room for any improvements. Our measurements were confirmed when we measured with and without EIST on Intel's low power optimized "Willowbrook/Chenbro" server.
The problem that the quad socket servers have is immediately apparent: they consume close to twice as much as the dual socket platform. That is rather bad news for the quad socket servers: it means that buying one large server instead of two dual does not result in any tangible power saving. One of the reasons is that for example our dual hex-core Opterons work together with very efficient 650W power supply, while our quad socket platform is using a relatively heavy 1200W PSU. Quad socket platforms still have a lot of room for improvement when it comes to power efficiency. What is worse is that the performance/watt of the dual servers is clearly better. Let us check the power numbers are full load.
Again, the quad machines do not really convince us that they may save us a lot of power, especially from the performance/watt point of view. If you are not performance limited but memory limited, the quad machines might still make some sense.
Facebook
Twitter
RSS
32 Comments
View All Comments
Photubias - Wednesday, October 7, 2009 - link
This is surely to be tested, but the Fiorano platform (as this AMD Chipset is called), is yet to be released.solori - Wednesday, October 7, 2009 - link
Fiorano (SR5690/SP5100, et al) are out now for Socket-F and really require an Istanbul to show their stuff (like IOV, etc). With a minor tweak on HT bus speeds, don't expect to see much improvement in memory bandwidth for Fiorano/Socket-F pairings. Where you should see improvement is in power consumption - pairing HE/EE Istanbul parts with Fiorano/Kroner should create a better performance/watt result in virtualization.Collin C. MacMillan
http://blog.solori.net">http://blog.solori.net
bpdski - Tuesday, October 6, 2009 - link
It is pretty amazing how fast the new 55xx chips are. Personally, I am holding out on any new server purchases and deployments until the EX systems come out next year. I am pretty excited about the performance potential of a dual or quad octal-core system. I feel for AMD, but if the EX systems scale as well as they should, they are really going to crush the Opterons.duploxxx - Wednesday, October 7, 2009 - link
2 answers to that, first off all looking at the design EX will be way more expensive creating a gap between 2 socket-4 socket platform even when only deploying 2 octa will be a very expensive baseline due to the motherboard layout. To expensive actually and a lot of focus trying to get risc/sparc marketshare.Second don't you think AMD knows this? The c32 G34 platform launch is much closer then people think, AMD made a clear roadmap and since 45nm all looks like going well on shape, keep in mind the cpu for the new platform is almost ready since it is based on istanbul and the new platform chipset was also released few weeks ago for the socket F platform, you will also see much more OEM activity with this platform due to one brand supplier, no longer need of the old nvidia/broadcom.
EX was delayed-delayed-delayed if it continues like this it will be launched more or less at the same time, so keep the feeling. BTW even if the 55xx sereis would be again a bad performing server part (which it is finally not thank you intel) 75% of the market would be still buying it just for the brand name.....:)
cosminliteanu - Tuesday, October 6, 2009 - link
Many thanks for this article !:)
BrightCandle - Tuesday, October 6, 2009 - link
A dual socket will easily fit in a 1U. But 1.25A is some serious extra cost within a colo.The 2U quad sockets on the other hand are a busting 500W+, again serious extra money in a colo.
The Colo's want you using 0.5A per 1U, there is a major mismatch from these machines to the reality of the power you can actually get. Love the speed, not liking the cost of running them.
sonicdeth - Tuesday, October 6, 2009 - link
Thanks for this. Personally I can't recommend any of the quad socket systems until we see Intels Nehalem-EX early next year. The dual core 55xx series is just fantastic for the price (especially with VMware). We've deployed several HP 380G6's and couldn't be happier.Bazili - Tuesday, October 6, 2009 - link
Great article. Congrats!!!Could you pleas include a software price analysis? I guess it can show huge differences among a 24 core box and a 8 core box.
tobrien - Tuesday, October 6, 2009 - link
these are amazing articles, you guys do such an awesome job with these.thanks a ton!
JohanAnandtech - Wednesday, October 7, 2009 - link
Thanks for the kudos! much appreciated :-)