AMD's Opteron hits 3.2GHz
- Posted in
- IT Computing
The Secret Boost of the Opteron 2224
Socket F Opterons have a small secret weapon: a speed bump offers more than just a faster CPU. To understand this, take a look at the table below. We measured the L2 cache's bandwidth with Lavalys Everest 3.51.
The L2 cache of the Opteron 8218 at 2.6GHz is slower than the Core 2's L2 cache at 2.33. At about 10-11 GB/s it barely matches the theoretical peak bandwidth that DDR2 at 667MHz can deliver (10.6 GB/s), while its exclusive nature also forces it to exchange quite a bit of data with the L1 cache. Now combine this table with the following one, where we measured memory bandwidth.
It is no secret that a higher clocked integrated memory controller can increase the actual delivered bandwidth of the same DDR2 modules. But it also helps that the L2 cache is able to swallow the bandwidth that the memory is capable of delivering. Also notice that without the use of SSE2 instructions, the memory subsystem of the 5000p chipset delivers relatively disappointing amounts of bandwidth. As most applications do not use carefully tuned SSE2 code to get data from memory, this should reflect the real world situation most of the time. And of course, until Intel introduces the Nehalem family, memory latency will continue to be one of the strong points of AMD.
The latency penalty that FB-DIMM introduces is huge. To get an idea, we added the latency measured with a Core 2 Duo 2.933 using 2x 2GB 533MHz DDR2. The staggering conclusion is that registered FB-DIMMs add - in the worst case - about 200 cycles or 66ns of latency. Sure, some of that latency can be attributed to the buffering which is necessary for server memory. Buffered memory contains registers which will actually hold data for one full clock cycle before it's passed on. So this means that registered memory should add about 8ns (2 clock cycles at 266MHz base clock, DDR2-533).
The secondary benefit of FB-DIMMs is that motherboards can use more DIMMs per bank, potentially increasing total memory capacity. AMD already gets around this quite easily with up to eight DIMM sockets per CPU socket, however, so this benefit really doesn't materialize in any reasonable form. The bottom line is that while FB-DIMMs were a potentially good idea from a purely theoretical point of view, it is rather obvious that in practice they have some pretty bad consequences.
Socket F Opterons have a small secret weapon: a speed bump offers more than just a faster CPU. To understand this, take a look at the table below. We measured the L2 cache's bandwidth with Lavalys Everest 3.51.
| Lavalys Everest 3.51 L2 Bandwidth | |||
| Read (MB/s) | Write (MB/s) | Copy (MB/s) | |
| Dual Xeon 5160 3.0 GHz | 22019 | 17751 | 23628 |
| Xeon E5345 2.33 GHz | 17610 | 14878 | 18291 |
| Opteron 2224 SE 3.2 GHz | 14636 | 12636 | 14630 |
| Opteron 8218HE 2.6 GHz | 11891 | 10266 | 11891 |
The L2 cache of the Opteron 8218 at 2.6GHz is slower than the Core 2's L2 cache at 2.33. At about 10-11 GB/s it barely matches the theoretical peak bandwidth that DDR2 at 667MHz can deliver (10.6 GB/s), while its exclusive nature also forces it to exchange quite a bit of data with the L1 cache. Now combine this table with the following one, where we measured memory bandwidth.
| Lavalys Everest 3.51 Memory Bandwidth | ||||
| Read (MB/s) | Write (MB/s) | Copy (MB/s) | Latency (ns) | |
| Dual Xeon 5160 3.0 GHz | 3656 | 2771 | 3800 | 112.2 |
| Xeon E5345 2.33 GHz | 3578 | 2793 | 3665 | 114.9 |
| Opteron 2224 SE 3.2 GHz | 7466 | 6980 | 6863 | 58.9 |
| Opteron 8218HE 2.6 GHz | 6944 | 6186 | 5895 | 64 |
It is no secret that a higher clocked integrated memory controller can increase the actual delivered bandwidth of the same DDR2 modules. But it also helps that the L2 cache is able to swallow the bandwidth that the memory is capable of delivering. Also notice that without the use of SSE2 instructions, the memory subsystem of the 5000p chipset delivers relatively disappointing amounts of bandwidth. As most applications do not use carefully tuned SSE2 code to get data from memory, this should reflect the real world situation most of the time. And of course, until Intel introduces the Nehalem family, memory latency will continue to be one of the strong points of AMD.
| Processor Latency Comparison | ||||||
| CPU | L1 | L2 | L3 | min mem | max mem | Absolute latency (ns) |
| Xeon 5160 3.0 - DDR2 533 | 3 | 14 | 69 | 380 | 127 | |
| Xeon 5160 3.0 - DDR2 667 | 3 | 14 | 67 | 338 | 113 | |
| Core 2 Duo 2.933 - DDR2 533 | 3 | 14 | 67 | 180 | 61 | |
| Quad Xeon E5345 2.33 - DDR2 533 | 3 | 14 | 80 | 280 | 120 | |
| Quad Xeon E5345 2.33 - DDR2 667 | 3 | 14 | 80 | 271 | 116 | |
| Xeon 7130M 3.2 - DDR2 400 | 4 | 29 | 109 | 245 | 624 | 195 |
| Opteron 880 2.4 - DDR333 | 3 | 12 | 84 | 228 | 95 | |
| Opteron 2224 SE - DDR2 667 | 3 | 12 | 72 | 189 | 59 | |
| Opteron 2218 HE - DDR2 667 | 3 | 12 | 62 | 157 | 60 | |
The latency penalty that FB-DIMM introduces is huge. To get an idea, we added the latency measured with a Core 2 Duo 2.933 using 2x 2GB 533MHz DDR2. The staggering conclusion is that registered FB-DIMMs add - in the worst case - about 200 cycles or 66ns of latency. Sure, some of that latency can be attributed to the buffering which is necessary for server memory. Buffered memory contains registers which will actually hold data for one full clock cycle before it's passed on. So this means that registered memory should add about 8ns (2 clock cycles at 266MHz base clock, DDR2-533).
The secondary benefit of FB-DIMMs is that motherboards can use more DIMMs per bank, potentially increasing total memory capacity. AMD already gets around this quite easily with up to eight DIMM sockets per CPU socket, however, so this benefit really doesn't materialize in any reasonable form. The bottom line is that while FB-DIMMs were a potentially good idea from a purely theoretical point of view, it is rather obvious that in practice they have some pretty bad consequences.
Originally published at https://www.anandtech.com/show/2294 (Wayback Machine).