Dezső Sima 20 11 November

1. Dezső Sima 2011 November Platforms I. (Ver. 1.4)  SimaDezső, 2011

2. Contents 1. Introduction to platforms 2. Main components of platforms 3. Platform architectures 4. Memory subsystem design considerations 5. References

3. 1. Introduction to platforms 1.1. The notion of platform 1.2. Description of particular platforms 1.3. Representation forms of platforms 1.4. Compatibility of platform components

4. 1.1. The notion of platform

5. 1.1The notion of platform (1) 1.1 The notion of platform The notion platform is widely used in different segments of the IT industry e.g. by IC manufacturers, system providers or even by software suppliers with different interpretations. Here we are focusing on the platform concept as used typically by system providers. System providers however, may use the notion platform either in a more general or a more specific sense. Interpretation of the notion platform Interpretation in a more general sense Interpretation in a more specific sense Unified system design A particular unified system architecture, developed for a given application area. such as a DT or MP platform

6. 1.1The notion of platform (2) Interpretation the notion platform in a more general sense Unified system design means that the system architecture is partitioned to asmall number of standard components, such as the processor, memory control hub (MCH), I/O control hub (ICH) that are interconnected by specified (standard) interconnections. Core2 Duo Core 2 Extreme (2C) FSB: 1066/800/533 MT/s speed FSB Two memory channels DDR2-800/666/533 Two DIMMs per channel 965 Series MCH ME C-link DMI ICH8 Intel’s Core 2 Duo (and Core 2 Extreme (the highest speed model) aimed DT platform (the Bridge Creek platform) Thus the notion platform designates system architectures with unified design in the above sense.

7. 1.1The notion of platform (3) Remark The need for a unified system design, called platform design, arose in the PC industry in the time when PCI-based system designs were substituted by port based system designs, about 1998-1999 .

8. 1.1The notion of platform (4) Pentium II/ Pentium II/ Pentium III Pentium III Pentium III Processor bus Processor bus Main Memory System System Main Memory AGP controller AGP (SDRAM) controller (EDO/SDRAM) 2xIDE/ Hub interface ATA 33/66/100 LPC PCI bus Peripheral Super I/O (KBD, MS, etc.) controller AC'97 2xIDE/ATA33/66 2x/4x USB (Legacy and/or PCI device adapter slow devices) Peripheral controller PCI bus 2xUSB PCI device PCI to ISA adapter bridge ISA bus ISA bus ISA device adapter Legacy devices ISA device adapter Late PCI-based system architecture(~ 1998) (used typically with Pentium II/III (built around Intel’s 440xx chipset) Early port-based system architecture(~ 1999) (used first with Pentium III (built around Intel’s 810 chipset)

9. 1.1The notion of platform (6) Interpretation the notion platform in a more specific sense In a more specific sensethe notion platform refers to a particular unified system architecture, that is developed for a given application area, such as a DT, DP or MP platform. In this sense the notion platform is interpreted as a standardized backboneof a system architecture developed for a given application area that is built up typically of • the processor or processors, • the chipset, • in some cases, such as in mobile or business oriented DT platforms also the • networking component [7], • the buses interconnecting the above components of the platform as well as • the memorysubsystem (MSS) that is attached by a specific memory interface. • . Basic components of a platform The memory subsystem (LAN controller) Chipset Buses interconnecting the preceding basic components Processor or processors Subsequently, we will focus on the interpretation of the notion platform in this latter sense.

10. 1.1The notion of platform (5) The primary goals of introducing unified system designs are • to minimize design reworkwhile moving from one processor generation to the next and • to stabilize interfaces for server and desktop designs [2] • to shorten the time to market.

11. 1.1The notion of platform (8) Example 1: Intel’s Core 2 aimed home user DT platform (Bridge Creek) [3] Platform 1066 MT/s Display card 2 DIMMs/channel 2 DIMMs/channel C-link

12. 1.1The notion of platform (9) Example 2: Intel’s Nehalem-EX aimed Boxboro-EX MP server platform, assuming 1 IOH Platform Xeon 7500 (Nehalem-EX) (Becton) 8C Xeon 7-4800 (Westmere-EX) 10C / SMB SMB SMB SMB Nehalem-EX 8C Westmere-EX 10C Nehalem-EX 8C Westmere-EX 10C QPI SMB SMB SMB SMB QPI QPI QPI QPI SMB SMB Nehalem-EX 8C Westmere-EX 10C Nehalem-EX 8C Westmere-EX 10C SMB SMB SMB QPI SMB SMB SMB QPI QPI 2x4 SMI channels 2x4 SMI channels 7500 IOH DDR3-1067 DDR3-1067 ME ESI SMI: Serial link between the processors and SMBs SMB: Scalable Memory Buffer Parallel/serial conversion ICH10 Interfaces connecting platform components ME: Management Engine

13. 1.1The notion of platform (9) The structure of a platform is termed as its architecture (or topology). It describes the basic components and their interconnections and will be discussed in Section 3.

14. 1.1The notion of platform (10) Many facets of the platform concept The platform concept as seen from the point of view of the system developers • The platform concept and platform based design will be considered as part of the system • level design. • It became the topic of scientific research at the end of the 1990s, see e.g. [4]. Main goals of the system level design are • to reduce the complexity of designing complex systems by partitioning it, • in this way to reduce the time-to-market of products, • to be able to enhance system components (such as processors) upward compatible as long as • the same interfaces (e.g. an FSB with a given max. frequency) are used. Co-design of platform components Platform components are typically co-designed, announced and delivered as a set.

15. 1.1The notion of platform (11) The platform concept as seen from the point of view of the manufacturers • With the platform concept in mind manufacturers, like Intel or AMD will plan, design • and market all key components of a platforms, such as the processor or the processors • and the related chipset as an integrated entity [5]. • This is benefitial for the manufacturers since it motivates OEMs as system providers, • to buy all key parts of a computer system from the same manufacturer.

16. 1.1The notion of platform (12) The platform concept as seen from the point of view of the customers The platform concept is benefitial for the customers as well since an integrated “backbone” of a system architecture promises a more reliable and more cost effective system.

17. 1.1The notion of platform (13) Historical remarks System providers began using the notion “platform” about 2000, like • Philips’ Nexperia digital video platform (1999), • Texas Intruments (TI) OMAP platform for SOCs (2002), • Intel’s first generation mobile oriented Centrino platform for laptops, designated as the • Carmel platform (3/2003). Intel contributed significantly for spreading the notion platform when based on the success of their Centrino platform they introduced this concept also for their desktops [5] and servers [6], [7] in 2004.

18. 1.1The notion of platform (14) Intel’s early server and workstation roadmap from Aug. 2004 [6] Note • This roadmap already makes use of the notion platform without revealing platform names. • b) In 2004 Intel made a transition from 32 bit systems to 64 bit systems.

19. 1.1The notion of platform (15) Intel’s multicore platform roadmap announced at the IDF Spring 2005 [8] Note This roadmap includes also the particular platform designations for desktops, UP servers etc.

20. 1.2. Description of a particular platform

21. 1.2 Description of a particular platform (1) Description of a particular platform Description of a particular platform Detailing the platform architecture Example: The Tylersburg DT platform (2008) Processor MCH ICH

22. 1.2 Description of a particular platform (2) Detailing the platform architecture includes the architecture of the processor-, the memory- and the I/O subsystems (to be discussed in Section 3). Example: The Tylersburg DT platform (2008) Processor MCH ICH It is concerned with issues, such as whether the processors of an MP server are connected to the MCH via an FSB or otherwise, or whether the memory is attached to the system architecture through the MCH or through the processors etc.).

23. 1.2 Description of a particular platform (3) Description of a particular platform Description of a particular platform Identification of the platform components Detailing the platform architecture Example: The Tylersburg DT platform (2008) 1. gen. Nehalem (4C)/Westmere-EP (6C) Processor X58 IOH MCH ICH10 ICH

24. 1.2 Description of a particular platform (4) Description of a particular platform Description of a particular platform Identification of the platform components Detailing the platform architecture Specification of the interfaces interconnecting the platform components Example: The Tylersburg DT platform (2008) 1. gen. Nehalem (4C)/Westmere-EP (6C) 1. gen. Nehalem (4C)/Westmere-EP (6C) Processor QPI X58 IOH X58 IOH MCH DMI ICH10 ICH10 ICH

25. 1.2 Description of a particular platform (5) Remark The specification of a platform will be completed by thedatasheets of the related platform components.

26. 1.2 Description of a particular platform (6) Dependence of the platform architecture on the platform category Platforms may be classified according to the target area of application, such as Platforms Desktop (DT) platforms Dual processor (DP) platforms Mobile platforms Quad processor (MP) platforms Of course, beyond the above categories also further processor categories and related platforms exist, such as embedded processors and related platforms. In conformity with different platform categories also different platform architectures arise, as indicated below. Platform architecture Architecture of DT platforms Architecture of DP platforms Architecture of mobile platforms Architecture of MP platforms In these slides platform architectures will be discussed in Section 3, nevertheless restricted only for DT, DP and MP platforms.

27. 1.3. Representation forms of platforms

28. 1.3 Representation forms of platforms (1) 1.3 Representation forms of platforms • Thumbnail representation • Roadmap like representation (an arbitrarily chosen representation form in these slides) • Block diagram of a platform.

29. 1.3 Representation forms of platforms (3) a) Thumbnail representation It is a concise representation of a particular platform. In particular, the thumbnail representation • reveals the platform architecture, • identifies the basic components of a platform, such as the processor or processors, the chipset, • in some cases (e.g. in mobile platforms) also the Gigabit Ethernet controller, • and specifies the interconnection links (buses) between the platform components. Example Core2 Duo Core 2 Extreme (2C) FSB: 1066/800/566 MT/s speed FSB Two DDR2 channels 965 Series MCH DDR2-800/666/566 Two DIMMs per channel ME C-link DMI ICH8 Intel’s Core 2 Duo aimed home user oriented platform (The bridge Creek platform)

30. 1.3 Representation forms of platforms (5) 6/2006 Bridge Creek DT platform 7/2006 Core 2 Duo (2C) Core 2 Extr. (2C) DTcore Core 2 Duo (2C):E6xxx/E4xxx Core 2 Extreme (2C): X6800 E6xxx/X68001: Conroe E4xxx)1: Allendale 65 nm Conroe: 291 mtrs/143 mm2 Allendale: 167 mtrs/111 mm2 Conroe: 4 MB/Allendale 2 MB L2 X6800/E6xxx: 1066 MT/s E4xxx: 800MT/s LGA775 6/2006 965 Series MCH (Broadwater) FSB 1066/800/566 MT/s 2 DDR2 channels DDR2-800/666/533 4 ranks/channel 8 GB max. 6/2006 ICH8 ICH Core 2-aimed (65 nm) Example for stating the compatibility range of a platform The Core 2 Duo aimed DT platformthat targets home users (designated as the Bridge Creek platform). Core2 Duo Core 2 Extreme (2C) FSB: 1066/800/566 MT/s speed FSB Two DDR2 channels 965 Series MCH DDR2-800/666/566 Two DIMMs per channel ME C-link DMI ICH8 Beyond the target processor this platform may be used also with • the previous Pentium D/EE and Pentium 4 6x0/6x1/EE and • the subsequent Core 2 Quad lines of processors, as shown in the next slides. 1The Allendale is a later stepping (Steppings L2/M0) of the Core 2 (Steppings B2/G0), that provided typically only 2 MB L2 and appeared 1/2007.

31. 1.3 Representation forms of platforms (5) 6/2006 Bridge Creek DT platform 7/2006 Core 2 Duo (2C) Core 2 Extr. (2C) DTcore Core 2 Duo (2C):E6xxx/E4xxx Core 2 Extreme (2C): X6800 E6xxx/X68001: Conroe E4xxx)1: Allendale 65 nm Conroe: 291 mtrs/143 mm2 Allendale: 167 mtrs/111 mm2 Conroe: 4 MB/Allendale 2 MB L2 X6800/E6xxx: 1066 MT/s E4xxx: 800MT/s LGA775 6/2006 965 Series MCH (Broadwater) FSB 1066/800/566 MT/s 2 DDR2 channels DDR2-800/666/533 4 ranks/channel 8 GB max. 6/2006 ICH8 ICH Core 2-aimed (65 nm) Example for stating the compatibility range of a platform The Core 2 Duo aimed DT platformthat targets home users (designated as the Bridge Creek platform). Core2 Duo Core 2 Extreme (2C) FSB: 1066/800/566 MT/s speed FSB Two DDR2 channels 965 Series MCH DDR2-800/666/566 Two DIMMs per channel ME C-link DMI ICH8 Beyond the target processor this platform may be used also with • the previous Pentium D/EE and Pentium 4 6x0/6x1/EE and • the subsequent Core 2 Quad lines of processors, as shown in the next slides. 1The Allendale is a later stepping (Steppings L2/M0) of the Core 2 (Steppings B2/G0), that provided typically only 2 MB L2 and appeared 1/2007.

32. 1.3 Representation forms of platforms (6) 6/2006 Support of Pentium 4/D/EE processors Bridge Creek 5/2005 2/2005 1/2006 7/2006 Pentium D/EE 8xx1 Pentium 4 6x0/6x1/EE Pentium D/EE 9xx2,3 Core 2 Duo (2C) Core 2 Extr. (2C) DTcores (Smithfield) 2x1C (Presler) 2x1C Core 2 Duo (2C):E6xxx/E4xxx Core 2 Extreme (2C): X6800 E6xxx/X68001: Conroe E4xxx)1: Allendale (Prescott-2M) 1C 65 nm Conroe: 291 mtrs/143 mm2 Allendale: 167 mtrs/111 mm2 Conroe: 4 MB/Allendale 2 MB L2 X6800/E6xxx: 1066 MT/s E4xxx: 800MT/s LGA775 90 nm 169 mtrs 135 mm2 2 MB L2 800 MT/s Two-way multithreading LGA775 90 nm 2x115 mtrs 2x103 mm2 2x1 MB L2 800/533 MT/s No multithreading LGA775 65 nm 2x188 mtrs 2x81 mm2 2x2 MB L2 1066/800 MT/s No multithreading LGA775 6/2006 965 Series MCH (Broadwater) FSB 1066/800/566 MT/s 2 DDR2 channels DDR2-800/666/533 4 ranks/channel 8 GB max. 1Pentium EE 840 supports only 800 MT/s 2Pentium D 9xx support only 800 MT/s 3Pentium EE 955/965 supports only 1066 MT/s 6/2006 ICH8 ICH Supports also Pentium D/EE processors/90/65 nm Supports also Pentium 4 6x0/6x1/EE processors/90nm Core 2-aimed (65 nm)

33. 1.3 Representation forms of platforms (7) 6/2006 Support of Core 2 Quad processors) Bridge Creek DT platform 11/2006 7/2006 Core 2 Duo (2C) Core 2 Extr. (2C) Core 2 Quad (2x2C) DTcore Core 2 Duo (2C):E6xxx/E4xxx Core 2 Extreme (2C): X6800 E6xxx/X68001: Conroe E4xxx)1: Allendale Core 2 Quad (2x2C): Q6xxx Q6xxx: Kentsfield 65 nm Conroe: 291 mtrs/143 mm2 Allendale: 167 mtrs/111 mm2 Conroe: 4 MB/Allendale 2 MB L2 X6800/E6xxx: 1066 MT/s E4xxx: 800MT/s LGA775 65 nm 2x291 mtrs/2x143 mm2 2*4 MB L2 1066 MT/s LGA775 6/2006 965 Series MCH (Broadwater) FSB 1066/800/566 MT/s 2 DDR2 channels DDR2-800/666/533 4 ranks/channel 8 GB max. 6/2006 ICH8 ICH Supports also Core 2 Quad processors/65 nm Core 2-aimed (65 nm)

34. 1.3 Representation forms of platforms (8) c) Block diagram of a platform Example: The Core 2 aimed home user DT platform (Bridge Creek) (without an integrated display controller) [3] 1066 MT/s Display card 2 DIMMs/channel 2 DIMMs/channel C-link

35. 1.4. Compatibility of platform components

36. 1.4Compatibility of platform components (1) 1.4 Compatibility of platform components One of the goals of platform based designs is to use stabilized interfaces (at least for a while) to minimize or eliminate design rework while moving from one processor generation to the next [2]. Consequently, assuming platform based designs,platform components, such as processors or chipsets of a given lineare typically compatible with their previous or subsequent generations as long as the same interfaces are used and interface parameters (such FSB speed) or other implementation requirements (either from side of the components to be substituted or the substituting components) do not restrict this.

37. 1.4Compatibility of platform components (2) Limits of compatibility In the discussed DT platform the target processor is the Core 2, that is connected to the MCH byan FSB with 1066/800/533 MT/s. The target processor of the platform however, can be substituted • either by processors of three previous generations or • processors of the subsequent generation (Core 2 Quad) since all these processors have FSBs of 533/800/1066 MT/s, as shown before. Core2 Duo Core 2 Extreme (2C) FSB: 1066/800/533 MT/s FSB Two memory channels DDR2-800/666/533 Two DIMMs per channel 965 Series MCH ME C-link DMI ICH8 Nevertheless, The highest performance level Core 2 Quad, termed as the Core 2 Extreme Quad, provided already an increased FSB speed of 1333 MT/s and therefore was not more supported by the Core 2 aimed platform considered.

38. 2. Basic components of platforms 2.1. Processors 2.2. Buses interconnecting platform components 2.3. The memory subsystem

39. 1.1The notion of platform (6) Basic components of platforms - Overview As already discussed in Section 1. the notion platform is interpreted as a standardized backboneof a system architecture developed for a given application area that is built up typically of • the processor or processors, • the chipset, • in some cases, such as in mobile or business oriented DT platforms also the • networking component [7], • the buses interconnecting the above components of the platform as well as • the memorysubsystem (MSS) that is attached by a specific memory interface.. Basic components of a platform The memory subsystem (LAN controller) Chipset Buses interconnecting the preceding basic components Processor or processors Subsequently, we will discuss the following three basic components of platforms: • Processors (Section 2.1) • Buses interconnecting platform components (excluding memory buses) (Section 2.2) and • The memory subsystem (Section 2.3).

40. 2.1. Processors

41. 2.1Processors (1) Intel’s Tick-Tock model Key microarchitectural features TICKTOCK 2 YEARS Pentium 4 /Willamette New microarch. 11/2000 180nm TICKTOCK 2 YEARS 130nm 01/2002 Pentium 4 /Northwood Adv. microarch., hyperthreading TICKTOCK Adv. microarch., hyperthreading, 64-bit Pentium 4 /Prescott 2 YEARS 90nm 02/2004 TICK Pentium 4 / Cedar Mill 01/2006 2 YEARS 65nm New microarch., 4-wide core, 128-bit SIMD, no hyperthreading TOCKCore 2 07/2006 11/2007 New microarch., hyperthreading, (inclusive) L3, integrated MC, QPI 11/2008 01/2010 New microarch. hyperthreading, 256-bit AVX, integr. GPU, ring bus, 01/2011 Figure 2.1: Overview of Intel’s Tick-Tock model (based on [17])

42. 2.1Processors (2) Basic architectures and their related shrinks Considered from the Pentium 4 Prescott (the third core of Pentium 4) on

43. 2.1Processors (4) In 2003 Intel shifted the focus of their processor development from the performance goal to the aspect of performance per watt, as stated in a slide from 4/2006, see below. Figure 2.3: Intel’s plan to develop their manufacturing technology and processor lines revealed at a shareholder’s meeting back in 4/2006 [18]

44. 2.1Processors (5) Table 2.1: Intel’s Core 2 based and subsequent multicore DT processor lines

45. 2.1Processors (6) Table 2.2: Overview of Intel’s multicore DP server processors

46. 2.1Processors (7) Table 2.3: Overview of Intel’s multicore MP server processors

47. 2.2. Buses interconnecting platform components

48. 2.2 Buses interconnecting platform components (1) 2.2 Buses interconnecting platform components Use of buses in Intel’s DT/DP and MP platforms Buses interconnecting processors (In NUMA topologies) Buses interconnecting processors to chipsets Buses interconnecting MCHs to ICHs (In 2-part chipsets) Xeon 6500 (Nehalem-EX) (Becton) Xeon E7-2800 (Westmere-EX) or SMB SMB SMB SMB Nehalem-EX (8C) Westmere-EX (10C) Nehalem-EX (8C) Westmere-EX (10C) QPI SMB SMB SMB SMB QPI QPI SMI links SMI links DDR3-1067 DDR3-1067 7500 IOH ME SMI: Serial link between the processor and the SMB SMB: Scalable Memory Buffer with Parallel/serial conversion ESI ICH10 Nehalem-EX aimed Boxboro-EX scalable DP server platform (for up to 10 cores) Remark Buses connecting the memory subsystem with the main body of the platforms are memory specific interfaces and will be discussed in Section 4.

49. 2.2 Buses interconnecting platform components (2) Implementation of buses used in Intel’s DT/DP and MP platforms Parallel/serial bus Parallel bus Serial bus (Point-to-point interconnection) 4-bit wide (4 PCIe lanes) 64-bit wide 8-bit wide 16-bit wide Used to interconnect MCHs to ICHs in previous platforms Used to interconnect processors to chipsets or MCHs to ICHs Used to interconnect processors to processors and processors to chipsets Used to interconnect processors to chipsets in previous platforms FSB (Front Side Bus) HI1.5 QPI (Quick Path Interconnect) QPI1.1 (Quick Path Interconnect v.1.1) DMI (Direct Media Interface) ESI (Enterprise System Interface) DMI2 (Direct Media Interface 2.G.)

50. 2.2 Buses interconnecting platform components (3) Buses used in Intel’s DT/DP/MP platforms Buses interconnecting processors (In NUMA topologies) Buses interconnecting processors to chipsets Buses interconnecting MCHs to ICHs (In 2-parts chipsets) FSB (64-bit: 1993) HI 1.5 (1999) • 64-bit wide • ~150 lines • 3.2-12.8 GB/s total • in both directions • 8-bit wide • 16 lines • 266 MB/stotal • in both directions Parallel bus Low-cost systems Parallel/serial bus High-performance systems QPI (2008) DMI/ESI (2008)2 DMI/ESI (20041) QPI (2008) • 20 lanes • 84 lines • 9.6/11.72/12.8 GB/s • in each direction • 4 PCIe lanes • 18 lines • 1 GB/s/direction • 20 lanes • 84 lines • 9.6/11.72/12.8 GB/s • in each direction • 4 PCIe lanes • 18 lines • 1 GB/s/direction Serial bus QPI1.1 (2012?) DMI2 (2011) DMI2 (2011) • 4 PCIe lanes • 18 lines • 2 GB/s/direction • 4 PCIe lanes • 18 lines • 2 GB/s/direction Specification na.