Lecture 3. APIC ID

1. COM509 Computer Systems Lecture 3. APIC ID Prof. Taeweon Suh Computer Science Education Korea University

2. APIC IDs • Physical ID • APIC ID (0xFEE0_0020) register in Local APIC • 8-bit xAPIC ID or 32-bit x2APIC ID • 8-bit xAPIC ID is set by hardware but, can be changed by software • 32-bit x2APIC ID is set by hardware and can’t be changed by software • Logical ID • LDR (Logical Destination Register, 0xFEE0_00D0) • 8-bit logical xAPIC ID is assigned by software • 32-bit logical x2APIC ID is automatically generated by hardware upon entry to x2APIC Mode and is deposited in the read-only LDR

3. Physical ID Assignment in xAPIC Mode • On the trailing edge of the reset signal, the processor samples a processor design-specific set of inputs to determine • Cluster ID • Physical processor ID (also referred to as the processor package ID) • Core IDs (if it is a multicore processor package) • Logical processor IDs (if HT is supported)

4. APIC ID Register (Physical ID) Max # of APICs 15 (0xF is for broadcasting) 255 (0xFF is for broadcasting)

5. Cluster ID • The Cluster ID identifies what cluster of processors the physical processor is a member of • The processor may be assigned a cluster number of 0, 1, 2, or 3 • Address signal line [12:11]# are sampled on the trailing edge of reset to determine the cluster ID

6. Physical/Logical Processor and Local APIC ID Assignment Four Pentium 4 Xeon MP processor Example

7. Misc • The BIOS and/or the OS can change the xAPIC ID at any time after the initial IDs are automatically assigned at startup time • Software must ensure that the xAPIC ID for each Local APIC is unique • In an HT-capable processor, the Local APIC associated with each logical processor is automatically assigned a unique xAPIC ID • When a CPUID request type 1 is executed, only the xAPIC ID of the primary logical processor (logical processor 0) is returned in the EBX register

8. /proc/cpuinfo in Linux

9. /proc/cpuinfo in Linux • The /proc/cpuinfo file contains a paragraph of data for each processor on the system. There are six entries in the /proc/cpuinfo description that applies to the multi-core and Hyper-Threading Technology detection: processor, physical id, siblings, core id, cpu cores and vendor id. • The processor entry contains a unique identifier for this logical processor. • The physical id entry contains a unique identifier for each physical package. • The core id entry holds a unique identifier for each core. • The siblings entry lists the number of logical processors that exist on the same physical package. • The cpu cores entry contains the number of cores that exist on the same physical package. • All logical processors that have the same physical id share the same physical socket. Each physical id represents a unique physical package. Siblings indicate the number of logical processors that exist on this physical package. They may or may not support Hyper-Threading Technology. Each core id represents a unique processor core. All the logical processors with the same core id exist on the same processor core. If more than one logical processor has the same core id, and the same physical id, then the system supports Hyper-Threading Technology. If there are two or more logical processors with the same physical id, but different core ids, then this represents a multi-core processor. Multi-core support is also indicated by the cpu cores entry. http://software.intel.com/en-us/articles/optimal-performance-on-multithreaded-software-with-intel-tools/

10. /proc/cpuinfo in Linux • As an example, if a system contained two physical packages, each contains two processor cores that supported HT Technology, the /proc/cpuinfo file would contain this data. • This example shows that logical processors 0 & 4 reside on core 0, physical package 0. This indicates that logical processors 0 & 4 are enabled for HT Technology. The same observation can be made for logical processors 2 & 6 on core 1, package 0, logical processors 1 & 5 on core 2, package 1, and logical processors 3 & 7 on core 3, package 1. The system is enabled for HT Technology because two logical processors share the same core. Multi-core support can be determined in two ways. Since cores 0 & 1 exist on package 0, and cores 2 & 3 exist on package 1, this is a multi-core system. Also, the cpu cores entry is 2, which indicates that two cores reside in the physical package. It is a multi-processor system because there are two packages.

11. Local APIC Addressing • Physical addressing: Single Target • If an inbound message indicates physical addressing (the Destination Mode bit in the message = 0), the target of the message is specified by • In xAPIC Mode: the 8-bit xAPIC ID in the message • In x2APIC Mode: the 32-bit x2APIC ID in the message • The interrupt message is accepted only by the Local APIC whose xAPIC ID or x2APIC ID matches the ID in the message

12. Local APIC Addressing • Logical addressing: Multiple Targets • If an inbound message indicates logical addressing (the Destination Mode bit = 1), the targets of the message are specified by • In xAPIC Mode: the 8-bit logical address in the message • In x2APIC Mode: the 32-bit logical address in the message • Upon receipt of the message, the Local APICs interpret the address as follows • xAPICMode: The Local APICs interpret the message’s logical address based on the topology model indicated in their Destination Format Registers (DFRs) • 1111b: Flat Model (in Intel manual) • 0000b: Cluster Model • x2APIC Mode: Only Cluster Model is used

13. Flat Model • Not supported in x2APIC • Each of the 8 bits in the message’s logical address field acts as a selector bit permitting the message to select up to 8 Local APICs by setting the appropriate bits to one in the message’s logical address • If the message’s logical address is B3h (1011 0011), for example, it selects 5 APICs

14. Flat Cluster Model • This addressing scheme is only supported on the Pentium and P6 Processors • The high-order 4-bits of the message’s logical address contains the target Cluster ID (1 out of 15, Cluster ID Fh is reserved for broadcasting), while the lower 4-bits are used as select bits to select up to 4 Local APICs

15. Hierarchical Cluster Model • This is the only logical addressing model supported in x2APIC Mode (?) • While the Flat Cluster model is limited to no more than 15 Local APICs due to electrical loading constraints (because all of the Local APICs reside on a single APIC bus), this model assumes that up to 4 Local APICs reside on each side FSB • A hierarchical network is created by including a Cluster Routing Device on each external interface • The Cluster Routing Device uses the target cluster ID in the message’s logical address to route the message to the target cluster’s FSB or QPI • Upon message receipt, a potential target Local APIC compares the upper 4-bits of the message’s logical address with bits [31:28] of the Logical xAPIC ID field in its LDR to determine if it’s a member of the target logical cluster • Assuming it is, the Local APIC ANDs bits [27:24] of the message’s logical address with bits [27:24] of the Logical xAPIC ID field in its LDR

16. MCM (Multi-Chip Module) • Processor dies in the same package • Core 2 Quad: 2 dual-core dies packaged in a MCM • Pentium D: 2 single-core dies packaged in a MCM • Pentium Extreme Edition: 2 single-core dies packaged in a MCM Intel Clovertown (Xeon 5300)