Our ‘nic.c’ module

1. Our ‘nic.c’ module We create a ‘character-mode’ device-driver for the 82573L NIC to use in futrure experiments

2. Character devices • The concept of a ‘character mode’ device is that it provides a ‘stream’ of data bytes which application-programs can access with these standard Linux system-calls: • ‘open()’ and ‘close()’ • ‘read()’ and ‘write()’ • ‘lseek()’

3. Requirement • The character-mode devices are known to a Linux system by their ‘names’ and by an associated pair of ID-numbers (major and minor) normally set up by a ‘superuser’ root# mknod /dev/nic c 97 0 root# chmod a+rw /dev/nic

4. Driver module nic.c this module’s collection of ‘payload’ functions get_info() read() isr() write() ioctl() character-mode device-driver’s required ‘file-operations’ data-structure fops required pair of module administration functions module_init module_exit

5. Overview user space kernel space Linux operating system Kernel standard runtime libraries file subsystem networking subsystem char driver module application program hardware

6. Transmit operation user space kernel space Linux OS kernel runtime library file subsystem nic device-driver write() my_write() application program packet buffer user data-buffer copy_from_user() DMA hardware

7. Receive operation user space kernel space Linux OS kernel runtime library file subsystem nic device-driver read() my_read() application program packet buffer user data-buffer copy_to_user() DMA hardware

8. Our packet-format • Our ‘nic.c’ driver elects to have the NIC append ‘padding’ to any short packets • But this prevents a receiver from knowing how many bytes represent actual data • To solve this problem, we added our own ‘count’ field to each packet’s payload 0 6 12 14 actual bytes of user-data destination MAC-address source MAC-address Type/Len count

9. Using ‘echo’ and ‘cat’ • Our device-driver module ‘nic.c’ is intended to allow two programs that are running on a pair of anchor-cluster PCs to communicate via that cluster’s Local Area Network Transmitting… Receiving… $ echo Hello > /dev/nic $ _ $ cat /dev/nic Hello _

10. “Jumbo” packets • One of the innovations provided by gigabit ethernet technology is support in the NIC for larger-than-normal size data-packets • The most common upper bound for data capacity in “jumbo” frames is 9000 bytes (plus the usual ethernet frame “header”) • Our 82573L network controller implements “long packet” support as an option

11. Receive Control (0x0100) 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 R =0 0 FLXBUF 0 SE CRC BSEX R =0 PMCF DPF R =0 CFI CFI EN VFE BSIZE 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 B A M R =0 MO DTYP RDMTS I L O S LBM S L U LPE MPE UPE 0 0 SBP E N R =0 EN = Receive Enable DTYP = Descriptor Type DPF = Discard Pause Frames SBP = Store Bad Packets MO = Multicast Offset PMCF = Pass MAC Control Frames UPE = Unicast Promiscuous Enable BAM = Broadcast Accept Mode BSEX = Buffer Size Extension MPE = Multicast Promiscuous Enable BSIZE = Receive Buffer Size SECRC = Strip Ethernet CRC LPE = Long Packet reception Enable VFE = VLAN Filter Enable FLXBUF = Flexible Buffer size LBM = Loopback Mode CFIEN = Canonical Form Indicator Enable RDMTS = Rx-Descriptor Minimum Threshold Size CFI = Cannonical Form Indicator bit-value Our ‘nic.c’ driver initially will program this register with the value 0x0400801C. Later, when everything is ready, it will turn on bit #1 to ‘start the receive engine’ 82573L

12. Project 1 • Revise our ‘nic.c’ device-driver module so it will allow application programs to ‘write’ (i.e., transmit) and to ‘read’ (i.e., receive) “jumbo” packets which contain up to 9000 data-bytes, plus ethernet header and CRC • Construct a pair of application-programs that you can use (in place of ‘echo’ and ‘cat’) to test your enhanced device-driver

13. In-class demonstration • We can use our (unmodified) ‘nic.c’ driver to study the way in which our 82573L NIC uses a portion of its internal i/o-memory • We can view the NIC’s internal memory using our ‘82573.c’ character driver and our ‘fileview’ navigation-tool • We’ve also created an additional module (named ‘nicmap.c’) to assist in this study

14. Rx-Descriptor Control (0x2828) 31 30 29 28 27 26 25 24 23 22 21 20 19 18 17 16 0 0 0 0 0 0 0 G R A N 0 0 WTHRESH (Writeback Threshold) 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 0 0 HTHRESH (Host Threshold) 0 FRC DPLX FRC SPD 0 0 0 0 I L O S 0 A S D E PTHRESH (Prefetch Threshold) 0 L R S T 0 0 0 0 “This register controls the fetching and write back of receive descriptors. The three threshhold values are used to determine when descriptors are read from, and written to, host memory. Their values can be in units of cache lines or of descriptors (each descriptor is 16 bytes), based on the value of the GRAN bit (0=cache lines, 1=descriptors). When GRAN = 1, all descriptors are written back (even if not requested).” --Intel manual Recommended for 82573: 0x01010000 (GRAN=1, WTHRESH=1)

15. Exploring RX Descriptor FIFO Timeout for an in-class demonstration

16. In-class exercise • Modify the code in our ‘nicmap.c’ module so that you can use it to explore the NIC’s TX Descriptor FIFO (begins at 0x07000)