1 / 61

The DØ Run 2b Level 2 Trigger Upgrade

The DØ Run 2b Level 2 Trigger Upgrade. James Linnemann, Michigan State University, for the DØ collaboration. Fermilab Director’s Review December 3, 2001. DØ Trigger Architecture. Level 1 Calorimeter trigger Fiber tracker trigger Preshower (e/ g) trigger Muon trigger Level 2

jacie
Télécharger la présentation

The DØ Run 2b Level 2 Trigger Upgrade

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. The DØ Run 2b Level 2 Trigger Upgrade James Linnemann, Michigan State University, for the DØ collaboration Fermilab Director’s Review December 3, 2001

  2. DØ Trigger Architecture • Level 1 • Calorimeter trigger • Fiber tracker trigger • Preshower (e/g) trigger • Muon trigger • Level 2 • Silicon track trigger • Introduce Correlations, Refine Level 1 decision • Level 3 • Full event information available • Farm of high-performance computing nodes 5kHz James Linnemann

  3. Run 2b L2 Upgrade Goals • Critical need to upgrade trigger systems • Increase trigger rejection to maintain existing rates with higher luminosity • Ensure efficient triggering for full range of Higgs channels • The challenge for Level 2 • Input rate to Level 2 limited to ~ 5 kHz (readout time) • Outputto Level 3 limited to 1.8kHz by calorimeter readout • Need to get the events to level 3, where more handles are available • Need for higher L1 rejection: moved L2 algorithms to L1 • How to maintain rejection in L2? • Do more processing in L2! • Before L=5x1032 cm-1s-1 • Beta processors to replace Alpha processors • Prepare L=5x1032 cm-1s-1 • Upgrade L2STT processor to take advantage of improved SMT • Upgrade L2 Processors with Faster Beta processors James Linnemann

  4. Why Beta Processors? • Motivation: low yield of Alpha processors • Only 15/38 worked • Mainly fabrication problems vias; BGA handling • Obsolete parts: no chance for extra production run • Need 16+8+2=26 (system + test stand + spares) for baseline system • Need test standto avoid deadtime for algorithm testing • Replace all Alphas with Betas • Commission with Alphas • No change in high-level code to run on Betas, but 2-3 times faster • Confine software changes for maximum performance to one flavor • Simpler software: advantage of raw processor speed • Pentium Linux support much better • If needed, interrupt routine much easier in Pentium • Phased Introduction of Betas • 38 production motherboards in 2002 • Designed to allow processor upgrades • 26 processors in summer 2002 • 12 additional (faster) processors: Upgrade for 2003/4 James Linnemann

  5. L2beta: 850MHz CPU; 64b 66MHz PCI • PIII Compact PCI card • 9U card with custom devices • 3 BGA’s: • Universe II VME interface • PLX 64-bit PCI interface chip • MBus and other logic in FPGA 9U board 64 bit J5 IDE 6U board <2MHz VME Compact PCI Drivers J4 UII J3 32 bits J2 66 MHz (max) 64 bits Local bus PLX 33 MHz 9656 PCI J1 Clk (s)/ FPGA Drivers roms ECL Drivers 128 bits SPY ~20 MHz MBus James Linnemann

  6. L2beta hardware advantages • Alpha is frozen in CPU power • No upgrade path • Beta benefits from Moore’s Law in cPCI marketplace •  2-3 CPU speedup over alpha gains 1.5-2 in throughput • Can use tradeoff to simplify online software • More gain with more sophisticated software • hide fixed overheads by overlapping operations • Will build all Beta motherboards • Maximum flexibility in adding CPU’s (more betas than baseline) • Could add 12 CPU’s rather than just replacing 12 • Incremental cost for 38 instead of 30 is only 20-30K (5%) James Linnemann

  7. New/Improved features of L2b • 3 CPU performance • 2 on-chip cache • Designed for Testing JTAG, done by assembler • More I/O pins • More control in interrupt/resets Cheap upgrade: add 2nd CPU (but: write new softwareto use) James Linnemann

  8. Level 2 beta Board Layout MBus Drivers VME Drivers VME Interface Voltage converters FPGA (MBus) Hard Drive PCI Interface Single Board Computer (SBC) Mounting holes For rails/stiffeners LEDs ECL out James Linnemann

  9. Level 2 Silicon Track Trigger • Vital for triggering on b-quarks • ZHbb • Zbb (top mass jet energy scale) • Goal: 98% efficiency for tracks with • PT > 1.5 GeV • Impact parameter < 2mm • L2STT 2a installation Summer 2002 • 1.8M NSF/DOE MRI funding (beyond 2a baseline) • Upgrade to accommodate design of new silicon detector • Most efficient: • build additional modules in early ’02 production James Linnemann

  10. TFC STC STC STC STC STC FRC STC STC STC STC TFC Sector 1 Sector 2 LVDS serial links for communication between boards L2STT Crate CPU spare spare VBD terminator spare spare spare terminator 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 James Linnemann

  11. Track Fit Card (illustration of motherboard design) main logic max 9 LVDS in (1 from FRC 8 from STC) link rx L3 buffer control James Linnemann

  12. Fiber Road Card receive SCL fan out L1CTT data manage L3 buffers arbitrate VME bus Silicon Trigger Card preprocess SMT data cluster into hits associate hits with CFT tracks Track Fit Card fit trajectory to hits CPU (commercial) initialization downloading monitoring resets VME Buffer Driver Readout to L3 Same as all other crates in DØ Card Flavors Glossary James Linnemann

  13. STT Status • Fiber Road Card being tested with real inputs at DØ • STC (silicon hit clusters in track roads) • Prototype I currently being tested • Second prototype being debugged • Track Fit Card tested at Stony Brook • Processed events with multiple track input roads • Running with DSP fitting code • Buffer Controller sub-card being tested Motherboard, transmitters, receivers used by all boards Link Transmitter STC Prototype I James Linnemann

  14. STT Upgrade: use all 6 SMT layers Reuse Run 2a STT (4 layers) Add 18 STC ( 2 new layers) 1 layer = full  coverage visually: 2 “layers” James Linnemann

  15. L2STT Upgrade Staging • Baseline: upgrade to use all 6 layers from new SMT • Add more Silicon Track Cards (STC) • Revise firmware and software to support new detector configuration • Add matching I/O cards to handle extra signals • Add extra track fit cards • Requires a new fanout card and replacing backplanes • Funding profile: can’t build cards during 2002 STT production run • Staging: instrument 5 layers first • Relatively inexpensive • Try to build boards from Run2a STT MRI contingency • Fewer STC and I/O cards needed • No new backplane or fanout card needed • Still need firmware and software updates • MC studies to study impact James Linnemann

  16. Level 2 Cost Summary James Linnemann

  17. L2 Schedule: Some Highlights L2 Beta (Alpha Replacement) Layout Done (except holes for rails) Prototypes (w/ Firmware) February 2002 fully funded Production, Installation July-December 2002 L2 Beta Upgrade Prototype Processor April-October 2003 Buy, Assemble and Install October 2003 - March 2004 Run 2a STT Production Start January 2002 fully funded Commissioning July 2002 Run 2b STT Stage 1 Parts Acquisition December 2001 funded Echo Board, Cables, etc. Start June 2002 9-21 mo. slack Software, Firmware start Start January 2003 12 mo. slack Test, Commission July-Sept 2004 James Linnemann

  18. Conclusions • L2 Betas replacing Alpha Processors for Level 2 Processors • Prototypes being built to replace Alphas now • Add upgraded commercial processors for Run 2b • L2STT StagedUpgrade to utilize Run 2b Silicon Tracker • Reuse all boards from Run 2a STT • Upgrade firmware, software • 5 layers first (relatively small increment) • Full 6 layers installation limited by funding profile James Linnemann

  19. DØ Trigger Architecture • Level 1 • Calorimeter trigger • Fiber tracker trigger • Preshower (e/g) trigger • Muon trigger • Level 2 • Silicon track trigger • Introduce Correlations, Refine Level 1 decision • Level 3 • Full event information available • Farm of high-performance computing nodes James Linnemann

  20. Beta Cost Summary James Linnemann

  21. Beta Schedule Summary L2 Beta (Alpha Replacement) Schematics - Done Layout – Done (except holes for rails) Device driver API – November 2001 Firmware – February 2002 Prototypes – February 2002 Hardware Integration – March 2002 System Integration – April 2002 Begin production – July 2002 Installation – November 2002 Engineering manpower: Contributed by Universities L2 Beta Upgrade Prototype Processor May-October 2003 Buy, Assemble and Install October 2003 to January 2004 James Linnemann

  22. L2STT Cost Summary James Linnemann

  23. STT Schedule Summary Run 2a STT Production Start January 2002 Run 2b should be at same time Commissioning July 2002 Run 2b STT Stage 1 Parts Acquisition December 2001 Link Echo Board Start June 2002 Backplane; Cables etc. start June 2002 about 1 year slack Software, Firmware start Jan 2003 about 6-8 months slack Test, Commission July-Sept 2004 James Linnemann

  24. No Upgrade: 4 layers Drop layers 1,3 Use only Run 2A STT hardwareonly instrument layers 0,2,4,5 James Linnemann

  25. Staged Upgrade: 5 layers first Use Run 2A STT + 6 STCinstrument layers 0,1,2,3,5 James Linnemann

  26. Comparison of old, new SMT James Linnemann

  27. Acceptance >98% of all tracks with b·2 mm and pT>1.5 GeV are contained in one 30o sector James Linnemann

  28. Sample run 2b Trigger rates • 5x1032 cm-2s-1, assuming no trigger upgrades • High pT trigger menu exceeds available bandwidth James Linnemann

  29. Calorimeter Trigger Upgrade • Rate improvements from cluster thresholds vs. tower thresholds 3 Single jet James Linnemann

  30. Conclusions • Upgrades needed at level 1 to cope with: • need for increased rejection at level 1 • Controlling rates even with multiple interactions • Triggering on correct beam crossing at 132 ns • Since last PAC meeting, extensive study of triggering options has been completed, resulting in a conceptual design for calorimeter and tracking upgrades • Scope has been limited due to constraints in schedule, manpower, and costs • Design offer considerable flexibility to deal with unanticipated physics requirements • We are ready to begin the detailed design and assemble the project • We request Stage 1 approval for the trigger upgrades James Linnemann

  31. Beta Software Issues • High level software unchanged (identical user interface) • However, alpha is more difficult to support than beta • Separate platform, and not supported by d0 environment • Very painful to support L2SIM if need for bit-by-bit verification • Supporting two platforms requires scarce high-level talent • Could run mixed system, say alphas as administrators, to help • At very least need enough betas for all workers • Interrupt processing (max performance) exceptionally complex in alpha • Simpler in beta • May not need in beta: trade CPU speed for software sophistication • Advantages to change drivers from process space to kernel space • Much harder in alpha • Much less community support for alpha • Example: active development for Linux Pentium process scheduler • May allow override of kernel much more simply than we could on alpha James Linnemann

  32. L2 Alpha Board Commissioning • First Production • 2/2 pre-production work • 7/24 production came up • (~9 months effort UIC/UM/FNAL) • Most: multiple vias fixed and a BGA replaced • 17 abandoned • Broken CIA BGA not replaceable (center of board) • DMA patches (wires, pin lifting) performed • PIO to Alpha not working (firmware) Aug/Sept • concentrate on 1 Alpha/crate Today’s score • 6 up; 3 down (2 in ICU) 3 Golden boards But 2 are pre-pro! 6/7 “good” production boards are fragile James Linnemann

  33. How many Alphas?15 + test stand in 2001 Min. Commissioning ~ 9 boards James Linnemann • Need 15 for nominal system (+7 for test stand)

  34. Where do we put our Alphas?Staging; rotating tests Aug-Sept (6 to 10 as) • 1 Maryland • 2 Test Stand/UIC • 2 Global • 2 Mu/Cal (turns?) • 3 in dry dock Oct (6 to20 as) • 2 Test stand • 2 Global • 2-4 Mu • 2-4 Cal • 2-5 CTT,PS • 1-4 UIC/Test Stand James Linnemann

  35. L2 Alpha Board b VME Interface DMA/ PIO • Biggest difficulties in • SBC section of board • Mfg. Problems • Obsolete parts • Debugging difficulty Separate SBC and IO James Linnemann ECL drivers 500 MHz  SBC

  36. Proposed Oct 2000 Baden/Hirosky • Minimize exposure to SBC difficulties • Remove dependence on short lifetime products • Maintain compatibility w/ Alpha L2beta “group” formed in Jan 2001 L2beta people: Bob Hirosky: UVa (Management, specs., device software Alpha transparency) Pierre Petroff, Philippe Cros, Bernard Lavigne: ORSAY (Management, engineering, 9U board production, prototype$) Drew Baden: UMD (Functional reqs., 1st round designs) James Linnemann

  37. James Linnemann I !like Ike!

  38. L2 Alpha Board Commissioning • Second Production ( 2 samples ): • DMA fixes incorporated in layout • Moved CIA BGA to a socket • Risky, but can’t replace this BGA if it fails • new supplier for raw boards • better assembly (failures diagnosed; site visits) • DØ pre-production at FNAL – no prompt • UM board up/down CIA SOCKET PROBLEMS • 11 to follow: earliest mid-September Probable decision: Drop socket, risk Mounting CIA James Linnemann

  39. L2beta: 933MHz CPU; 64b 66MHz PCI 9U board 64 bit J5 IDE 6U board <2MHz VME Compact PCI Drivers J4 UII J3 32 bits J2 66 MHz (max) 64 bits Local bus PLX 33 MHz 9656 PCI J1 Clk (s)/ FPGA Drivers roms ECL Drivers 128 bits SPY ~20 MHz MBus James Linnemann

  40. SBC • Single/Dual PIII up to 933MHz • 64-bit, 66MHz PCI • Mech. shock tolerance 50g for transit (immune to ‘Eisenhower effect’?) James Linnemann

  41. Linux • software compatibility! • programmer conservation PLX CFG ROM Xilinx FPGA/Verilog James Linnemann

  42. fifo_full fifo_empty dma_access Rst & Display & Clock ecl_access mb_access XCV405E pci_access mbd<127..0> plx_access MAGIC BUS mba<31..0> vme_access Display management mod_done<18..0> ev_loaded<3..0> vme_activity n_bossreq n_dstrobe mb_clk_in n_bossin gmb_clk_in Address translator bossgrin pci_clk_in n_bossout gpci_clk_in Clock management Address decoder bossgrout n_ddonein n_vme_rst_out doneout n_vme_rst_in Add mapper n_ddoneout n_sw_rst_off n_rd_wr n_sw_rst_on n_startload n_led_reset n_bufin1 Reset management State machine n_bufin0 n_bufout1 Add translator n_bufout0 fifoemptyin n_fifoemptyout n_resetin n_ads n_resetout ale n_mbmaster n_bigend LOCAL BUS INTERFACE n_crmaster n_blast n_mben breqi n_mbdatdir breqo n_mbadddir n_bterm n_den dmpaf_eot dp<3..0> n_dt_r lad<31..0> n_lbe<3..0> lhold Address decoder PIO Block TSI Block lholda vbd_start_req n_lw_r vbddone n_lserr l2_answer_ready n_ready ext_tsi_int_req n_wait PIO registers TSI registers j2_resv_out<7..0> n_linto j2_resv_in<7..0> n_dack<1..0> State machine n_gap n_dreq<1..0> n_ga<4..0> n_ccs test_out<3..0> lclk n_linti State machine State machine tsi_out<31..0> n_lb_reset useri_llocki usero_llocko pmereq TEST POINTS test_pt<31..0> James Linnemann

  43. LOCAL BUS INTERFACE 32 PCI In Address Register pci_en_wr_mem_1 n_ads pci_ en_rd_mem_1 32 ale Address decoder pci_ en_buf_mem_1 PCI Out Address Register Combinatorial Logics lad<31..0> pci_ en_buf_mem_n data_in<31..0> pci_add_in<31..0> data_out<31..0> State machine pci_add_out<31..0> pci_ en_wr pci_ en_rd n_bigend n_blast breqi breqo n_bterm n_den dmpaf_eot int_pci_mb dp<3..0> n_dt_r int_mb_pci n_lbe<3..0> lhold dma_pci_mb lholda n_lw_r dma_pci_mb n_lserr n_ready n_wait n_linto n_dack<1..0> n_dreq<1..0> n_ccs lclk n_linti n_lb_reset useri_llocki usero_llocko pmereq James Linnemann

  44. MAGIC BUS INTERFACE Magic Bus In Address Register 32 mb_en_wr_mem_1 mb_en_rd_mem_1 32 Address decoder mb_en_buf_mem_1 Magic Bus Out Address Register Combinatorial Logics mba<31..0> mb_en_buf_mem_n mb_add_in<31..0> mb_add_out<31..0> State machine mb_en_wr mb_en_rd mod_done<18..0> ev_loaded<3..0> n_bossreq n_dstrobe int_pci_mb n_bossin bossgrin int_mb_pci n_bossout bossgrout n_ddonein dma_pci_mb doneout n_ddoneout dma_pci_mb n_rd_wr n_startload n_bufin1 n_bufin0 n_bufout1 n_bufout0 fifoemptyin n_fifoemptyout n_resetin n_resetout n_mbmaster n_crmaster n_mben n_mbdatdir n_mbadddir mbd<127..0> data_in<127..0> data_out<127..0> James Linnemann

  45. Production/Assembly • Assembly by Thomson (Thales) of France • produce PCB (subcontract) • assemble components • component acquisition under study • design/manufacture of mechanical components • rails for 6U card • stiffeners for 9U card • front panel (ORSAY design) • (Mech. drawings in early December) • electrical testing (JTAG scans) • Xilinx / PLX / UII support interface James Linnemann

  46. Cost to build L2beta system ~$5450/board James Linnemann

  47. number of STCs/TFCs 14 TFC/STC slots/crate 2 TFC slots, 16 TFC/STC slots/crate James Linnemann

  48. FRC outputs SCL receiver main logic max 15 LVDS out (1 each for STC/TFC) link tx L3 buffer control James Linnemann

  49. Geometry 1 • 6 layers • rmin = 15 mm • layer 0: 128/256 strips • 44 fibers/crate  11 STCs James Linnemann

  50. Geometry 2 • 6 layers • rmin = 20 mm • layer 0: 256 strips • 44 fibers/crate  11 STCs James Linnemann

More Related