Embedding deserialisation of LHC experimental data inside Field Programmable Gate Array

1. Embedding deserialisation of LHC experimental data inside Field Programmable Gate Array Speaker: T. Romanteau(projet engineering manager) Laboratoire Leprince Ringuet

2. Contribution • This project was made possible by a close collaboration of LLR, CERN and Xilinx: Ph. Busson, L. Dobrzynski, A. Karar, T. Romanteaufor LLR P. Moreirafor CERN J.L. Brelet, J.R. Macéfor Xilinx France M. Défossezfor Xilinx Benelux N.Brady, P. Clinton, M. Rochefor Xilinx Design Services Project Introduction

3. Project goals • Verify that commercial embedded Serdes inside FPGA chips can be used for LHC • Measurement of the parameters critical for the LHC detectors e.g. time latency, synchronization delay, BER and SEU. • Decomposed in two parallel project phases • Perform a measurement of the data transfer quality between GOL and a commercial chip (TLKx501). Used as a reference. • Perform a measurement of the data transfer quality between the GOL and the MGT cores embedded in the new Xilinx Virtex2Pro FPGA family. Project planning Project Introduction

4. Virtex2Pro Family Device • 10 devices • Virtex2 features & fabric • Higher memory / logic ratio • 3K+ to 125K+ logic cells • Up to 24 x 3.125 Gb/s SerDes (MGT) • Up to 4 x PowerPC 405 cores Project Introduction

5. Phase 1 : GOL / TLKx501 • Designed and Performed by LLR with commercial deserialiser chip from Texas (TLKx501) • Based on use of LVDS demo board provided by Xilinx-France with embedded Virtex2 FPGA • Tests performed at 1.6 Gbps • Measurement of critical parameters for LHC detectors e.g. time latency, BER and SEU • First step to perform testing of an other link solution • Acquire experience with the GOL test board and his internal test code GOL / TLKx501 and V2 Embedded Tester

6. Constraints and Choices • Use of coaxial wire media in first level test • As no GOL or TLK behavioral model are yet available, we use “Xilinx ILA Chipscope” tool for debugging purposes • Display result on LCD display available on “LVDS demo board” • Full hardware solution without “embedded processor” is chosen to rapidly implement platform test • Statistical measurement can be performed with external standard tool like an “oscilloscope” • Two independent clock source are used on each board. Low jitter is required GOL / TLKx501 and V2 Embedded Tester

7. GOL / TLK Test Platform GOL / TLKx501 and V2 Embedded Tester

8. First results • Emulated SEU event from GOL board is detected • No error detected after 60 hours of operation at 1.6 Gbps. In test platform the clock jitter has been characterized • Latency of the TLK2501 device was measured. After synchronization the latency stays constant but can exhibit different values. • We chose to drive the embedded tester with the recovered receiver clock. No fifo buffer required in this case. GOL / TLKx501 and V2 Embedded Tester

9. Clock Jitter Characterization GOL / TLKx501 and V2 Embedded Tester

10. Latency measurement GOL / TLKx501 and V2 Embedded Tester

11. Integrated Logic Analyzer capability GOL / TLKx501 and V2 Embedded Tester

12. Phase 1 - completion • A final release of the PCB with a physical demonstrator was available by middle July • This project was useful to acquire experience with GOL device and his existing test structure. • A previously designed IP serialiser will be tested and qualified with this platform. This IP works at 800 Mbps and uses the standard FPGA fabric. • BER versus clock jitter tradeoff must be characterized • The TLK chip must be replaced to run at 800 Mbps - future extension GOL / TLKx501 and V2 Embedded Tester

13. Phase 2 : GOL / V2Pro-MGT • Perform the necessary work to test a high speed serial transfer between the GOL device and the embedded Multi Gigabit Transceiver (MGT) included in Virtex2Pro device from XILINX • Characterize the specific timing of this kind of links and verify the compatibility with the LHC experiment requirements • LLR signed a contract with Xilinx Design Services (XDS) to create a design and perform tests based on technical requirement document provided by LLR GOL / V2Pro-MGT with Embedded Tester

14. Why subcontracting ? • To save time by overlapping the two development phases of the project. • Base the test on the LM320 (restricted access) high quality MGT characterization board provided by Xilinx US with embedded Virtex2Pro FPGA • For this new technology, internal Xilinx specialists can be accessed by XDS if necessary • Profit from the experience of XDS to design and manage a mixed hardware/software project • To consolidate our partnership with Xilinx on others aspect of our projects GOL / V2Pro-MGT with Embedded Tester

15. Xilinx Design Service • Based at Dublin in Ireland, it is the European centre for design, test and supply of FPGA components and software. 336 employees in 2002 • Xilinx Design Services (XDS) team formed at July 2000 and it’s composed of 70 engineers worldwide (40 in Europe) covering hardware and software design activities for customer applications • XDS has been in involved in many innovative design projects for the customers in the telecoms and others industry sectors • They have experiment in designing high performance hardware and software and in management of projects, both large and small

16. Requirement • In a first stage the connection between the boards was made with high quality coaxial cable, for simplification purpose • Transfer speed between GOL and LM320 board will be tested at 0.8 and 1.6 Gbps • Measurement of critical parameters for LHC detectors e.g. time latency, resynchronization time and standard BER count • Based on use of PowerPC for processing and servicing facilities with console user interface • At the completion of the project, a test report and all the material necessary to modify the design must be available GOL / V2Pro-MGT with Embedded Tester

17. V2PRO Design Architecture Power Pc Processor UART to / from RS232 Driver Processor Bus SD + MGT 1 800 Mbits/s Receiver Measurement Block MGT 2 1.6 Gbits/s Receiver SD + SD - SD - GOL / V2Pro-MGT with Embedded Tester

18. GOL / LM320 Test Platform GOL / V2Pro-MGT with Embedded Tester

19. PowerPC User Console Interface • Parametrical embedded test program with user console • Switching capability between test on 0.8 and 1.6 Gbps • Switching capability between different kind of test GOL / V2Pro-MGT with Embedded Tester

20. First Results • No synchronization loss and BER = 0 after a 67 hours run at 1.6 Gbps and 17 hours at 0.8 Gbps • The latency measured at 1.6 Gbps is bounded between 414.4 and 434.2 ns, and between 732 and 763 ns at 0.8 Gbps. This results are compatible with the use of MGT in data link path • The MGT attempts automatically to receive a comma character after synchronization loss. If internal PLL is already locked, only one comma and 4 valid characters are needed to synchronize again • Synchronization loss can be only simulated with use of MGT serialiser GOL / V2Pro-MGT with Embedded Tester

21. Phase 2 - completion • A final release of the project with a physical demonstrator has been available at end of August • Measure the bit-error rate and count losses of synchronization for various levels of clock jitter • Measure the impact of real LHC clock distribution scheme on this predefined setup • Implement and test a demonstrator with few links and his multi channel synchronization material • The GOL test board could be updated to introduce the capability to disable the output of frame. With this scheme, the real synchronization time can be measured - future extension GOL / V2Pro-MGT with Embedded Tester

22. First Conclusions • A high reliability solution can be based on use of MGT embedded inside FPGA fabric. This device can include today up to 24 MGT • This FPGA can also include an in-line specific computing process based on use of in board PowerPC (up to 4). This could be useful and versatile on data path link of LHC detectors • Using this high integration solution can have a dramatic impact on size, complexity, number and cost of PCB data board • Use of a specific serialiser IP with low latency could be also included in a FPGA based solution GOL / V2Pro-MGT with Embedded Tester

23. References • A test report of phase 2 is already available and can be accessible on demand to : romanteau@poly.in2p3.fr • Others useful documentation will be accessed soon at the WEB address : http://polywww.in2p3.fr/Electronique/cms/cms_num_new/index.html • Information and documentation about Virtex2Pro and Virtex2 are available at the WEB address : http://www.xilinx.com/products/platform/ • Xilinx Design Services can be accessed at the WEB address : http://www.xilinx.com/xds/index.htm • XDS present at CERN on 24, 25, 26 September. For contact : ludwik.dobrzynski@cern.ch GOL / V2Pro-MGT with Embedded Tester