Paul Scherrer Institut

1. Paul Scherrer Institut Ernst Johansen The „Pie in the Sky“ PSI, 11. März 2010

2. Agenda Unified Processing Architecture Motivation Technology Feasibility Proposal Cost Discussion PSI, 11. März 2010

3. Motivation PSI Motivation We will soon operate four Machines (2x proton, 2x electron) Maintenance will become a huge Challenge Focus on synergies between all machines We need very long lifetime for the control solutions Personal Motivation Bring in +15 years of experience in designing Industrial Control Systems I like do design systems on a “decent” platform PSI, 11. März 2010

4. Strategy Vision Take initiative to get into win-win situations Proposed Strategy We should develop a common embedded control framework, reusable for all of our digital control applications. This framework should be designed for long lifetime and be based on reuse. Don’t reinvent the wheel… Pre-study results… PSI, 11. März 2010

5. Potential for Cost Reductions Cooperate with Industry to find a Common Platform Sell Platform into different Markets SwissFEL RF Applications PSI IOxOS PECTRON Process Automation PSI, 11. März 2010

6. Unified Processing Architecture ? Use Cases PLC Siemens, Rockwell, Wago, Beckhoff IOC PPC, Intel VxWorks, Linux VPC PPC405, DSP No OS, Xilkernel GPAC PPC440 SLS, HIPA DSP, uC 1. One Architecture 2. Higher Rel (ECC) 3. Lower cost (SW) 1. One Architecture 2. Higher Performance 3. Lower eng. cost 1. One Architecture 2. Higher Performance 3. Lower eng. cost 1. One Arch. 2. Retrofit 3. Compatibility 1. One Architecture 2. Synergies RF 3. Lower eng. cost UPA Common CPU, OS Common FW PSI, 11. März 2010

7. Platforms Religion…. They somehow do it all… don’t they ? Its very expensive to change them You need alliances to be strong… Technology Are there any new technologies around ? How to apply these to different platforms ? Significant Improvement PCI Express (FMC) How to apply PCI Express to VME64x ? PSI, 11. März 2010

8. Design – VME Bottleneck CPU CPU VME64x ≈250MB/s (2Gbps) FPGA FPGA FPGA PSI, 11. März 2010

9. Design – Additional Bandwidth at Low-Cost GEthernet 2x 2Gbps Duplex CPU PCIe 4x 8Gbps Duplex VME64x Card FPGA SFP 4x 15Gbps Duplex PSI, 11. März 2010

10. Design – Scalable Bandwidth COTS 10G Ethernet Switch N x 2Gbps CPU CPU CPU N x 8Gbps FPGA FPGA FPGA N x 15Gbps PCI Express, Fiber Optics N = 1 .. 21 PSI, 11. März 2010

11. Improved P0 – VME64x Compatible Improved P0 • 2mm Legacy Compatible • High-Speed Links • SFP 7Gbps SPF PCIe VersaLink PSI, 11. März 2010

12. VITA57.1-2008 FMC – High-Speed I/O Defined PSI standard, but… VITA57.1-2008 FPGA Mezzanine Cards is here • Supports the PSI Use Cases • Broad industrial acceptance • 69 x 76.5 mm2 (small) Examples • 4DSP • 8 channel ADC 250Msps @ 14-bit • 4 channel ADC 125Msps @ 16-bit • 1 channel ADC 5Gsps @ 8-bit • 4 channel DAC 1000Msps @ 16-bit • Curtiss-Wright • 4 channel DAC 500Msps @ 16-bit Proposal • VITA57.1-2008 compatibility • Support for extended PCB size where required PSI, 11. März 2010

13. EtherCAT – Medium-Speed I/O Bus Master • Based on standard Ethernet • Very low protocol overhead Key Data • Performance • 256 Digital-I/Os in 12 µs • 1.000 Digital-I/Os in 30 µs • 200 Analog-I/Os (16 Bit) in 50 µs • 100 Servo-axis in 100 µs • 12.000 Digital-I/Os in 350 µs • Suitable PCI / PMC replacement • Suitable IP-module replacement Technology • http://www.beckhoff.de PSI, 11. März 2010

14. Hardware Design Are there any reusable existing products on the market? What has to be extended ? IOxOS TOSCA II Design Kit Designed for Reuse Based on high quality TOSCA product: VME64x - PCIe Bridge Supports cost efficient Xilinx Virtex-6 CXT Architecture PCIexpress 2.0 non-blocking Switch High performance VME64x interface VHDL Simulation Framework Linux Drivers and Test Framework Excellent documentation All VHDL source and hardware design files available !! Dual Core CPU Extensions High-Rel CPU with ECC on memory (Atom has not ECC) Long term availability - Freescale Low power 45nm process PSI, 11. März 2010

15. Unified Processing Architecture – IOxOS P2020RDS reuse TOSCA II HW+FW reuse PSI, 11. März 2010

16. Unified Processing Architecture – HW Benefits High-Reliability Dual Core IOC ≈250USD Ethernet Remote debugging & download Large and fast FPGA ≈300USD FMC (Digitizers) XMC (PCIe) PMC (PCI) compatible Fast VME64x ≈100USD Electrical PCI Express (P0) High-End Workstation PSI, 11. März 2010

17. IOxOS Thermal Design Rear IO PMC / XMC / FMC Cooler PMC / XMC / FMC PSI, 11. März 2010

18. Software Design Are there any existing products on the market ? What has to be extended ? IOxOS TOSCA II Linux Drivers and Test Framework P2020RDS Linux BSP CoDeSys High-Performance soft PLC with EtherCAT Stack Linux PREEMPT_RT Open Source Real Time Linux PREEMPT_RT patch SMP up to 10kHz, AMP >10kHz EPICS Linux Port PSI, 11. März 2010

19. Linux on P2020 - Capabilities Similar to Emerson MVME4100 performance, but has additional Second CPU core for real-time DSP Real-Time PREEMPT_RT on P2020 in SMP-Mode (One operating system – two cores) Patch is easy to apply and well supported Moved all Processes to PPC0 Moved all Interrupts to PPC0 100% PPC0 load (cyclictest) Executed EPICS on PPC0 Executed cylictest on PPC1 PPC1 Results Min 5us Avg 5us 40 us Conclusions Guarding one CPU enables hard real-time behavior running EPICS in parallel Preliminary results shows that 10kHz @ 50% CPU load is feasible. PSI, 11. März 2010

20. Unified Processing Architecture – Software/Firmware UPA Rear I/O Timing Shared Mem Matlab MC SFP EPICS IOC EPICS Ethernet PCIe switch CoDeSys EtherCAT (EPICS) EtherCAT I/O Motion ≈300USD Workstation Shared Mem PCIe FMC IOxOS TOSCA II Power Units Interlock VersaLink RF PSI, 11. März 2010

21. IOxOS - Design Partner IOxOS • Swiss Company Gland by Geneva • Experts VME, PCIe, Board Design Products • VME-PCIe High-Performance Bridge • TOSCA Switch Fabric Status • NDA Signed • Offer Design UPA • Sell Into research community PSI, 11. März 2010

22. PECTRON – Power Electronics Control Status Swiss start-up company Share NRE Sell into Power Electronic Market Crate ELMA 2-Slots PLC CoDeSys SoftPLC IEC61131-3 Programming EtherCAT Beckhoff PSI, 11. März 2010

23. 3S - CoDeSys SP V3 (OEM) IEC61131-3 • De-facto Industrial Standard • Used by several hundred companies • Free Download • Supports all IEC61131-3 Languages • Easy to learn • Fast engineering • Runtime engine is licensed Status • Offer Porting to UPA PSI, 11. März 2010

24. VITA57.1-2008 FMC – Partner Strategy • Cooperate with Industry • Utilize latest available technology Curtiss-Wright • NDA • Next Generation FMC • ADC 250Msps @ 16-bit ??? 4DSP • Next Generation FMC • 6-channel ADC 250Msps @ 16-bit ??? PSI, 11. März 2010

25. Conclusions We need a long term strategy for our control systems UPA enables one single framework for PLC, IOC , Medium-Speed- and RF applications VME64x is the most stable specification and compatible with SLS and HIPA  Retrofit Higher integration we can considerably reduce the cost of VME64x The UPA has low component count and ECC – suitable for high-reliability applications With UPA we can achieve near to zero long-term royalties With strategic sourcing a lifetime of 20 years is feasible (if we want to) We get complete design in source code – including hardware and VHDL!!! There are potential partners to get into a win-win situation!!! PSI, 11. März 2010

26. The „Pie in the Sky“ PSI, 11. März 2010