Status and future perspectives of the analysis language (eDSL) for HWC and LHC Operation KajetanFuchsberger TE-MPE TM, 2014-02-27 On behalf of the TE-MPE-MS Software Team: D. Anderson, M. Audrain, M. Dragu, J.-C. Garnier, A. Gorzawski, M. Koza, K.H. Krol, A. Moscatelli, K.Stamos,P.C. Turcu
Warning! 100° Visionarity Temperature might be High! (But not unrealistic ;-) VisionarityTemperature 50° 0°
eDSL ? eDSL embedded Domain Specific Language • Hwc; Systems; Accelerators; Signals;
Why a DSL at all? • Currently: • E.g. writing a PM module (or an analysis for a test) is pure coding (Java/LV). A lot of knowledge required, which distracts from the real logic: • Wheretogetthedatafrom? • What tools to use to display it? • What algorithms to apply? • A lotofcodeduplication. E.g.: • E.g. ArrayUtils in PM modules • Every application again connects in its own way to the Logging DB, PM, reads files.... • DSL focuses on thelogicofthedomain:
Why (Java) embedded? • Integration within the Accelerator SW Architecture Java • We are not language designers, but software developers. • Reuse: • All the native language features of Java, while providing the required readability for non-programmers. • Tools for writing and verification of analysis scripts (eclipse). • Strongly typed; with the help of an external library, it was easy to enforce the correct usage of physical units. • It provides the necessary possibility to constrain the user input to the available functionality.
Wherecoulditbeused? • Test Analysis withinAccTesting (origin) • PM modules • Pre/Post operational checks • periodic calls + reports: • E.g. Splice measurements • LHC statistics • …? • Any Java Application! 100° VisionarityTemperature 50° 0°
What about a better name? Brainstorming…. Zelda GENIAL ZuperEasyLanguage for Data Analysis Analysis Language for Operation and Hardware Commissioning GenericIntegrated AnalysisLanguage
Actual Status 100° VisionarityTemperature 50° 0° Courtesy: M. Audrain
Full Automation of PNO.d1 and PCC.1 Executed ~2000 times within an HWC campaign 100° VisionarityTemperature 50° 0° See Scott & Zinur‘s presentation
Whatarewemissing? • More signals from post mortem(E.g. QPS buffers) • Circuit parameters (deferred scalars) • Calculations: • All logging variables (code generation) 100° VisionarityTemperature 50° 0°
How to test? - Framework • On our (SW) side: http://sonar.cern.ch See also Cristina‘s recent Presentation:https://indico.cern.ch/event/293574/
Howtotest? – Module Developer • Use a set of known cases for which the module passes or fails. • Tobeautomated (bamboo)!
Integration in AccTesting Tocome: Add possibilitytodisplayarbitrarysignals
(mis ;-) use for data extraction • Do not care, wheredatacomesfrom! 100° VisionarityTemperature 50° 0°
Behind thescenes „compile“ Result Data request Onlyextractnecessarydata!
The Tree … Courtesy: Kamil Krol
Resolving…. Main Principle: Askthe Modules, toresolvethebiggestsubtreepossible in onego.
A more abstract view • Requirements for data Analysis: • Data incompleteness (e.g. a beam positions along the ring: added/removed BPMs) • Data invalidity: (e.g BPMs might be misbehaving) • Mathematical operations: (e.g. average/addition/scaling) • Physical units: (Different units in storage -> conversion left to the user) • Error propagation: left to the user 100° VisionarityTemperature 50° 0°
Anything out there? 100° VisionarityTemperature 50° Courtesy: K. Stamos, D. Anderson 0°
Tensorics® • Everythingis a tensor. • Common Operations. • Direct and deferred Usage. • Choosable strategies(with useful defaults): • Error propagation • Data incompleteness • Units • To be released as Open Source soon. 100° VisionarityTemperature 50° Courtesy: A. Gorzawski 0°
The dispatcher – possibleImprovements • Caching • All operations are Stateless (basically mathematical functions) easy to parallelize. 100° VisionarityTemperature 50° E. Montesnitsalis: UFO searchas an exampleUse Case 0°
Blockers & Wishes ;-) • Java 8: Closures! • Would allow us to make the language more dynamic and easily extensible. • Will be production released on 18th March. • Unfortunately blocked by CO. Will not come before end of Run 2! • Cluster! • To evaluate different options (parallelization, Scalability) • Useful also for different other R&D: PM storage, distributed file systems, test environments, R&D for future protection systems. 100° VisionarityTemperature 50° 0°
Onlymeasureddata? • Circuit Models? • Accelerator Models (MadX)? • Acquisitions (live data) • … • Beam Commissioning? 100° VisionarityTemperature 50° 0°
Summary & Outlook • Ready (from language point of view) to analyse PNO.d1 and PCC.1 • Next features: More Signals, Circuit parameters (scalars), Calculations. • Switch to Tensorics® when ready. • Optimization (Caching, Parallelization…) to come? … as soon as we can get some cluster! • Can be use in other fields (Models? Big Data analysis? Supervision? Statistics…) • Wishes/dreams: Java8 and a Cluster ;-) 100° 100° 100° 100° 100° VisionarityTemperature VisionarityTemperature VisionarityTemperature VisionarityTemperature VisionarityTemperature 50° 50° 50° 50° 50° 0° 0° 0° 0° 0°
Growing a Language • http://www.youtube.com/watch?v=_ahvzDzKdB0 • Guy Steele ‚Growing a Language‘(Keynote OOPSALA Conference, 1998) • „Ifoneletstheusershelp do thework, growthcanbe quick!“ • „ifmanypersonsworksidebysideandthebestworkisaddedwith care andgood taste, a great deal canbeadded in short time.“
Resolving…. • Ask each resolver: can resolve ? • If a resolver can resolve the given node remember. • If none of the resolvers can resolve the node, then the same procedure is recursively invoked with all the children of the node (e.g. node B in the example). • Else (at least one potential candidate resolver), the dispatcher stops the iteration through the tree in this branch and does not try to resolve the child nodes anymore (in the initial state of the example, this would happen for node C). • As soon as the iterations through all branches are the dispatcher selects one of the candidate resolvers per node, invokes it and rebuilds the tree with the resolved node. • After all resolvers return, the whole algorithm is looped starting at item (1), until all nodes (including the root node) are resolved.
The Tree … Courtesy: Kamil Krol