1. PVManager Gabriele Carcassi Feb 18 2010

2. PVManager goals • Simplify data collection and aggregation • Simplify (UI) development • Re-use code as much as possible

3. So you want to write a UI… channel.addListener(new Listener() { public void doSomething(Event evt) { myWidget.setFoo(evt.getWhatever()); } } Toolkit is single threaded!

4. So you want to write a UI… channel.addListener(new Listener() { public void doSomething(Event evt) { Toolkit.execute(new Runnable() { myWidget.setFoo(evt.getWhatever()); } } } Lack of synchronization!

5. So you want to write a UI… channel.addListener(new Listener() { public void doSomething(Event evt) { Toolkit.execute(new Runnable() { synchronized(evt) { myWidget.setFoo(evt.getWhatever()); } } } } No change by another threadDURING/between notifications!

6. So you want to write a UI… CACHE! Aggreagate events! channel.addListener(new Listener() { public void doSomething(Event evt) { Toolkit.execute(new Runnable() { synchronized(evt) { myCopy = copy(evt.getWhatever()); myWidget.setFoo(myCopy); } } } } Need to query metadataon connection! Lacking Disconnection andREconnection notifications! Should not update UI at the same rateas data on the network!

7. Wish one could write • PV<List<VStatistics>> pv = PVManager.read(listOf(statisticsOf(vDouble(pvNames)))) .atHz(10); • pv.addPVValueChangeListener(newPVValueChangeListener() { • public void pvValueChanged() { • myWidget.setFoo(pv.getValue().getMoo()); • } • });

8. UI subsystem requirements • Single threaded • Notification must be done on event thread • Data objects must be changed on the event thread only (avoid inconsistencies and simplify responding to events) • Work must be offloaded to other threads as much as possible (to prevent unresponsiveness) • Refresh rate up to 50Hz (faster is useless and counter productive)

9. Channel Access • Multi threaded • Notification done on connection threads • Data objects changed on connection threads • Rate is not limited: aggregated can be KHz

10. Collect, aggregate and notify Notifier Collector Monitor • Need to collect the data at the source rate • Collector could be a queue, a cache, a timed cache • Aggregate the data at the UI rate, and notify on UI thread

11. Transform and calculate Notifier Collector Monitor • While we are on other thread, we compose and calculate what we need (single array for a table, FFT of a waveform, statistics, prepare synch’ed arrays, …) PV Function PV Function Cache

12. UI or not UI • Everything you can do in the UI you should be able to do at the command line • PVManager can work without a UI (notification is simply done on the timer thread) • Command line clients • Logging/archiving • Republish the data through PVData/PVAccess • PVManager is a standalone library • pvmanager.sourceforge.net

13. 3 APIs for the price of one • Data interfaces: where to put the data • Decoupled from the other two. Needed so that the client can focus more on what he needs and can work with both EPICS V3 and EPICS V. • Other talk • Building blocks API: perform computation • API based on interfaces, you can add your pieces though the goal is to have most of them already there • Expression API: define what you want to read • Internal DSL (Doman Specific Language), type safe

14. Building blocks api

15. User objects Notifier Collector Monitor PV Function PV Function Cache

16. PV<T> • Defines: • void add/removePVValueChangeListener (PVValueChangeListener listener); • String getName() ; • T getValue(); • void close(); • booleanisClosed(); • PVValueChangeListener defines: • void pvValueChanged();

17. Under the hood Notifier Collector Monitor PV Function PV Function Cache

18. Function<T> • Defines: • T getValue(); • Class<T> getType() • A function object returns a value of a particular type. • Argument number and types are determined by the constructor of an actual function.

19. Under the hood Notifier Collector Monitor PV Function PV Function Cache

20. Collector<T> extends Function<List<T>> • Defines: • void collect(); • List<T> getValue(); • Collects a bunch of values of the same type • Typically takes a Function<T> as argument • On a collect call, will take a new value • On getValue returns a number of values • Could be a queue (values returned only once) or a cache (returns the last n values, last 30 sec of values)

21. Under the hood Notifier Collector Monitor PV Function PV Function Cache

22. ValueCache<T> extends Function<T> • Defines: • void setValue(T); • You can put a value in (and read a value out)

23. Under the hood Notifier Collector Monitor PV Function PV Function Cache

24. DataSource • Defines: • void connect(DataRecipe recipe); • void disconnect(DataRecipe recipe); • DataRecipe defines: • Map<Collector, Map<String, ValueCache>> getChannelsPerCollectors() • For each collector, list of channels with their caches • To update a channel: • Lock Collector, update cache, Collector.collect()

25. Under the hood Notifier Collector Monitor PV Function PV Function Cache

26. Notification • ThreadSwitch defines: • void post(Runnable run); • It allows to execute a piece of code on a target thread • Notifier defines: • Notifier(PV<T> pv, Function<T> function, ThreadSwitchonThread); • void notifyPv(); • On notifyPV() calculates the new value, switches thread and sets the pv’s value.

27. Expression api

28. PVManager • Example: • PV<List<VStatistics>> pv = PVManager.read(listOf(statisticsOf(vDouble(pvNames)))) .atHz(10); • Defines: • static <T> PVManagerExpression<T> read(SourceRateExpression<T> pvExpression); • static <T> PVManagerExpression<T> read(DesiredRateExpression<T> pvExpression);

29. PVManagerExpression<T> • Defines: • PVManagerExpression<T> from(DataSourceconnectionManager) • PVManagerExpression<T> andNotify(ThreadSwitchonThread) • PV<T> atHz(double rate) • Default DataSource and ThreadSwitch can be specified

30. Source/DesiredRateExpression<T> • Two different types for the two different data rates • The collectors change Source to Desired: • static <T> DesiredRateExpression<List<T>> queueOf(SourceRateExpression<T> expression) • External libraries for either types or operations defines static constructors for expressions: • static SourceRateExpression<VDouble> vDouble(String name) • static <T> DesiredRateExpression<List<T>> listOf(List<DesiredRateExpression<T>> expressions)

31. Example (again) List<String> • PV<List<VStatistics>> pv = PVManager.read(listOf(statisticsOf(vDouble(pvNames)))) .atHz(10); • pv.addPVValueChangeListener(newPVValueChangeListener() { • public void pvValueChanged() { • myWidget.setFoo(pv.getValue().getMoo()); • } • });

32. Example (again) List<SourceRateExpression<VDouble>> • PV<List<VStatistics>> pv = PVManager.read(listOf(statisticsOf(vDouble(pvNames)))) .atHz(10); • pv.addPVValueChangeListener(newPVValueChangeListener() { • public void pvValueChanged() { • myWidget.setFoo(pv.getValue().getMoo()); • } • }); Which implies creating the caches for all channels

33. Example (again) List<DesiredRateExpression<VStatistics>> • PV<List<VStatistics>> pv = PVManager.read(listOf(statisticsOf(vDouble(pvNames)))) .atHz(10); • pv.addPVValueChangeListener(newPVValueChangeListener() { • public void pvValueChanged() { • myWidget.setFoo(pv.getValue().getMoo()); • } • }); Creates all the collectors, and the functions that calculate the statistics

34. Example (again) DesiredRateExpression<List<VStatistics>> • PV<List<VStatistics>> pv = PVManager.read(listOf(statisticsOf(vDouble(pvNames)))) .atHz(10); • pv.addPVValueChangeListener(newPVValueChangeListener() { • public void pvValueChanged() { • myWidget.setFoo(pv.getValue().getMoo()); • } • }); Creates the function that assembles the list

35. Example (again) PV<List<VStatistics>> • PV<List<VStatistics>> pv = PVManager.read(listOf(statisticsOf(vDouble(pvNames)))) .atHz(10); • pv.addPVValueChangeListener(newPVValueChangeListener() { • public void pvValueChanged() { • myWidget.setFoo(pv.getValue().getMoo()); • } • }); Creates the Notifier, the DataRecipe, passes it to the DataSource, …

36. Low complexity point • The API is very expressive • You can combine operators, the syntax is very compact, you specify things once • But it does not hide choices • You need to specify all transformations, including how to go from source rate to desired rate • It hides the complexity of the implementation of your choices • The API actually does something!

37. Eclipse/CSS integration • The library is packaged in org.csstudio.utility.pvmanager • Extension point based on DataSource • The activator configures the default ThreadSwitch to use SWT and default DataSource to use a CompositeDataSource made up of all extensions point • All other plugins can simply use PVManager.read() • Other plugins for extension points • org.csstudio.utility.pvmanager.sim for simulated data • org.csstudio.utility.pvmanager.epics for epics v3

38. What’s done • SimulationDataSource • Same syntax for channels as utility.pv.som • JCADataSource • Supports VDouble, VInt, VString and VEnum • Probe was ported to use PVManager (in BNL branch) • OrbitViewer prototype built on PVManager

39. What’s need to be done • JCADataSource • Mass connection can be optimized to scale better • Operators • Will need to understand which operators are useful, how to define them for “maximum combination power” • Allow to easily define “custom functions” • Add other common operators • A CALC operator? • A histogram operator? • An FFT operator? • …