Configuration Database Review

1. Configuration Database Review David Forrest University of Glasgow CM24 @ RAL :: 1st June 2009

2. Configuration Database • Overview of the database • Status • Future Work

3. Overview :: Scope • The configuration database deals with cabling, calibration, geometry and set value information ONLY • Its use cases are useful for offline analysis, reconstructing the configuration of the experiment • Eg “Give me the configuration of the experiment at 19:15 on 31st May 2009”

4. Overview :: Temporal Data • Imagine uploading some calibration valid for a given month. • Store the period of validity and label this the valid time. • Then later we find a better calibration for that same valid time. We want to keep both, with the same valid time. • We also store the time the information was given to the database, which is the transaction time. • Now we can get the most up to date calibration by default, but also recall the other calibrations by meta data. • Our database is bi-temporal • Example holds for geometry (eg misalignment) and cabling also. Not currently implemented for set values.

5. Overview :: API • There are many applications which will read from and write to the database • The database should not make any impositions upon their design and implementation • When something is updated it would be easier to update code in one place rather than in many places, versions, etc • So we have an API, which stands for Application Programming Interface • The API is also independent of the reading and writing applications and the database management software. • David Forrest will write the API and Database. Malcolm Ellis will write the code for G4MICE to interact with the API. Other people may be needed to write other client applications eg James Leaver for the EPICs interface.

6. API-2 Control Room Apps Database Developer DB API DB G4MICE RAL Firewall Note: Following discussions from Software session, DB will be hosted on a new machine in the control room which must be bought (see Paul K or Malcolm for details) Grid Users

7. Overview :: Implementation • We use the Postgres 8.3 Database management software. Open source. • The API is a server with code written in Java using JDBC. C++ code, etc, can of course connect to it! • All the client has to do is interpret the feedback which will be of a guaranteed agreed format • Full database has been built up incrementally, testing and prototyping. It is based at RAL, and can be accessed remotely

8. XML • It has been agreed that input/output from API will be in XML • This means we can always be backwards compatible. XML is extendable. You can put more stuff in if you have to. • API can check XML is well formed, and call the right functions itself, so potentially all the user need do is pass an XML file (eg with a geometry) so you barely need to know even what functions the API has • A well defined interface between the database and the client application means that the chance of distributed bugs over many client applications is reduced.

9. Consistency • A series of constraints will be included in the final product to maintain consistency eg Step VI cannot have three trackers. These constraints have not been completely catalogued yet. • Failure on one constraint results in rolling back the database to a state just before the constraint was violated. • It is envisioned that there will be a log, of every single operation and query on the database

10. Backup • Envisaged to make daily backups of the database contents to a box on the mice network for which the disk contents are regularly backed up • This will piggy-back on the backup strategy for other systems

11. Status: Calibrations • It has been challenging to tie down all detector groups to a calibration format • So we won’t. The idea now is to have a flexible calibration format which does not require us to know the format of a calibration in advance • Calibrations can now be a vector of values of elements of different types – double floating point number, string, boolean, integer • Meta data included + Temporal overlap

12. Calibrations - 2

13. Status: Cablings • Just getting to grips with this recently • Take detectors as roots of cabling - unless theres a good use case for the configuration DB storing detector independent cabling • Previously had a fairly loose approach • We now have detector specific cabling schemes – are these complete (so far as is related to a real use case of the config DB)? • Every component of cabling should have a unique ID, often the serial number. This is unique not only over all components of same type, but over all components. Eg so a splitter can be assigned an ID number for each channel without knowing in advance the type of device it is connecting to.

14. Cablings-2 Prelim – will likely have a couple of changes to TOF, KL, and arising from implementation issues.

15. Geometry

16. Status: Set Values • The following are known and recorded as being needed: • Magnet currents (quads, dipoles); RFCC currents; diffuser thickness; decay solenoid current(s); spectrometer solenoid currents; focus coil currents; target depth & delay

17. Set Values • We have two types of configurations with regard to set values. Run config and saved config. • Run config: This is an XML file sent from EPICS which is saved by the database. It is not parsed and read into the database structure, it is only tagged in terms of time, version and run number. It is used once for a run with a time validity. • Reference config: You can tag a configuration and use it many times. This is read into the database structure. • If some aspect of a run configuration is implemented in EPICs but not yet in the configuration database structure, this does not introduce delay – its stored as a flat file.

18. Status: API • The API exists now, and sits at RAL • It really should be moved to a box which is visible to everyone. Any offers? • It currently takes a geometry encoded in XML and translates that into SQL and uploads it to the database • XML is standardised and extendable, and highly recommended as a means of communicating with the API, which has good XML parsing functionality already • In an ideal world, for writing data, just send an xml file and the API knows what to do eg if its geometry, calibration, cabling etc.

19. Early Testing • Geometry almost fully implemented, with valid time (but not transaction time yet) and full module hierarchy • 60+ geometries were uploaded and typical questions asked about the geometry at a given time. Valid time introduces no significant slow down to Postgres. Transaction time will increase complexity only slightly • This still has to be translated from SQL response to XML, and G4MICE reconstructing the hierarchy may or may not place further obligations on the database, but current results are promising, unlikely that there will be any grievous slowdown.

20. Current State + Future Work • Geometry prototyping is going well • G4MICE will very soon be able to, as well as send a geometry to the db (already done), also reconstruct a geometry from database into memory and compare that with what it sent. • Would like to work on calibrations and, mainly, cablings, next • Once cablings are fully prototyped, much of the hardest work will be done • We may still need people to write code which sends requests to the API and digests response (no db knowledge required) • Progress at this collaboration meeting in defining seams with other systems, eg G4MICE, control room apps, EPICS, DAQ by meeting up with JSG, Malcolm, Pierrick, and others • It is likely that the db will be placed in a box in the control room – probably a new box unless a suitable one is available.

21. Project Plan • Human resource: David Forrest 50% • Long term maintenance is an open issue • Scheduled milestones • Reconstruction of Geometry: end of June • Prototyping and Testing of Cabling and Calibration : end of July • Save and recall run configurations: end of August

22. Temporal but not temperamental