pvData,pvAccess,javaIOC,pvService Status EPICS Meeting

1. pvData,pvAccess,javaIOC,pvService StatusEPICS Meeting Brookhaven National Lab USA, October 2010Marty Kraimer, Guobao Shen, and Matej Sekoranja

2. pvData,pvAccess,javaIOC,pvService Outline of Talk • What are pvData, pvAccess, javaIOC, pvService? • Brief definition • Changes since EPICS meeting June 2010 in France • Very brief description of changes to pvData,pvAccess,javaIOC • Guobao will describe pvService changes in later talk • C++ implementation • pvData well underway • pvAccess starting soon • Currently no plans for ioc except • pvAccess server for V3 database • Arrays of structures • Main topic of this talk • Important for efficient implementation of pvService • Brief history • Current implementation • Some design thoughts

3. pvData,pvAccess,javaIOC,pvService Context • This 30 minute talk is two 15 minute agenda talks • PvData changes and pvAccess changes • pvData, pvAcccess, javaIOC during 2010 • Main emphasis has been to support pvService • This talk does NOT • Describe how data is presented to service client • This talk does discuss how • pvData, pvAccess, javaIOC are used to implement services • Later talks will discuss how data is presented to clients

4. pvData,pvAccess,javaIOC,pvService Accessing Source Code • At this time no official releases • Must access via sourceforge project epics-pvdata • pvData, pvAccess, pvService • Developed as Eclipse projects • Currently CVS. Later Mercurial. • Host: epics-pvdata.cvs.sourceforge.net • Repository Path: cvsroot/epics-pvdata • Modules: pvData, pvAccess, pvService • javaDoc: project and package overviews • pvDataCPP • Developed as an EPICS V3 client application • Uses EPICS build system and libCom • Mercurial • epics-pvdata.hg.sourceforge.net/hgroot/epics-pvdata/pvDataCPP

5. pvData,pvAccess,javaIOC,pvService pvData, pvAccess, javaIOC,pvService • pvData • Full support for memory resident structured data • Introspection and Data interfaces • Client sees top level PVStructure • pvAccess • Network support for pvData • javaIOC • PVRecord + database of PVRecords • Record Scanning – Periodic and Event • support – no distinction between record and device support • any field can optionally have associated support • standard: alarm, timeStamp, scan etc. • extensible – Can be used wherever appropriate • pvService • Services implemented via PVRecord + support. • ItemFinder, gather, model server, etc.

6. pvData,pvAccess,javaIOC,pvService Recent Changes to pvData, javaIOC • Two major refactoring changes • Both resulted from comments by Nikolay Malitsky • Definition of structure arrays changed • Old definition was “strange” • PVRecord and much else moved from pvData to javaIOC • pvAccess only needs top level structures NOT records • Major simplification of pvData • Everything else really belongs with javaIOC • Major simplification for pvDataCPP • Required almost NO changes to pvAccess • Moved local server implementation to javaIOC • Moved one interface from pvData to pvAccess • Really belongs there

7. pvData,pvAccess,javaIOC,pvService Changes to pvAccess • Allow client to set array capacity and length on server. • Before a client could not make a direct request. • Server automatically extended capacity and length • But client could not ask that either be made smaller. • Support for ChannelArray • Allows a client to access a sub-array. • Now allows client to set capacity and length. • Client can determine immutable fields. • Provides a “BitSet mutable”. • Only to client connect methods. • Immutable normally only set during field creation.

8. pvData,pvAccess,javaIOC,pvService Structure Arrays: Brief History • Originally pvData allowed arrays of structures and arrays of arrays • Caused major problems. No time for details. • Removed. • At BNL meeting last January • News about removal not warmly received • Re-think • If single introspection structure for all array elements then • Efficient • Major problems disappear • Has proven to satisfy pvService • At BNL meeting last July Nikolay pointed out problem • A scalarType was pvStructure. Strange. • Implementation changed.

9. pvData,pvAccess,javaIOC,pvService Structure Array Introspection Enum Type{scalar,scalarArray,structure,structureArray} Interface Field { Type getType(); String getFieldName(); ... } Interface Structure extends Field { Field[] getFields(); ... } Interface StructureArray extends Field { Structure getStructure(); // get introspection interface for element } • structureArray is a new basic type • getStructure provides the element Structure introspection interface • Every element will share the exact same introspection interface

10. pvData,pvAccess,javaIOC,pvService Structure Array Data Interfaces class StructureArrayData { PVStructure[] data; Int offset; } interface PVArray { Int getLength(); void setLength(int length); Int getCapacity(); void setCapacity(int capacity); ... } interface PVStructureArray extends PVArray { StructureArray getStructureArray(); // get introspection interface for elements int get(int offset, int length, StructureArrayData data); // get data int put(int offset,int length, PVStructure[] from, int fromOffset); //put data } • StructureArrayData provides the PVStructure[] • PVArray provides capacity and length • PVStructureArray provides • The introspection interface for the array elements • Ability to get/put arrays of PVStructure. • Each PVStructure element is a top level structure.

11. pvData,pvAccess,javaIOC,pvService Item Finder Service • Given a search string return • An array of channel names • For each channel a set of properties • Name, value, and owner • Implemented as a javaIOC record with support • Client side of service issues a channel putGet with option to process • Server record performs database search and returns result • Guobao will provide more details in later talk • This talk concentrates on data • Implementation uses array of structures

12. pvData,pvAccess,javaIOC,pvService Syntax structure structureType // structureType can be used as the type for a field type fieldName // scalar Or type[] fieldName // array ... type is a primitive type or a structureType primitive type is one of boolean, int8, int16, int32, int64, float, double, string Example structure timeStamp Int64 epochSeconds Int32 nanaSeconds structure myStruct timeStamp time double[] data ... • Syntax is NOT valid PVData syntax • It is for descriptive purposes only • More concise than Pvdata • Does not distinguish between introspection and data

13. pvData,pvAccess,javaIOC,pvService ItemFinder structure itemFinderChannel string name string[] propertyName string[] propertyValue string[] propertyOwner structure arguments string search structure result string status itemFinderChannel[] channel structure itemFinder alarm alarm timeStamp timeStamp arguments arguments result result • channel is a structureArray where each element is a itemFinderChannel • Each itemFinderChannel has fields • A string name • String arrays for propertyName, propertyValue, propertyOwner

14. pvData,pvAccess,javaIOC,pvService Client Connection To Record • Client issues a Channel.createPutGet request • Client and server each create a structure that has fields • argument • result • Client puts a value to argument.search and issues putGet request • pvAccess sends argument to server • Server gets argument, does search and puts result • pvAccess sends result back to client • Data passes between the structures created because of createPutGet

15. pvData,pvAccess,javaIOC,pvService Why Not? structure channelProperty string name string value string owner structure itemFinderChannel string name channelProperty[] property • Thus let each itemFinderChannel hold • A channelProperty array instead of separate arrays for propertyName, propertyValue, propertyOwner • Was original implementation • Reason is number of fields, which also means number of objects

16. pvData,pvAccess,javaIOC,pvService Example Number of Fields • Assume 1000 records with 10 properties per channel • First Way • 1000 channel structures • Each structure has four fields • Total = 1000*4 = 4000 fields • Second way • 1000 channel structures • Each channel has 2 fields: name and channelProperty • Each channelProperty has 10 elements • Each element has 3 fields • Total = 1000*(1 + 1*10*3) = 1000*31 = 31000 fields

17. pvData,pvAccess,javaIOC,pvService When do new fields get created? • Remember that when Channel.createPutGet is issued: • Client and server each create a structure to hold data • Arrays have capacity and length • Thus • New channels get created only when a search finds more channels • Property capacity changes when search finds more properties • Thus performance gets better with time • But still lots of Strings getting created. • In a later talk Guobao Shen will give some performance results