1 / 34

Play it again NewIO

Play it again NewIO. ITS Offline Piotr Krzysztof Skowroński 19 May 2003. S T A T U S. D O N E Still few features are planed to be done, but they are minor cosmetics UUID object written to every file associated with one session Quality assurance handling

carter
Télécharger la présentation

Play it again NewIO

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Play it againNewIO ITS Offline Piotr Krzysztof Skowroński 19 May 2003

  2. S T A T U S D O N E • Still few features are planed to be done, but they are minor cosmetics • UUID object written to every file associated with one session • Quality assurance handling • First we need to now how we want to implement QA itself • etc NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  3. Goal • Separation of I/O from AliRun and AliDetector • All data types goes to the separate file. • Each event is stored in separate root-directory within root file • One tree per event for each data type – no event number embedded in the tree name • Data exchange through folders. NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  4. Kine.root Event #1 Event #2 gAlice TreeK TreeK Object TPC.Hits.root Event #1 Event #2 Folder TreeH TreeD TreeD TreeH File TPC.Digits.root Event #1 Event #2 Root dir Object ITS.Tracks.root TPC.RecPoints.root ITS.Hits.root TPC.RecPoints.root ITS.Digits.root TPC.Tracks.root galice.root RunLoader TreeE Tree NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  5. Idea of Loaders • Loader is an object responsible for opening/closing files • Manages files and directory (OS as well as ROOT) names • Posting data • from file to folder – reading • and vice versa - writing • Plus some shortcut methods for accessing data from folders • e.g instead of calling dynamic_cast<TTree*> (fEventFolder->FindObject(fgkHeaderContainerName));each time, one can useloader->TreeE(); NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  6. AliLoader (ITS) AliLoader (PHOS) Architecture This object takes care of Kinematics Track References Header AliRunLoader ..... These ones takes care of any data assiciated with one detector Hits Summble Digits Digits NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  7. AliDataLoader (Kinematics) AliDataLoader (Track Refs) AliDataLoader (Header) AliDataLoader (RawCluseters) AliDataLoader (Hits) AliDataLoader (Tracks) AliDataLoader (Hits) AliDataLoader (Tracks) AliDataLoader (SDigits) AliDataLoader (RecPoints) AliDataLoader (SDigits) AliDataLoader (RecPoints) AliDataLoader (Digits) AliDataLoader (Digits) AliBasicLoader (TreeD) AliBasicLoader (TreeD) AliBasicLoader (QATask) AliBasicLoader (QATask) AliBasicLoader (AliPHOSDigitizer) AliBasicLoader (AliPHOSDigitizer) AliBasicLoader (QAObject) AliBasicLoader (QAObject) Architecture cont... It gives better scalablity Adding new type is very easy AliRunLoader AliLoader (ITS) AliLoader (PHOS) NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  8. Architecture cont. AliRunLoader One per Run ..... AliLoader (PHOS) One per Detector ..... AliDataLoader (Digits) One per Data Type ..... AliBasicLoader (TreeD) AliBasicLoader (QATask) One per Object AliBasicLoader (AliPHOSDigitizer) AliBasicLoader (QAObject) NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  9. Four types of loaders:1. AliRunLoader • Decouples all I/O management from AliRun • Manages all central data • AliRun - gAlice • Header – TreeE • Kinematics - TreeK • Stack • Track References – TreeTR • Run Loader itself • (ESD) • It stays itself in folder as well • Is written to galice.root file as an object • Manages event reloading (as an only one object) • Is an owner of AliStack and AliHeader NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  10. Four types of loaders:2. AliLoader • Manages all data related to each detector • Hits – TreeH • Summable Digids – TreeS • Digits – TreeD • Reconstructed Points – TreeR • Tracks – TreeT • AliLoader implements all standard functionality. However, if any detector needs other behaviour of loader than standard – it should create detector specific loader and overload proper methods • The same applies if extensions are needed • See example of AliPHOSLoader and AliITSLoader NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  11. Four types of loaders:3. AliDataLoader • Manages all objects associated with one data type i.e. Digits • Tree • Task pruducing data • QA • QA Task • This object manages file – normally one file is associated with one date type • Openning and closing • Name swapping • Root directories managemant NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  12. Four types of loaders:4. AliDataLoader • Basic loader (AliBaseLoader) and its derivatives (AliObjectLoader, AliTreeLoader, AliTaskLoader) • Manages one object (TObject, TTree, TTask) NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  13. Architecture • AliRunLoader is a central I/O manager • It is created in Config.C and set to gAlice object • AliRunLoader has a list of AliLoaders • Each detector registers its loader during MC initialization • Detector itself stores the pointer to its loader • Faster data access • If detector is read from file, it gets pointer to its loader automatically • AliDetector is not „owner” of loader! NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  14. AliITSLoader • ITS currently needs 2 additional data types • Raw Clusters • Back Propagated Tracks • AliITSLoader has just 2 additional Data Loaders, one for each data types above. • Thanks to modular system design all class has less than 150 lines code • If desired storing Back Propagated Tracks in file with Tracks, just add proper Tree Loader (AliBaseTreeLoader) to Data Loader responsible for Tracks, and redirect your shortcut methods properly. User will not even realize the change. NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  15. Usage • Each session should be started from loading Run Loader itself static AliRunLoader*AliRunLoader::Open(const char* filename ="galice.root", const char* eventfoldername= AliConfig::fgkDefaultEventFolderName,Option_t* option = "READ"); For example AliRunLoader *rl = AliRunLoader::Open("galice.root","MyFolder","READ");if (rl == 0x0) { ::Error("MyPlace","Can not open session for file galice.root");return; } NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  16. AliRunLoader::Open This method tries • to open file in mode specified by option • find the run loader object in file • If file option is „new”, „create” or „recreate” it creates new AliRunLoader instead of trying to read • calls AliConfig::BuildEventFolder to create folder structure for event • puts AliRunLoader in folder and returns pointer to it • in case of error returns NULL • in case session is already opened for a given file, with the same folder name and with not a contrary option, is returned Run Loader from folder. NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  17. Usage cont. (AliRunLoader) In order to load data LoadgAlice – loads AliRun LoadHeader – Loads TreeE, creates AliHeader, sets address in tree and gets header for current event LoadKinematics – Opens kine file, posts TreeK for current event to folder. If header is loaded calls AliStack::GetEvent which connects Tree. LoadTrackRefs – Opens file with track references and puts them in folder NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  18. Usage cont. (AliRunLoader) • To write data, call • WriteKinematics • WriteHeader • WriteTrackRefs • To unload data • UnloadKinematics • UnloadHeader • UnloadTrackRefs • UnloadgAlice NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  19. Usage cont. (AliRunLoader) • Current active session (thanks to Fons) • In case it is impossible to access Run Loader in some place of the code, and it is inconvenient to pass folder name or pointer to proper AliRunLoader, there is implemented Current Active Run Loader in the same style of gDirectory or gPad in ROOT AliRunLoader *rl = AliRunLoader::Open("galice.root","Event",“read"); rl->cd(); CallYourProcedure(); //..... //In your prcedure call AliRunLoader* rl = AliRunLoader::GetRunLoader() NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  20. Usage cont. (AliLoader) To load detector specific data call following methods LoadHits – loads hits for given detector LoadSDigits – loads summable digits LoadDigits – LoadRecPoints – LoadTracks – All methods above takes Option_t as a parameter, which is passed to TFile::Open NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  21. Usage cont. (AliLoader) To write detector specific data call following methods WriteHits – writes hits for given detector WriteSDigits – writes summable digits WriteDigits – WriteRecPoints – WriteTracks – They take option, which can be „OVERWRITE” to force overwriting NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  22. Usage cont. • To unload call UnloadDataType e.g. • UnloadHits • To make new container (tree) call MakeDataTypeContainer • e.g. • AliLoader::MakeHitsContainer • AliRunLoader::MakeKineContainer • Old style MakeTree(Option_t*) is supported as well • AliRunLoader::MakeTree(”K”) – makes TreeK • AliRunLoader::MakeTree(”H”) – makes TreeH for all loaders in the list • AliLoader::MakeTree(”D”) – makes TreeD for only one detector • However it does not support „TR” option since it will be recognized as Track Tree and Recostruction Tree NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  23. Usage cont. • To get access to data (trees) one can use shortcut methods like • AliLoader::TreeH() • AliLoader::TreeS() • AliRunLoader::TreeK() • AliRunLoader::TreeE() NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  24. Usage cont. (AliRunLoader) • To get loader for specific detector use • AliLoader* GetLoader(const char* name) • AliLoader* GetLoader(AliDetector* name) • Getting event: AliRunLoader::GetEvent(Int_t) • Reloads all loaded data, for exampleAliRunLoader *rl = AliRunLoader::Open("galice.root","MyFolder","READ");rl->GetEvent(0);//(1) – redundant: default is event 0rl->LoadKinematics("READ");//(2)-loads kinematicsAliLoader* tpcloader = rl->GetLoader("TPCLoader");//(3)tpcloader->LoadDigits("RECREATE");//(4)-loads Digits for TPCrl->GetEvent(1);//(5) At the moment of loading data (2),(3),(4) they are loaded for event 0 (current event). GetEvent in line (5) causes that all loaded data are reloaded for event 1. NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  25. Usage cont. (AliRunLoader) • To load detector data you can use methodAliRunLoader::LoadDataType (Option_t* detectors="all", Option_t* opt="READ")eg: • AliRunLoader::LoadHits() – calls LoadHits("READ") for all loaders • AliRunLoader::LoadHits("TPC ITS PHOS") calls LoadHits("READ") for TPC, ITS and PHOS loaders • AliRunLoader::LoadHits("all","update") calls LoadHits("update") for all loaders NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  26. Usage cont. • To obtain debugging information printout from I/O, use static method AliLoader::SetDebug(Bool_t debug); • To set up file compression level • For all files AliRunLoader::SetCompressionLevel(Int_t cl); • For all files of given detector AliLoader::SetCompressionLevel(Int_t cl); • For given data type AliLoader::SetDigitsComprLevel(Int_t cl); NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  27. Detector code Status • New I/O version is being merged with v3-09-08 • It compiles for all detectors but EMCAL • Currently excluded from Makefile • TPC, ITS and PHOS work • Take them as a practical example • display.C works as well • I already wrote some documentation up to now rather poor however...http://alisoft.cern.ch/people/skowron/codedoc/split NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  28. OLD I/O AliRun::GetEvent(Int_t) AliRun::Stack() AliRun::GetHeader() AliRun::TreeK() AliRun::TreeE() AliRun::TreeH() AliRun::TreeS() AliRun::TreeD() AliRun::TreeR() NEW I/O AliRunLoader::GetEvent(Int_t) AliRunLoader::Stack() AliRunLoader::GetHeader() AliRunLoader::TreeK() AliRunLoader::TreeE() AliLoader::TreeH() AliLoader::TreeS() AliLoader::TreeD() AliLoader::TreeR() Substitutions NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  29. Clue • How to get access to Run Loader? • Add a data member with folder name (string) • In AliDetector is not needed – it already have pointer to its AliLoader. • AliRunLoader is easily accessible from folder using static method static AliRunLoader* AliRunLoader:: GetRunLoader(const char* eventfoldername); • Set the event folder name from the place you open session – macro, manager or superior class NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  30. Practical Clue by Peter Hrisov • “If you want to do something with New I/O, and you don’t know how to prceed, just print this presentation, and apply what is written here” • Namely: • List of substitutions • Understandig architecture always helps to live with given software. NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  31. Example AliRunLoader *rl = AliRunLoader::Open("galice.root","Event",“read"); rl->LoadgAlice(); gAlice = rl->GetAliRun();//!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! AliTPC *TPC = (AliTPC*)gAlice->GetDetector("TPC"); AliLoader * tpcl = rl->GetLoader("TPCLoader"); tpcl->LoadHits("READ"); tpcl->LoadSDigits("RECREATE"); Int_t nevent = rl->GetNumberOfEvents(); for(Int_t eventn =0;eventn<nevent;eventn++) { rl->GetEvent(eventn); TPC->Hits2SDigits2(eventn); tpcl->WriteSDigits(“OVERWRITE”); } delete rl; NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  32. Conclusions • Advantages • I/O is decoupled from AliRun • User can load only desired data (+galice.root which is < 1MB) • Memory consumption management • Only really necessary files must be downloaded to perform analysis • Infinite number of sessions may co-exist, including AliRun object for each • High stability and robustness • Now all simulation and reconstruction can be done in single root session for many events for all detectors, even ITS! • Infinite number of events may be generated • Easy to control files size • User can steer how many events go to single file • No risk of file bigger then 2GB • Easy I/O bug tracing • Assures no memory leaks on I/O side NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  33. Conclusions • Advantages cont. • Easy to use • All file and string management and manipulation, object posting, file saving, current (root) directory management, data reloading, folder structure creation is decoupled from analysis code. • See how AliRunDigitizer now is cleanly implemented • No arrays of trees, no filenames management in this class • Possibility of co-existence of many copies of the same data type • Just change filename and process with new options than load in new session and compare results • Easy to extend • See AliPHOSLoader for functionality extension example • See AloITSLoader for an another data type extension • Almost no influence on performance NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

  34. Conclusions • Disadvantages • Not backward compatible • A little bit different from previous schema • ? NewIO status P. K. Skowroński ITS Offline - ITS Week May 2003

More Related