Subject: Hybrid Simulation Chain
From: <Saved by Windows Internet Explorer 7>
Date: Sun, 11 Jan 2009 17:47:12 +0200

Hybrid Simulation Chain

Purpose
Data storage
Structure of the federation
The chain itself
Code - location, compilation, linking and running
Current status

Purpose

The Hybrid Simulation Chain is a set of different pieces of software to produce simulated events in Chorus detector. It is part of the Monte Carlo simulation software of the experiment. The goal is to have coherent set of data for simulated events from both components of the apparatus: emulsion target and electronic detector. Data is stored in one logical entity - object-oriented data-base. Pieces of software for storing, retrieving and processing events from this data-base comprises the links of the chain.

In CHORUS, due to different acquisition methods involved and for historical reasons, data from emulsion target and electronic detector are processed with different software: NetScan for the former and Chant for the later. NetScan analysis package is written in C++, while Chant is a Fortran77 application. CHORUS main Monte-Carlo program, Eficass, is written in Fortran77, too. Inevitably, Hybrid Simulation Chain includes pieces of software, written in both languages.

Data Storage

Special Objectivity™ federation of type ChorusDB has been created to contain data for simulated events: of NetScan type for emulsion target and of Chant type for the electronic detector.

The system files of federation are located in our Objectivity server, chorus001d.cern.ch, in the directory /shift/chorus001d/data10/MC-production/SysFiles/. The boot file is ChorusFD, federation file - ChorusFD.FDB . Thus, the boot file name to be used in an Objectivity™ application is: chorus001d::/shift/chorus001d/data10/MC-production/SysFiles/ChorusFD .

Few database files of frequently used databases are located in directory /shift/chorus001d/data10/MC-production . All the rest is in CASTOR hierarchical tape storage system. Access to the federation is only remote, through AMS server running on chorus001d. Due to some Objectivity™ peculiarities, one cannot run application that accesses CASTOR based files on chorus001d.

Structure of the Federation

The federation has the same structure and schema as the production federation ChorusDB (for real data). Therefore, the same software can be used to process data from both federations. This has been achieved by using ooinstallfd utility. Production federation has been installed (in fact, copied) in new location. Then all unnecessary data-bases have been removed from the catalog. Data-bases with real events, based on Feb02 map file (names ending with Feb02) are used for collecting of "empty events". So, they stay in the catalog.

We have here non-standard situation when a data-base file is attached to two different, and independent, federations. This could cause problems, even data corruption, if handled without care!

For technical reasons (not suitable data-bases' ID numbers) Feb02 data-bases, produced by AnalysisReconstruction, have been copied and attached to Monte-Carlo ChorusFD. All the other Feb02 data-bases are shared between production and Monte-Carlo federations.

The Chain itself

The Hybrid Simulation Chain starts with events, simulated by Eficass and processed with Chant. Data for these events are in FZ (Zebra) files.

First link

of the chain is ReadWriteEficass package, part of chorusdb CVS module: chorusdb/examples/ReadWriteEficcass and based on Chant capabilities of writing into Objectivity™ federation.

The package loads Chant events (EVNT Zbank tree) into federation. Only events with primary (neutrino interaction) vertex in bulk emulsion sheets (emulsion itself + plastic base) are selected and loaded. Selection is based on MC truth, NOT on reconstructed primary vertex position

Chant "summary" dst of simulated events with vertex in target region are used as input. Typically, the above requirement selects ~55% of input events.

SUMM stream for MC events contains, in addition to what is there for real data, so called MC truth: most of the MCEV bank, hooked to SUMM bank. (Description of SUMM and MCEV banks can be found in /afs/cern.ch/chorus/offl/share/dev/src/bankdoc/summ/ and /afs/cern.ch/chorus/offl/share/dev/src/bankdoc/mcsim/ , respectively.) Organization of loaded Chant data is the same as for real events: one data-base per run with name ChantDataRunXXXX , where XXXX stands for run number (at least 4 digits, can be more), and one container per event. EVNT tree for each event is very simple:

EVNT -> OTHR -> SUMM

|

................-> MCEV -> MCTR

MCEV bank is retained here for historical reasons and has only one daughter, MCTR. MCTR is new bank, created on fly. Technically it is more convenient to gather what is needed from MC truth for the next step of the chain, ProcessSimulationDB , in separate bank tree. This is MCTR.

Second link

SetModule is auxiliary one. It sets the stack and module number members of the PChorusEvent object for a given event. We know from MC truth where the interaction took place and this is used to assign emulsion stack and module number to the event. These are needed for ProcessSimulationDB The code is in /chorusdb/src/tasks, setModule utility.

Third link

is ProcessSimulationDB package, part of emulsion CVS module: emulsion/netscan-analysis/Programs/Simulation/ProcessSimulationDB . ProcessSimulationDB is a specialization of ProcessSimulation code of NetScan software. It selects events with a given stack/module number and within given run and event number ranges, reads MCTR banks for selected events, generates NetScan events and writes PEventSimulation objects and NetScan raw data for them.

NetScan events can be simulated in two different ways, controlled by the bool parameter EficassDigitization = (true,false) specified in ProcessSimulationDB.ini file.

EficassDigitization = true . Hits in emulsion, stored in Eficass EMUL bank (daughter of MCEV) are used to build NetScan segments and tracks. This means that results of GEANT transport of particles through emulsion media are used.

EficassDigitization = false . Hits in emulsion are not used. NetScan segments and tracks are build based on a model for multiple scattering, coded in the package.

Important: For realistic simulation one needs simulated tracks and background. In ProcessSimulation "empty events" are used as background. Those are NetScan acquisitions for predicted but not found events in the same module. For the sake of preserving event geometry simulated segments are being overlaid in the appropriate "empty event". In fact, simulated event is translated from its original location to the location of the "empty event". Because of this, matching with electronic detector tracks can only be done by using track angles, not positions.

Major differences of ProcessSimulationDB with respect to standard ProcessSimulation are:

specialized version of event generator class, CEventGeneratorDB, is coded, which reads MC truth for simulated events from DB (MCTR bank)rather than from ASCII files. The new class has been invented as a child class of CEventGenerator. A few of base class member functions dealing with reading of MC truth data have been overwritten. Code is in emulsion/netscan-analysis/Libraries/Simulation .
EficassDigitization = true branch is fully operational;
Segments' pulse height is simulated using tables per microscope stage. The stage where event has been scanned (in fact, "empty event", used as background here) is different, in general, for different events;

Fourth link

is ProcessEvent ( emulsion/netscan-analysis/Programs/ProcessEvent). Special initialization file for processing of MC simulated events, processEventMC.ini is invented with appropriate data-bases' names. Standard ProcessEvent has to be run with this ini file to perform alignment, track building and vertex-ing (ecfsal + ecvtxa) of NetScan raw data generated with ProcessSimulationDB.

Fifth link

is AnalysisReconstruction (emulsion/netscan-analysis/Programs/Analysis/AnalysisReconstruction ). Special initialization file for processing of MC simulated events, analysisReconstructionMC.ini is created with appropriate data-bases' names. Standard AnalysisReconstruction has to be run with this ini file to perform matching of tracks in emulsion and target tracker. One needs here segment measurements' uncertainties and detector-emulsion matching parameters. These have to be calculated using ReadAnalysis + processing with Mathematica™, and put in respective ./MC_ini_files/moduleXXX.ini files (one per module, XXX being module number).

Code - location , compilation , linking and running

Source code for chain links is located in different CVS modules: ReadWriteEficass is in chorusdb, the rest is in emulsion. Respectively, two sets of libraries are needed. Makefiles are different, and of different style, too. This has been dictated by historical development of CHORUS software and it is not author's whim.

-=- ReadWriteEficass -=-

This is FORTRAN application, based on chlib and chant . One needs to checkout chorusdb/examples/ReadWriteEficass directory. FORTRAN source files are in user/src and user/comm subdirectories. Access to the Objectivity™ federation is done through calls to some C-functions that, in turn, call the stuff of chorusdb. Thus, some C++-libraries are needed on linking stage.

The available ./user_Makefile (and gmake) should be used for compilation and linking. One could set variable ROOT_DIR there if gmake is to be run outside chorusdb/examples/ReadWriteEficass . Search paths for chorusdb and NetScan libraries in the makefile are set to point to official places: $(CHORUS_ROOT)/lib/chorusdb and $(CHORUS_ROOT)/lib/netscan-analysis with CHORUS_ROOT = $(CHORUS)/dev. If the user wants to use private libraries, the above settings have to be changed. The same is true for chant and chlib include files used.

Executable file, ReadWriteEficass, is binded in ./bin directory. A script for running this executable is in ./run_load . This is usual script to run chant/chlib application with some new data cards:

*DO OBJY -iI #. control of what to write in FDB

2 #. Iwhich

1 #. Ireplace

Iwhich must be equal to 2 (Chant event), otherwize nothing will be stored. About Ireplace the situation is a bit controversial. In the last official release chorusdb_cvs-02-05 it works as described in the comments of the card. But in the last CVS version of the code deleting of existing event is disabled, due to risk of deleting off not we wanted if link ChantEvent in PChorusEvent is corrupted and does not point to the EVNT bank. In this case deleting is better to perform via removing by hand the whole run, i.e. ChantDataRunXXXX database.

*DO STACk -iI #. events with vertex in this [1-8] emulsion stack will be stored.

9 #. 0 -> all stacks [1-4]; 9 -> all stacks [5-7]

With this card one could choose emulsion stack of interest and, simultaneously, sets the range of stack numbers, i.e. 1 - 4 or 5 - 9. The stack, where interaction has taken place, is determined in the code from MC truth. Then, if the parameter here is 0, all events are stored and stack number is from 1 to 4. If it is 9, the stack number will be from 5 to 8. When one sets value from 1 to 8, only events in respective stack (1st, 2nd, 3rd, 4th if the value is 1 or 5, 2 or 6, 3 or 7, 4 or 8, respectively) are selected for loading.

*DO FINI -AC #. ini file for Objy: boot file path, DB hosts and paths

:ReadWriteEficass.ini

This card gives the name of NetScan type ini file to be read. Its minimal content is like the following:

// -*- C++ -*-

[ General ]

BootFilePath = "chorus001d::/shift/chorus001d/data10/MC-production/SysFiles/ChorusFD"

ChorusEventDirHost = "chorus001d"

ChorusEventDirPath = "/shift/chorus001d/data10/MC-production/"

[ ChantData ]

DBhost = "chorus001d"

DBpath ="/castor/cern.ch/chorus/objy-data/MC-production/ChantData"

DBhost and DBpath could be different, if one stores Chant data in a place, different from CASTOR, or in different CASTOR directory. In this case an AMS server must be running on DBhost , unless DBhost is the client machine. If one omits FINI card chorusdb default values will be used, i.e. boot file of production federation for real data and DB hosts and DB paths there. Do not omit the card!

-=- setModule -=-

The code is in chorusdb/src/tasks/setModule.cpp. Before any operations, chorusdb/setupMC.sh should be sourced. It sets paths for Objectivity™ libraries and executables, federation default boot file and variables

CHORUSDB= "Your top directory for ChorusDB work"

NETSCAN = "Your top directory for NETSCAN work"

Definitions of these two should be given according to user's local directory structure. NETSCAN could be left to point to official NetScan location, while CHORUSDB must point to local site, default is directory, from where setupMC.sh is sourced. chorusdb/src/tasks/GNUmakefile could be used for compilation and linking. setModule executable is binded in $(CHORUSDB)/bin. Command line to run it is:

setModule iniFile [[[[[[runMin] runMax]] evMin]]]] evMax]]]]]

First argument is mandatory and is the name of ini file with data cards. Optional 2nd - 6th arguments are run number min and max and event number min and max. If one wants to specify , e.g., fifth argument, he/she must give all before.

-=- ProcessSimulationDB -=-

This step is a C++ application based on NetScan ProcessSimulation. The code is in emulsion/netscan-analysis/Programs/Simulation/ProcessSimulationDB/ . Event generator class, EventGeneratorDB, is in emulsion/netscan-analysis/Libraries/Simulation/ . One should use standard NetScan tools for compilation, linking and running ProcessSimulationDB (see available documentation in emulsion/netscan-analysis/Documentation ). Ini file to be used is processSimulation.ini. Comments there could help in understanding what is read from where and what is written in where. This step processes events in a given emulsion (half)module. Selection is based on values of stack and module members of PChorusEvent class. In addition to that one can select ranges for run numbers and event numbers by setting runMin, runMax and eventMin, eventMax in processSimulation.ini. Again, PChorusEvent respective members are used and these are equal to Eficass (in fact, set by GBEAM) run/event numbers. Command line arguments, understandable by ProcessSimulationDB executable (the same as for ProcessSimulation ) are:

1st - the name (and path) of the ini file to be read, processSimulation.ini in our case.

2nd - the number of the (half)module to be processed, in three-digit format, e.g. 860.

Arguments are mandatory.

One should keep in mind that two types of data are stored in this step:

- some high-level information for the simulated event (PEventSimulation objects and event directory for them);

- NetScan raw data that can be read and processed with standard NetScan chain.

In fact, ProcessSimulationDB produces output similar to what BuildModule writes for real events.

-=- ProcessEvent -=-

The code is in emulsion/netscan-analysis/Programs/ProcessEvent. This link is just the same as for real data. Nothing has been changed except ini file. For simulated events one has to use processEventMC.ini file. Comments there help to understand data flow. Meaning of prefixes are as in ProcessSimulationDB.ini file.

Command line arguments for ProcessEvent:

1st - the name of the ini file, e.g. processEventMC.ini ;

2nd - the number of the (half)module to be processed, in three-digit format, e.g. 860;

Caveat. If one runs ProcessEvent for the second time for the same module, same events and with the same names for data-bases, he/she will not be able to run AnalysisReconstruction for this module afterward. (Segmentation fault!) Bart and I have tried to find the reason without success... The solution is to discard by hand (oodeletedb) data-bases for this module, created by ProcessEvent, and start it from scratch...

-=- AnalysisReconstruction -=-

The code is in emulsion/netscan-analysis/Programs/Analysis/AnalysisReconstruction . The link is the same as for real data. Only the ini file is different. For simulated events one has to use analysisReconstructionMC.ini . Comments there help to understand data flow. Meaning of prefixes are as in ProcessSimulationDB.ini file. Main task of this step is to perform matching of tracks in emulsion and target tracker. Segment measurements' uncertainties and detector-emulsion matching parameters are needed for that. They are specified in moduleXXX.ini files, located in directory, specified as parameter ModuleIniPath value in the ini file. Figures are to be calculated using ReadAnalysis + processing with Mathematica™. But, for that one needs output data-bases of AnalysisReconstruction. Thus, one has to run AnalysisReconstruction twice. First, with arbitrary (more or less)figures in moduleXXX.ini files. (One could use respective files for real data.) Second, with appropriate parameters, calculated for simulated events. How to obtain these parameters with ReadAnalysis and Mathematica™ is out of scope of this note.

Command line arguments for AnalysisReconstruction:

1st - the name of the ini file, e.g. analysisReconstructionMC.ini ;

2nd - the number of the (half)module to be processed, in three-digit format, e.g. 860;

These two are mandatory, next - not:

3rd - number of the run to be processed. Default - all events in the module;

4th - number of the event in this run to be processed. Default - all events in the run.

Note: To run NetScan executables, i.e. ProcessSimulationDB, ProcessEvent, AnalysisReconstruction, one can use target "run" in the (GNU)makefile like this:

cd emulsion/netscan-analysis/Programs/Analysis/AnalysisReconstruction

gmake run ARGS="analysisReconstructionMC.ini 860"

Instead of giving arguments on the gmake command line, one could edit the GNUmakefile in the current directory.

Current status of event data in Monte-Carlo federation ChorusFD

I have loaded (December 2002) 9368 (~146 per module, 64 modules, stacks 5-8) Eficass generated and Chant processed events in the federation. Chant data are in data-bases ChantDataRunXXXXX. XXXXX stands for run number. XXXXX = 7yyzz, where yy = 1 - 30, zz = 1 - 20; 600 runs altogether.

FZ files with Chant SUMM stream, used for loading, are:

/castor/cern.ch/user/z/zucv5/netscan/muccdischarmemul.ex.0105.S4.summ.fz /castor/cern.ch/user/z/zucv5/netscan/muccdischarmemul.ex.0610.S4.summ.fz /castor/cern.ch/user/z/zucv5/netscan/muccdischarmemul.ex.1115.S4.summ.fz /castor/cern.ch/user/z/zucv5/netscan/muccdischarmemul.ex.1620.S4.summ.fz

Second link, setModule has been run to set run/event members of the objects of type PChorusEvents.

Third link, ProcessSimulationDB , has been run with EficassDeigitization = true. Data-bases produced are:

EDig_MC_NetScanDir

EDig_MC_StXModYz_NetScanRaw

EDig_MC_StXModYz_eDirEventsSimulation

EDig_MC_StXModYz_EventsSimulation

X = 5 - 8, stands for stack number; Y = 1 - 8, stands for module number, z = t,b is for top or bottom half of the module. Only (half)modules that have counterparts with real data in production federation have been processed.

Fourth link, ProcessEvent , has stored, for each half-module, data-bases with names:

EDig_MC_StXModYz_EventsProcessed

Fifth link, AnalysisReconstruction , has been run only for two half-modules, 711 and 860, with parameters in ./MC_ini_files/moduleXXX.ini files calculated for MC events. Thus, events here can be used for pilot analysis and comparison. Users, who need more statistics, could run this step by themselves.

Note: Prefix EDig_ of new data-base names means that parameter EficassDigitization = true. Next prefix, MC_ , obviously indicates that these are data-bases of data for simulated events.

Roumen Tsenov, January 2003