Subject: ChorusDB
From: <Saved by Windows Internet Explorer 7>
Date: Sun, 11 Jan 2009 17:17:26 +0200

The ChorusDB electronic data + emulsion scanning feedback Data-Base

1. Introduction
2. Structure of ChorusDB
3. Physical location
4. Source code
5. Access to data
a) in C++ application
b) in Fortran application via CHLIB (Chant framework)
6. List of events in ChorusDB

1. Introduction
ChorusDB is a relational data-base, where the data for CHORUS events is stored.
It uses commercial object oriented Objectivity/TM data-base.
CHORUS event is defined as group of data with the same event tag,
e.g. run number, event label and event number in this run as stamped by online Data
Acquisition System when experiment was run. Only "physics" events (label = 111) are
stored in the data-base.
For years 1994-97 we have the following runs with data:
1994:  261 - 2066
1995: 2106 - 4717
1996: 4734 - 7149
1997: 7150 - 8235

Event numbers in a given run start from 1 up to 99999.
The following types of data for an event can be stored:

1. Results of Choral reconstruction program. It has been decided that
only  Choral mini-dst (STEV bank and its tree) is to be stored.
2. Results of Chant reconstruction program. There's not yet decision
what subset of Chant maxi-dst is to be stored. For the time being full
maxi-dst (EVNT bank and its tree) is stored. Raw data banks (except of CCD) and
unpacked data banks are also there.
3. Results of emulsion scanning. Up to now two types of emulsion data
are available:
NetScan data from Nagoya and
Emulsion FeedBack (EMFB) data from Bari.

TotalScan data from Salerno will be available in  autumn 2000 and,
probably, result from scanning at CERN.
One can have more information about design ideas and evolution of
ChorusDB status with time in the following presentations:
April 1999 Software meeting; Bart Van der Vyver ( html )
December 1999 CHORUS Collaboration meeting; Roumen Tsenov ( html )
April 2000 Chorus Collaboration meeting; Roumen Tsenov ( html )
June 2000 Chorus Collaborating meeting; Roumen Tsenov ( ps, ppt )
July 2000 Chorus Collaboration meeting; Roumen Tsenov ( ps, pdf, ppt )

2. Structure of ChorusDB
The highest logical level in Objectivity/TM is federation of data
bases (FDB) The so called schema is defined on this level. The schema
is a kind of interface between physical representation of  data and
applications accessing it. Here the types of chunks (objects) of data
that one can store in this FDB are defined.
We have one FDB called ChorusDB and one schema for it. Class hierarchy, as well as all the C++ code of ChorusDB is documented with  DOC++ system.
The FDB contains one or more databases (DB). They are physically
stored as separate files.
DB contains one or more containers. In these containers one can store
objects - chunks of data of types (C++ classes) defined in the schema.
ChorusDB structure is like follows:
----------------------------------------------------------------------
|    DB          | Containers        | What does it contain          |
----------------------------------------------------------------------
ChorusEventDir    ChorusRunDir        Objects of PChorusRun type;
one per run
RunDirXXXX(*);      Objects of PChorusEvent type;
one per run                one per event

MiscEmulData        Objects of PEmulsionData type;
one per event with any type of
emulsion data available

ChoralDataXX;     RunXXXX(*);         Objects of Zbank type,
one per year          one per run           representing ZEBRA banks
of Choral dst.
ChantDataXX;      RunXXXX(*);         Objects of Zbank type,
one per year           one per run          representing ZEBRA banks
of Chant dst
EMFBDataXX;       RunXXXX(*);         Objects of Zbank type,
one per year          one per run           representing ZEBRA banks
of EMFB data
NetscanIndex      Index               Objects of type PEventRawIndex
(one per FxxRaw DB)
FxxRawYYYY(***)   (unnamed            Objects of several types
(several)           containers)        representing NetScan Data

----------------------------------------------------------------------
(*)   XXXX = run number
(**)  XX = 94,95,96,97;
(***) YYYY = four-digit number; possible range 0000 - 0999;
the sequential number of DB with raw NetScan data

3. Physical location
ChorusDB files are physically located on RAID disks mounted on a
special server, shd98.cern.ch. This server is used as federation host
and lock-server. Server is maintained by CERN IT division. Login access
to it is reserved for authorised persons only.

4. Source code
The code is predominantly C++ with small number of Fortran routines.
It is in CVS module chorusdb in our CVS repository
$CHORUS/cvsroot. Code from there can be checked out only by members of
the group v5:cvschorusdb. (Ask our system administrator <Helge> if you
need access there.) Exported version of the code normally resides in
dev area, namely $CHORUS/dev/src/chorusdb/chorusdb_cvs. One can freely
copy sources from there.   Organisation is as follows:

./src:
ChangeLog.txt    Log file for changes in the code
GNUmakefile      Makefile to build everything
setupCDB.sh      setup file that has to be "sourced" inside the current
shell (window) to set environment variables to work
with Objectivity at CERN.
./src/chant:     Fortran routines for writing Chorus type FZ structures
into ChorusDB. For more info see README.txt file there.
./src/config:    Configuration files (to be included in makefiles or to
set environment)
./src/ddl:       Part of schema (*.ddl) files for ChorusDB. They define
types of our objects (classes)
./src/kodama_ns: Another part of schema files and other routines
related to NetScan data
./src/readdb:    Fortran routines to test reading ChorusDB data through
Chant frame
./src/services:  Service classes and methods for ChorusDB (open,
close, read, write etc.)
./src/tasks:     Complete applications for various service tasks
./src/toroot:    Interface to ROOT package (not up-to-date)
./src/z2o:       Last part of schema files and a class for converting
ZEBRA bank tree into associated persistent (e.g. stored
in the data-base) objects and vise-versa
HTML documentation of the code, produced by DOC++ system, can be found here.

5. Access to data
For the time being, this can be done only from Linux machine at CERN
with RedHat5 or higher.
a) in C++ application
One can build his/her own C++ application accessing ChorusDB with the
following steps:

# copy the content of $CHORUS/dev/src/chorusdb/chorusdb_cvs somewhere (./)
# edit ./src/setupCDB.sh to define LS_HOST, FD_HOST and CHORUSDB variables
# go to ./src
# execute "source setupCDB.sh". Now one has the environment set.
# execute "gmake -f user_Makefile".
This will build up libraries:
libDDL.a
libalignment.a
libZ2O.a,
libservices.a,
libffunc_objectivity.a
in directory ./objects.
There are GNUmakefiles in every subdirectory. ./src/user_Makefile
invokes them to build these libraries. One can use them directly
from within the subdirectory.
# write application code to access the objects in ChorusDB. Examples
can be found in ./src/tasks. Also, .src/tasks/GNUmakefile can serve
as template makefile.

b) in Fortran application via CHLIB (Chant framework)

# follow the instructions for a CHANT user (CHANT manual)
to build up Fortran application with access to ChorusDB.
There are only few special things to observe here:
- don't forget to execute in your current shell:
prompt> source  $CHORUS/dev/src/chorusdb/setup.sh
- compile and link your application with Chant's user_Makefile
- put in FILI card option :CHORUSDB
- select desired run&event range with DRUN and DEVENT cards
- select which data for an event is to be read with option :DBPT
(card OPTION)
- execute your application. You'll get (hopefully) one after
the other all events for which at least part of data you
requested is available. You may not get events in
ascending order of run&event number. The order depends on
the state of the data-base. But it is guaranteed that
you'll get all events satisfied given criteria.
6. List of events in ChorusDB
# Here is the table
of runs with number of events stored and how many events with
various type of data are available.
# Here is the list
of events for which Chant data (EVNT bank tree) is available.
For time being Chant data are loaded, as they become available,
for events where NetScan data exists. Some parts of EVNT bank tree
can be missing for some reasons. Details are here.
# Here is the list
of events for which NetScan emulsion data is available. For the
moment this data is available only trough C++ application.
# Here is the list
of events for which EMFB emulsion data (EMFB bank tree) is available.

Roumen Tzenov , 27.09.2000