university of Mainz
Feb, 2002, south pole
The goal of the software is to merge TWR DAQ events and muon DAQ events into a single stream of data.
This is mainly done by selecting those events of both data streams which have the closest GPS times to each
Besides the main wavemerger executable there is a second application called cleanup.wm.pl,
which basicaly does some clean up work and keeps you (the poor winterovers...) informed if something goes seriously wrong.
This will hopefully never happen. ;-)
The package constists basically of two executable files:
Since everything is quite primitive at the moment, you find all the source code and the Makefile together with the
object files and the two executables above in one directory; currently ~monitor/wavemerger.
- wavemerger - the actual application.
- cleanup.wm.pl - a little perl script that does the dirty work for us.
how to handle the WAVEMERGER package:
To start WAVEMERGER,
- log in as user monitor on bos2
- change into ~/wavemerger
- confirm that there is no other instance of wavemerger running (ps -x |grep wave)
- remove the lock file
if it is present
- say crontab -e. Below the monitoring entries, enter the following lines:
0,15,30,45 * * * * /home/monitor/wavemerger/wavemerger &> /home/monitor/wavemerger/log/msg.txt &
40 * * * * /home/monitor/wavemerger/cleanup.wm.pl &> /home/monitor/wavemerger/log/cuscr.txt &
I also added these lines to the monitoring cron_job_all file; so if you restart the monitoring, the
cron daemon will know about wavemerger, too.
The stopping procedure is a bit primitive...
- just delete the crontab entries for wavemerger by
crontab -e. (not crontab -r!!!)
- If no new input files arrive, WAVEMERGER will stop running after a while in a controlled manner. If you
can't or don't want to wait, kill it. (Look for the pid with the command
or by saying
ps ax | grep wav and ps aux | grep cleanup.
kill -9 pid
for both processes.
This, however, will result in the lock file remaining at its location. Wavemerger can not be restarted unless this file
is removed by hand.
The confoguration of WAVEMERGER is all done in its header file. There you will find everything you desire -
input directories etc. Be careful when changing directories - the last "/" is important, since acces to the files is
done by concatenating dir names and file names. If you find that the incoming directories of wavemerger grow too big,
you can decrease the time for which the incoming files are kept there. Since that task is handled by the cleanup
script, you have to configure this number there.
- location can be changed - as all other configurable
parameters - in the file wavemerger.h. The default place for it is
The Log File
Wavemerger keeps track about what it's doing
in this file. The file is always opened in append mode. If it grows too big, just delete it
while the program is NOT running. The cleanup perl script cleanup.wm.pl will also keep an eye on this.
It tries to delete the log file as soon as it grows larger than 100 Megs, however, it will only be able to
do so if wavemerger is not running at the moment of that check. A secure way is to stop wavemerger and delete the file
by hand, should that be neccessary. If you realise that the code
of wavemerger has a bug (which it surely has), want to fix it and need debug output, you can set the debug-flags
in wavemerger.h or in the code itself. A hellish load of debug output will then go into that
logfile. Depending on how closely you followed the crontab instructions above, there will also be the
This holds the short information wavemerger
screams when it is ivoked; you can check by the date and contents of this file how the last
attempt to start wavemerger ended. This does not provide much information, though. Rather go for
the central log file.
- of WAVEMERGER is done by the two classes
The File Handling
that are very similiar to each other. The most important methods in both classes are
- f2k_handler and
These methods make up the nucleus of the file handling. If the incoming events of TWR and
muon daq are very far from each other, the methods
- int <...>::get_next_event and
- int <...>::get_next_file
are called; which, however, only call the routines above in an appropriate way.
- int <...>::jump (gps_time t)
Both file handling methods keep track of the input files they have processed already in specific files
in the log directory (default: /home/monitor/wavemerger/log)
When jumping to a new input file, the new incoming file is compared to the contents of these file lists.
You can edit or delete these lists if you wish wavemerger to restart processing the files in its
The file handling routines basically assume that the incoming data files arrive ordered by time.
If this is not the case, nothing serious should happen. However, whenever the central loop described
below reaches the end of a file, this file will be considered as processed and not being opened again.
This leads to a possible loss of data, but I can't think of a managable way to avoid this.
Incoming f2k and twr files are transferred into the directories
by polechomper. WAVEMERGER writes its outgoing data files into
Further monitoring histograms are written into
The Event Merging
is handled by wavemerger.C and relies heavily on the two classes
These buffers spit out the right events once the times of the muon DAQ events reach
the time of the twr event - that is the time at which we consider to merge two events.
- f2kev_buffer and
The inner merging structure located in wavemerger.C is summarized by the following
algorithm. The parameter epsilon turns up at two positions in the code and is a very
important value to configure... the sake of our merging effort depends on it!
- while (both new f2k and new twr files are present)
| while (f2k event time < (twr event time - epsilon))
- get next f2k event and write it into buffer
- search the f2k event buffer for the event
- (1) closest to our last twr event
- (2) that has a multiplicity 80 trigger flag
- search the twr event buffer for the event
- closest to the previously selected f2k event
- if (the time gps time difference between these two events is smaller than epsilon)
- while (twr event time < f2k event time)
- get next twr event and write it into buffer
- always follow the file names of the current f2k input files. WAVEMERGER extracts year, run,
day and file number of the incoming data files and forms output file names like (for instance)
The Output Files
Except for the very last events in a f2k file, all merged events will end up in a file with the same time stamps
carried by the file they originated from. This exception happens because the output files are prepared at the same time as
the input files are opened, whereas the events are written into these files only when they run in danger to be overwritten in
the ring buffer. Besides files like the one above, you will also find names like
Such a name is given when the last present twr file is finished while the current f2k file - in our example
ab_2002_032_5153_015.data.mu.wf.f2k.do_not_transfer - is not yet fully processed.
When WAVEMERGER is opened the next time, this (the last) f2k file is still considered as not processed.
It will be opened again, as will the corresponding output file. At this point, however,
WAVEMERGER will prevent the already existing output file from being overwritten. Instead,
a different filename will be given.
The handling of output files is quite difficult to understand. Don't mess with it unless you're sure you have to!
Finally I apologize to the poor folks who undertake the effort to understand this software. Especially the buffer classes
are not very well constructed, and also the file handling is far from being perfect. However, WAVEMERGER merges, and it merges fast,
and after all, we're at south pole.
From May, you can mail to
with some hope to get an answer. ;-)