The FRS data aquisition is based on standard NIM and CAMAC electronics which process the analog detector signals, define the trigger logic, and finally digitize the signals and send them to the front-end computer. In this booklet, we assume everything is ready up to this point, so if you need help with the analog electronics (such as setting up timing, gating, trigger logic etc.) please contact some experienced FRS person!
The standard FRS front-end equipment consists of three E7 VME processors run in a Multi-Branch System (MBS) under the Lynx operating system. One E7 acts as master and the two others are slaves. The slaves each control one branch which can consist of one or more CAMAC crates. The standard FRS detectors (multi-wires, scintillators and MUSICs) are read out via the "standard" FRS branch (previously: right crate) while any user-specific detectors should be handled by the "user" USR branch (was: left crate). The standard branch also controls an additional CAMAC crate containing CFDs (for the multi-wires).
The philosophy behind the master-slave setup is to increase the maximum data throughput by dividing different tasks between the processors: the slaves control the respective readouts while the master collects all information and acts as the main event builder. The master also directly controls the taping and a server process (either a stream server or a (local) event server). The latter sends out data over the network to be sampled by on-line analysis clients like GOOSY, PAW etc.
The FRS Multi-Branch System. The data acquisition is controlled by up to three E7 processors placed in a VME crate. The readout electronics is CAMAC based, the FRS branch consisting of two CAMAC crates, #2 (normal readout electronics) and #3 (programmable constant fraction modules), and the USR branch containing crate #4.
The FRS has three Eltec E7 processors for controlling the data acquisition. One of them, E7_11, serves as "Master". The "Slave" processors (E7_5 and E7_6) each control one "branch" of data taking electronics: the FRS branch for the standard detectors, and the USR branch for experiment-specific or other not-yet-standard setups.
The VME processors are all connected to the GSI Ethernet network
via individual front-panel connectors. On the back-side, the Slave
processors are connected to their respective branches via VSB cables
(These thick white cables are very sensitive!). VSC cards mounted on
the VME crate backside serve as interfaces.
IMPORTANT: All cables and interface connectors are very
sensitive! Avoid touching and/or disconnecting, and don’t bend or
step on the cables!
Each E7 has two (blue) multi-position switches on the front panel. The top one selects the memory address offset and should never be changed except by very experienced users. The default addresses are E7_11: F, E7_5: 4, E7_6: 1. The bottom switch selects options for booting - "8" indicates automatic booting which is preferable in the case of a power failure.
A DLT tape drive is directly connected (via a flat cable) to the SCSI bus of the Master E7 processor (E7_11).
The E7s should only be booted when absolutely necessary, that is either after the power to the VME crate has been cycled, or when the data aquisition system is in a state from which it cannot be recovered via software. Although recent (December 1996) soft- and hardware changes have resulted in a fast and efficient booting process, heavy net traffic can slow the downloading of software via the Network File Server (NFS) system. Therefore only reboot those E7s which show problems! Note that in the case of a power failure, the E7s should reboot themselves automatically when power is restored.
Situations where it is necessary to boot the E7s include:
If you think something is wrong, use the VT100 terminal (next to
the tape drive) and connect it directly to one E7 at a time with
the grey cable provided for this purpose. The E7s can be manually
rebooted by flipping their front panel toggle switch downwards.
The rebooting process can be monitored on the VT100 terminal. If the
processors fail to boot even after repeated attempts, contact
DV/EE personnel (Nikolaus Kurz), as this might indicate difficulties
with either (i) the network connection, (ii) the central Network File
Server (NFS), or (iii) the E7s themselves.
NOTE: The DLT tape drive should be connected to E7_11
directly via a SCSI flat cable link. If the E7_11 has to be rebooted,
check that the tape drive is connected and turned on before
proceeding - otherwise the drive will not be recognized by the
system!!
It is beyond the scope of this Manual to describe the workings
and applications of standard NIM and CAMAC electronics modules.
There are, however, a few important CAMAC units that deserve a
closer look: the CBV crate controller and the GSI Trigger Module.
CAUTION! Remember to turn off the power to the CAMAC crates
before removing and/or inserting modules. Severe damage to the
modules and the crate can otherwise occur!
Each CAMAC crate that is to be connected to the system needs a crate controller. These handle the shuffling of data to and from the other modules over what are known as address and function lines. Controllers currently in use at GSI include CBVs and CVCs. Whereas the CBVs can do little else than coordinating the communication with modules in its crate, the CVCs are so-called intelligent controllers also capable of running data acquisition system software like MBS. The FRS standard data acquisition does not include CVCs at this time, but if you want to set up a data acquisition system for test purposes, a CVC-based setup is an alternative worth looking into!
If a branch is to contain more than one CAMAC crate, their controllers must be chained together via VSB cable connections. There are special CBV/VSB interface modules available which make this possible. It is important to terminate the VSB chain: this is done either by connecting a terminator chip to the CBV board itself or,if the last CBV module has a spare interface, a special VSB terminator connector must be used. All CBVs must be assigned a memory base address, or crate number, which is unique to the branch it is located in. This is done by setting the front panel switch to a non-zero value. Currently, the FRS branch consists of crates 2 and 3, while the USR branch controls crate 4, see the figure.
One CAMAC crate per branch must be equipped with a GSI trigger module which controls the hardware readout timing cycle. The GSI trigger modules are programmable, and during the initialization of the MBS system parameters such as the longest possible conversion time (CVT) and the fast clear time must be programmed into it. Since the branch processors are VME and the crate controllers are CAMAC, the FRS system uses the polling mode for triggering (rather than an interrupt-driven one). One trigger module in the system must be designated as the master trigger module. Only this master module should receive the trigger signal provided by the (NIM el ectronics) trigger logic setup. To ensure correct synchronization of the hardware readout, any other trigger modules in the system must be connected to the master module via a trigger bus cable.
The system allows a total of 15 different trigger sources or trigger types. Trigger types 14 and 15 are reserved by the system (start and stop of the readout, respectively), but the other 13 are available to the user. The FRS system normally uses only one trigger, trigger type 1, but in principle completely different readout patterns can be designed for different triggers, for example scalers could be read out only on trigger type 3, while energy and time information is read on trigger type 6.
The trigger type is determined by the trigger type bit pattern. The front panel of the trigger module provides four bit inputs (making it easy to set trigger types 1, 2 or 4): bit 0 in channel 0, bit 1 in channel 1 etc. The trigger module also provides some interesting output signals, including accepted triggers (output channel 0) and total dead time (channel 4). When a trigger input occurs, the trigger module checks to see if the system is busy. If it is not, the trigger word is sent on to the branch processor and an accepted trigger is output. If the system is already busy processing a previous event, the trigger is not accepted. The ratio between the free and accepted triggers is a measure of the dead time of the system. The total dead time (TDT) signal is present when the system is busy, and can therefore be used to gate other electronics. See the GSI VME Trigger Module manual for more detailed information!
Finally, dataway display modules can be helpful to diagnose CAMAC related problems. These are special units that indicate which address and function lines are in use. They also show whether the crate is inhibited etc.
The FRS data acquisition requires two separate computional environments: the (UNIX-like) Lynx system for the control of the VME processors, and the DEC Open-VMS system running on the GSI Alpha cluster (CLEX2) for the analysis clients like GOOSY. Before continuing, you should have some basic knowledge of these two environments, e.g., you should know how to login, copy and edit files, create directories and work with logical names and command files etc. Introductory information can be obtained from the GSI computer department User Services (Benutzerberatung) and from the World Wide Web ( http://www.gsi.de/computing/dvee.html).
It is recommended to use the FRS AlphaStation 200 AXP636 to run the on-line data acquisition, and a dedicated X-windows terminal for the VME processor control; both can be found in the FRS Messhütte. In addition, there are also a number of other workstations, X-terminals and alpha-numerical terminals like VT320 (this is the cheapest version) available throughout GSI. Mass storage is done on hard disks or tape drives (DLT2000) for listmode data. After the experiment, the listmode data can be stored on the GSI tape robot system ADSMCLI.
The FRS on-line analysis files are kept on either of two hard disks: all GOOSY files (except the data base) are kept on FRS$ROOT, which is cross-mounted to almost any ‘normal’ Alpha. Because of their size and since speedy access is needed, the data base files are kept on the 4GB local AXP636 disk ($36$DKA100:), which can only be accessed after logging in on AXP636.
At the FRS, the actual data acquisition from CAMAC via VME is done using the MultiBranch System MBS developed at GSI. This is based on a number of standard routines (known as tasks) and the user must only supply the actual initialization and readout functions. The following sections attempt to briefly describe the experiment-specific files needed for the data acquisition itself.
A few short words about analysis programs: The FRS on-line analysis has traditionally been performed in the framework of the GSI On-line Off-line System GOOSY. This powerful (but quite comlex) program package has been extensively documented, and copies of the various manuals can be obtained via GSI’s WWW server or from the GSI ‘Benutzerberatung’. More FRS-specific information can be obtained from Klaus Sümmerer. (The GOOSY online help is also quite useful!) Another option is to use PAW for the on-line analysis; contact José Benlliure for information on this. A third possibility is the simple histogramming utility that comes with the MBS system itself, see Chapter 10 of the GSI MBS User Manual.
A number of FRS-specific GOOSY analysis routines have been written to check the performance of the FRS standard detectors by displaying raw and calibrated information. The structure of these routines has been designed to make it easy to include experiment-specific parameters from non-standard counters such as telescopes, germanium detectors etc. in the USER routine. Again, talk to Klaus Sümmerer for more information!
Although one strictly only needs to connect to the Master E7
processor in order to run the data acquisition, it is good to
establish connections to all E7s that will be used. Therefore, open
windows on the X-terminal in the FRS Messhütte (next to the AXP636)
and establish telnet connections:
$> telnet node
Log in as profi (at the moment, no password is required). The
default directory will be /nfs/user/group/frs.
All experiments should have their own directories and MBS files. If this is the first time you log in while preparing your experiment, you need to create your own experiment-specific directories first. Ask Klaus Sümmerer or another high-ranking FRS person for the run number (xx below) allocated to your experiment.
Create an experiment-specific directory:
E7_11> mkdir /frs/usr/profi/mbsrun/runxx
and go there:
E7_11> cd mbsrun/runxx
Templates for the needed files can be copied from a previous experiment or from the set of template files is provided under /frs/usr/profi/templates. As many files contain explicit path statements, it is important to verify that all these files are updated for the current experiment!
New: Two Lynx script files have been set up to set up the necessary subdirectories and to copy the standard template files to a newly created experiment directory. (The script files themselves are located under /frs/usr/profi/templates.)
If you are only using the standard FRS detectors, make
sure you are in the experimental directory (check your location
with the Lynx pwd command) and then simply type 1brcopy.
This will copy Master and Slave files from the
/frs/usr/profi/templates/1br/frs and .../frsbranch directories.
If your setup also includes user-specific detectors (meaning you will use both branches!), instead use the 2brcopy command. (This uses files from /frs/usr/profi/templates/2br and subdirectories.)
The files needed by the Master processor (E7_11) are located in the directory .../runxx itself, while the files used by the slave processors are placed in separate subdirectories: .../runxx/frsbranch for the FRS branch Slave (E7_5) and, when needed, .../runxx/usrbranch for the USR branch files.
NOTE: All MBS files are stored on the same file system, and most of them are freely accessible from any Lynx account!!! Files can therefore be copied, edited, and also deleted by most anybody! For this reason, take great care when you modify files! Critical files can be protected with the help of the Lynx/Unix chmod command (see the GSI Unix documentation).
The following files should all be present in the respective directories. The files indicated in bold type need to be customized; they contain information (paths etc.) that has to be updated (or at least checked) for each experiment. The extension .scom indicates a MBS command file, while .usf files contain user-specific setup parameters.
| caminit.scom | List of CAMAC instructions to be executed if the CAMAC crate power has been cycled. Nodes! |
| cfdread.scom | Command file to send CAMAC instructions (parsed via e7_5) for reading the threshold and walk setting of the programmable CFDs in crate 3. |
| cfdset.scom | Command file to send CAMAC instructions (parsed via e7_5) for writing the threshold and walk setting of the programmable CFDs in crate 3. Values are beam/fragment dependent! Values! |
| closefile.scom | Closes any opened file and shows tape status and current event stats. |
| mbslog.l | Log file containing all messages passed by the message server. |
| mlstart.scom | Start-up commands for the master processor. See below! |
| mlstop.scom | Stops all E7_11 tasks. Called by stoptasks.gcom. |
| newfile.scom | See the Taping section! File names! |
| node_list.txt | Listing of all nodes in the multi-layer, multi-branch system. |
| set_ml.usf | File containing the SETUP_ML multi-layer system setup parameter set SETUP_ML. See chapter 7 of the GSI MBS User Manual. Paths! |
| start.scom | Command file to create the MBS environment. See below! Paths! |
| stoptasks.scom | Stops all tasks (except message logger) on all nodes. |
| crate2_init.c | Include-file to f_user.c, containing initializion instructions for the FRS branch CAMAC modules. CNAFs! | ||
| crate2_read.c | Include-file to f_user.c, containing readout instructions for the FRS branch CAMAC modules. CNAFs! | ||
| f_user.c | Main user-defined readout program. Written in the C language. | ||
| f_user.h | Include-file containing some declarations used by f_user.c. | ||
| Makefile | Script file that compiles f_user.c and links it to standard object file libraries, producing a m_read_meb FRS branch master event builder code. | ||
| m_read_meb | The executable produced by make’ing f_user.c. | ||
| setup_mbs.usf | File containing the SETUP single-branch (slave subsystem) parameter set (CAMAC address definitions, trigger module parameters,...) See chapter 6 of the GSI MBS User Manual. | ||
| start_with_prm.scom | Start-up commands for the FRS branch slave processor.
See below!
Paths!
| stop_with_prm.scom
| Stops the individual E7_5 tasks. Called by
stoptasks.scom
| |
| crate4_init.c | Include-file to f_user.c, containing initializion instructions for the FRS branch CAMAC modules. CNAFs! | ||
| crate4_read.c | Include-file to f_user.c, containing readout instructions for the FRS branch CAMAC modules. CNAFs! | ||
| f_user.c | Main user-defined readout program. Written in the C language. | ||
| f_user.h | Include-file containing some declarations used by f_user.c. | ||
| Makefile | Script file that compiles f_user.c and links it to standard object file libraries, producing a m_read_meb USR branch master event builder code. | ||
| m_read_meb | The executable produced by make’ing f_user.c. | ||
| setup_mbs.usf | File containing the SETUP single-branch (slave subsystem) parameter set (CAMAC address definitions, trigger module parameters,...) See chapter 6 of the GSI MBS User Manual. | ||
| start_with_prm.scom | Start-up commands for the USR branch slave processor.
See below!
Paths!
| stop_with_prm.scom
| Stops the individual E7_6 tasks. Called by
stoptasks.scom
| |
This command file calls the other nodes in the multi-layer multi-branch system, one by one, and executes "local" command files containing the actual startup commands. At the FRS, the master node startup file is .../runxx/ml_start.scom, while the slave node startup commands are contained in the files start_with_prm.scom located in the respective branch subdirectories.
This file first starts the m_util task and then loads the multi-layer system setup parameter set SETUP_ML contained in the file set_ml.usf. If the master processor needs to control CAMAC crates directly (usually NOT the case), the SETUP parameter set file setup_mbs.usf must also be loaded. The clear daq counter command resets the event number and rate counters. Subsequently, the m_transport and m_collector tasks are started, followed by one of the (mutually exclusive) server tasks m_event_serv (local event server) or m_stream_serv (for use with a remote event server). The optional m_esone_serv server task (for CAMAC control) is started last.
These slave processor startup files commence by starting the m_util task and loading the setup_mbs.usf file. The next step differs slightly between the "master" slave E7_5 (which controls the branch containing the master trigger module) and the other, E7_6. For the latter branch, the trigger module should be set up in slave mode: set trig_mod -slave (default is as master!). Since the triggering of the FRS readout is based on polling instead of being interrupt-driven, the interrupt must be disabled (disable irq). Then the respective branch master readout task m_read_meb is started, followed by the m_esone_serv task which enables sending CAMAC instructions directly to the crates included in the setup_mbs.usf file.
The user has to provide the data acquisition system with information about what action should be taken upon startup, when a trigger comes, etc. This information can be provided either in the form of a readout table or, as is the case at the FRS, in a readout function. This function, typically named f_user.c, is written in the C programming language. It consists of a standardized framework into which the user can add experiment-specific instructions for initialization and readout.
To make it easier to adopt the initialization and readout sequences for the individual experiments, the respective CAMAC calls have been broken out of f_user.c, into the include files cratex_init.c and cratex_read.c, where x=2 for the FRS standard detector crate and x=4 for the USER crate. In many cases, the standard crate setup can be left as it is, but it is best to check that all necessary parameters are being read out! It is very useful to start the include files with so-called "stuffing schemes", i.e., a sequential (ordered after station number) listing of all the CAMAC modules in use, their type, GSI serial number and which detector signals were recorded. This listing can then serve as a record after the experiment, indicating which modules were used for what purpose.
A selection of CNAF sequences for commonly used CAMAC modules is listed in the Appendix.
The initialization and read out sequence now has to be adjusted to the actual setup. The init-files contain all initializing commands required like fast clear, scaler resets, LAM clears, etc. The syntax depends a little bit on whether any value (such as thresholds) needs to be written to the module or a simple CAMAC call (e.g., a clear) is issued. It is important to keep in mind that different pointer functions are used for the different branches; don’t mix codes for the FRS standard branch with the USR branch!
| m_dispatch | M, S | Controls the components running in its environment, dispatching MBS commands to the respective tasks that executes them. The dispatcher hibernates until it receives an acknowledge message from the task. |
| m_msg_log | M, S | The message logger process. Accepts messages and writes them to the log file and/or to th eterminal. The message logger on the node running the prompter acts as a server, connecting via TCP to client message loggers on other nodes. |
| m_prompt | M | The prompter program itself. Can be left temporarily with exit or quit. |
| m_read_meb | S | The master readout runs on the branch controller nodes. The readout can be specified using tables or user-written functions (FRS). The executable m_read_meb is loaded at the startup of the acquisition. |
| m_collector | M | Composes events from the subevent pipes filled by the readout tasks, and formats them into GSI buffers before passing them to the m_transport task. |
| m_transport | M | Writes all data buffers to tape or disk, network (via TCP), or both. The TCP channel must be activated by the enable tcp command. The output channels are synchronous. |
| m_event_serv | M | Sends samples of the data to a connected client. Must be activated after startup by start event_serv. Exclusive to the m_stream_serv task. |
| m_stream_serv | M | Writes list mode data samples (streams) to connected clients. Used with remote event servers. Exclusive to the m_event_serv task. |
| m_esone_serv | (M), S | Executes CNAF functions. |
| m_util | M, S | Executes most of the MBS commands. |
The Master/Slave system is easy to control from one terminal window via a single process. This prompter program (started from the Lynx prompt by prm), which runs on the E7/network file server system under the Lynx operating system, coordinates the different MBS tasks and the information flow between them. Starting the prompter also invokes the message server, the remote dispatchers and message logger clients. It can connect or disconnect remote dispatchers as well as send commands to these.
The nodes of the environments to be controlled by the prompter must be listed in the text file node_list.txt. (Don’t change this name - several processes look for this file by default!) The prompter sends commands to specified nodes via TCP, hibernating until it gets an acknowledge message from the remote dispatcher. The prompter can be left temporarily by the commands exit or quit -- NEVER type ctrl-Z! -- and then reentered again by typing prm at the Lynx prompt.
Once in the prompter, the data acquisition environment is
started by executing the command file start.scom:
E7_11> @start
This calls other command files containing the startup sequences for
the E7 processors, see above. It is important
to check for error messages during the startup process.
If everything is OK, proceed by initializing the CAMAC crates:
E7_11> @caminit
and resetting the thresholds and walks of the programmable constant
fraction discriminator modules in crate 4:
E7_11> @cfdset
The data acquisition is controlled by the node in which the
master trigger module is residing. This is normally the FRS branch
node, controlled by E7_5. For this reason, the command to start
the data acquisition process must be sent via this node.
This can be done in two ways:
E7_11> e7_5::start acquisition
OR
E7_11> set dispatcher e7_5
E7_5> start acquisition
Note how, in the latter example, the prompter changes from "E7_11>"
to "E7_5>" because the dispatcher node is changed. To return to the
E7_11, simply use the set dispatcher command again! The
"double-colon method" provides a convenient shortcut, and will be
used in the following.
The acquisition is correspondingly halted with the stop
acquisition command, again issued via E7_5:
E7_11> e7_5::stop acquisition
To occasionally check the rate, use the show rate
command:
E7_11> show rate 5
shows the total number of buffers and events since the start of
acquisition as well as buffer, event and kilobyte rates averaged
over the time interval chosen (here 5 seconds).
The acquisition rate can also be continuosly monitored with the
rate command:
E7_11> rate -tn
gives similar information, updated every n seconds. The
monitoring can be exited with Ctrl-C.
The contents of the events themselves can be monitored with the
type event command. The optional -v qualifier selects
the verbose listing.
E7_11> type event -v
The output lists the event and subevent headers, from which event
number and length can be deduced. The verbose listing also includes
the actual data (in hexadecimal notation!) in the order of
the readout. This can be very useful for diagnostics purposes, and
also provides an at-a-glance overview of the data stream. The
experienced user can, based on knowledge of the parameter order,
immediately check the performance of specific detectors by checking
their presence in the event.
NOTE: Since the Master node, normally E7_11, controls the event building and data transfer, all rate and event monitoring commands must be issued via it.
Sometimes it is necessary to directly communicate with
CAMAC electronics modules, e.g., to read status words, thresholds
etc. This is made possible via the esone server tasks that run
on the Slave processors, with which CNAFs (crate, number, address,
function) can be sent. The Appendix lists
commonly used CNAFs for some typical modules used at the FRS.
Example: Issue an F(2)A(0) read-and-clear command to the
pattern unit in the FRS crate:
El_11> e7_5::camac cnaf 2 20 0 2
NOTE: The FRS crate is controlled via E7_5, hence the "e7_5::"
prefix. The FRS standard crate is number 2, and the pattern unit sits in slot 20.
If you started the data aquisition successfully, all detectors are operating and everything is OK, you may wish to write some listmode data to tape. For this purpose, a DEC Linear Tape (DLT) drive, supporting compression, is directly connected to the Master E7 processor (E7_11) via a SCSI cable. Please note that the SCSI address number of the DLT drive (as set on the back) must be 4! The drive is usually placed next to the VT100 terminal used for direct communication with the E7s.
NOTE: The taping is now completely independent of any on-line analysis clients! The rate at which events can be written to tape is, in principle, limited either by the overall speed of the acquisition processes (dependent on the memory available to the master E7, the number of parameters, etc.) or by the maximum bytes/second limit of the tape drive/taping process. The data acquisition process itself is not affected by the taping process (or by the on-line analysis clients).
The MBS system writes the listmode files in ANSI format using big endian ordering. Since computers running under OpenVMS (Alphas and VAXes) use another standard format, programs reading the data files must be capable of performing byte swapping! As an example, this affects the GOOSY and PAW unpacking routines!
The tapes have to be initialized and mounted from within a running
Prompter (prm) session. If a previously used tape is mounted, the
system will proceed to the EOT (last EOF) marker before any new data
is written to it. If you want to use a new tape, or reuse an old one
(ALL old data will be lost!), you must first initialize the tape and
give it a label (max. 6 alphanumeric characters):
E7_11> init tape label
It is a good rule to label all tapes from a given run with Rxx_01 etc, where xx is the run number.) The initialization takes some time - be patient. If you think something is wrong, check for blinking lights on the DLT drive front panel! If you receive error messages, it could be because you were too quick issuing the INIT command after inserting the tape - the drive takes some time to recognize the tape. Try again! (If the error messages persist, you may want to try a new tape OR reboot the E7_11 - do this only when all else fails!)
NOTE: It is virtually impossible to retrieve any old data previously written to a re-initialized tape. Make absolutely sure that the tape in the drive is the correct one before issuing the init command!
The tape is mounted with the simple command:
E7_11> mount tape
If there are old files on the tapes, the system will read until it
reaches the end-of-tape (EOT) marker. It can take quite some time to
reach the end of a previously used tape - patience!!!
NOTE: If there is no proper EOT marker, it is not possible to write further files to the tape. This could be the case if the data acquisition crashed while a file was still open. Therefore, if you are in doubt, take another tape.
A list-mode file (the only type that the MBS system can write) is
opened with:
E7_11> open file name
OR
E7_11> @newfile (see below)
There are a number of qualifiers to the open file command, including -size and -auto. When a file has reached the length specified by -size, it is automatically closed and a new file opened. The filenames consist of a common root to which a four-digit index number is added. The MBS command file newfile.scom provides a convenient file handling - edit this file to supply a base name for your files. Traditionally, Rxx_F has been used, to which MBS adds the four-digit index and the extension LMD.
The first number to be used during a prompter session
is determined (stupidly enough!) by what is written to the file
/frs/usr/profi/filenum.set. If you want to start from scratch,
either delete this file (it will be automatically recreated) or edit
it and put "1" as the starting number. This will be incremented by +1
when the next file is opened (with the -auto qualifier specified).
Files are closed with the corresponding command close file. A
command file closefile.scom is provided which, in addition to
closing the file, shows the tape status as well as the time and total
number of buffers processed:
E7_11> @closefile
NOTE: At the first sign of difficulties with the data acquisition program (prm), close the current file if one is open! If you have to exit the Prompter (or worse, reboot the E7), and there is no end-of-file (EOF) written at the end of the last file on the tape, this tape cannot be mounted again! (However, all files can be read from it, including whatever information is contained in the last file - no panic!)
At any given time, the status of the tape drive and taping can be
obtained with:
E7_11> show tape
The label of the mounted tape (if any), the name and status (open/closed) of the current or last opened file, and the number of megabytes written to the file and in total to the tape, are indicated. At present, 30 Gbyte can be written to each tape. (Compression is default!) It is recommended to change tapes once each day (for safety), and to collect the data onto one or several tapes after the experiment (while making the recommended backup copy!) - remember that it is unwise to put all your eggs in one basket! If the physical end of the tape is reached while writing, the tape will automatically be rewound and dismounted. There is only one warning given as you approach the end of the tape, so keep an eye open to avoid problems!!!
To change tapes, just close the current file, and dismount the
tape:
E7_11> dismount tape.
When the Prompter indicates that the tape has been rewound, you may
press the unload button on the DLT drive front panel and, when the
green light goes on, operate the handle. It is good practice to
write protect the used tape immediately after removing it!
Insert a new tape and close the handle. Wait until the “Tape in use”
light has stopped blinking before proceeding with initializing and
mounting the new tape.
At the moment of writing (September 1997), the previously existing special program (GOOUFC) used to read out the FRS magnets and preparing file headers with their settings is not available on the Alpha or Lynx platforms. An intermediate solution has been found, in which a special GOOSY session is started on the accelerator control cluster machine CLARA4. Via this GOOUFC session, the magnet and drive ("feedthrough") status can be obtained and stored as text files. These can then be transferred to the PROFI account on the Alphas and printed from there. See the On-line Quick Reference part of this manual for detailed information!
If you finally have enough (of data, or of struggling with the system...), you should shut down the data aquisition system in an
orderly manner. Following this simple procedure leaves the whole
system in a well-defined status, making it easier for the next group
coming along!
Remember to report all problems related to the data acquisition to Klaus Sümmerer -- if bugs are never reported, they can't and won't be fixed!
NOTE: Please do not leave the VME and/or CAMAC crates permanently turned off! Only turn crates off if you need to exchange or insert modules, and then switch them back on immediately thereafter — this helps ensure stable operation.
For speedier access to the list mode data after the experiment, the files should be stored on the ADSM tape robot system. This system is controlled from the AIX UNIX cluster, but the stored data are also available to programs running under VMS on the Alpha cluster. The data are stored in archives which can be organized into directories and subdirectories. The FRS group already has an archive frs which is accessible via the UNIX profi account (ask Klaus Sümmerer for the password!). Individual users wishing to have their own archive should contact the GSI Computer Center (B. Loos). Note also that if you wish to copy entire tapes using the * wildcard, a managment class privilege must be obtained (M. Feyerabend).
Once the files are stored on the tape robot system, the archive can be accessed for different purposes. NOTE: The following commands can be entered both from the UNIX and VMS operating systems!
Many examples and more detailed information about the ADSM tape library can be obtained from the GSI Benutzerberatung or via the WWW.
This section tries to give some hints on how to deal with some
frequently occurring errors. Don’t worry if you don’t understand the
problem at first — the idea is to overcome it!
Rule 1: "Nothing works as it should."
Rule 2: "Don’t panic" — unexpected obstacles are to be expected!
Unfortunately, the system error messages can sometimes be rather cryptic. Especially if a problem keeps recurring, it is very important that the circumstances leading up to the crash are recorded together with the error messages (make a screen dump, or copy the screen contents to a file)! Only very brief information is written to the MBS log file (mbslog.l), and it is usually not sufficient for meaningful troubleshooting.
If bus errors occur during the startup of the MBS environment, the cause is most likely that not all CAMAC crates are turned on, or that some of the VSB connections are broken. Check that everything is turned on and that the VSB cables are firmly connected. If you run with both Slave branches, verify that the GSI trigger modules of both branches are connected to each other with the trigger bus cable! Correspondingly, if only one branch is to be used, the cable should be disconnected!
If the problems persist, the VSB cable connections between the E7s
and the CAMAC crates via the CBV crate controllers can to a certain
extent be tested using the program CVC developed by the DV/EE
group. The program is invoked (from the Lynx prompt) with
e7_11> cvc node
where node is the E7 (or CVC) you want to connect to.
A menu will appear allowing CAMAC function calls to be made.
As an example, entering crate number 2, station number 1, substation number 3 and function code 0 will attempt to read the conversion time programmed into the trigger module in slot 1, crate 2 (the standard FRS crate). The X (execute) and Q (query) status of the operation is indicated by the program. There is a provision for loop testing, i.e., repeating the same command continuously. Should the VSB connection be broken, a bus error will occur and the CVC program exits automatically. In this way the function and address lines to be used can be tested. A successful CVC test, however, does not prove that all individual VSB cable connections are OK!
Old “hanging” network connections between the E7s can cause
problems when starting a new prompter session. (This can, of course,
also be the case if the other nodes are down or hung up!) The message
logger on the main prompter node (usually E7_11) cannot establish
connections to logger client processes on the other nodes:
...: prompt: could not connect to server on ... ...: prompt: try again later...OR
E7_11: msg_log:[...] Message logger running E7_5: msg_log: [...] Message logger client running E7_6: dispatch: -I- f_stc The specified Internet Address and port is already in use E7_6: dispatch: could not create tcp server! Exiting...
Start by checking the network connection status with the netstatus command (at the Lynx prompt). While CLOSE_WAIT indicates that the connection has been marked for disconnection, a FIN_WAIT or FIN_WAIT_2 status indicates a problem. Also check for subprocesses in strange states with the (Lynx) command ps -axn. Tasks that are hung up (it ended abnormally) are indicated by "Z" (for zombie!) in the status column. Try stopping such processes with kill -9 pid (kill the zombie and send it to Hell!), where pid is the process id number. If this is not successful, or network connections refuse to disconnect, the processors involved should be rebooted.
NOTE: even after a remote reset has been performed it can take several minutes before the old connections are removed -- be patient!
Mistyping the name of a command, or trying to execute a command
file that doesn’t exist, normally only results in warning
messages:
E7_11: dispatch: Command not found: "ztart"! E7_11: dispatch: Command completed with status 30
Simply retype the correct command.
Occasionally, some task involved in the data acquisition (especially m_collector or one of the m_read_meb tasks) hangs up or crashes. If this happens, immediately close any open files and dismount the tape. Proceed by first stopping the acquisition and then the other tasks. If this is not possible, exit the prompter with ctrl-Z. Start from scratch with a remote reset.
Typical causes are bad VSB connections or external noise
influencing the trigger bus and/or the trigger module(s), but deeper-
lying problems with the event and sub-event structure can also be the
culprit! For instance, if the subevent length is an un-even number of
words, the event builder gets confused. This could lead to:
...: read_meb:exiting... SIGSEGV (segment. fault) signal ...: dispatch: -I- f_stc read interrupted, recover tasks. ...: dispatch: task m_read_meb ID ... removed!
Less serious problems include occasional buffer write
errors. These sometimes occur when the connection between the
local event server task m_event_serv and an external analysis
client has been improperly broken (GOOSY crash).
E7_11: event_ser: E-f_ev_flush-EV_WRTBUF: Error writing buffer E7_11: event_ser:E-main-EV_WRTBUF: Error writing buffer. Flush buffer clnt-no:... E7_11: event_ser: I-f_ev_mrkcln-EV_CLAMRK: Warning: Client already marked for deletion...
Sometimes local event counter mismatches occur. In this
case, the GSI trigger module detects what seems to be an event
synchronization problem: the event number of the event currently
being processed differs by more than one from that of the previous
event. This leads to errors like:
E7_5: read_meb: local event counter (lec) mismatch E7_5: read_meb: lec differs more than 1 count to previous one E7_5: read_meb: previous lec: -1, lec: 1 exiting... E7_5: dispatch: -I- f_stc read interrupted, recover tasks E7_5: dispatch: Task m_read_meb ID ... removed!
These errors are difficult to diagnose. If there is something physically wrong with the trigger module or the trigger bus cable, it is usually the collector task rather than the readout that crashes. The readout crashes seem to hint at hardware difficulties with either the CAMAC crates or specific modules involved in the readout.
The m_transport task will, after mounting a previously
used/initialized tape, attempt to reach the end of tape (EOT) marker
and then position the drive write head there. If the last file on the
tape was improperly closed, the EOT is missing, leading to the
following slew of messages:
E7_11: transport: volume header found, label= ...
E7_11: transport: File ...LMD found
etc. ...
E7_11: transport: no EOF records found, last file has not been closed
E7_11: transport: dismounting tape
E7_11: transport: error mounting tape
Sometimes SCSI errors occur while writing to tape. These are
generally detected by the m_transport task, resulting in
error messages like:
E7_11: transport: WRITE: variable length record write error E7_11: transport: SCSI write error, tape dismounted
Experience seems to suggest that whenever SCSI-related errors occur, the power to the tape drive should be cycled (turn it off, wait 30 seconds, turn it back on) and the E7_11 rebooted. The errors typically occur either when opening a new file or when closing an old one. In any case, it is unlikely that the tape can be used further. Wait until the "tape in use" LED on the tape drive is no longer blinking. Press the "unload" button, wait for the green "operate handle" LED and remove the old tape, write-protecting it. While waiting for the tape to come out, stop the acquisition if it is running. Close down and, if possible, exit the MBS environment in a controlled way (see the On-line Quick Reference part of the manual). Before rebooting E7_11, perform a remote reset.
The following is a list of some (C)NAFs for the CAMAC modules most commonly used at the FRS. The first column contains the CAMAC function number F and the subaddress A to be used. If not specified, the station number N is that of the module being addressed. The second column indicates whether the command is issued during initialization (I) or readout (R). In those cases data should be passed, either decimal or hexadecimal ("hex", preceeded by "0x") notation can be used.
| N(28)F(26)A(8) | I | "Z". Initializes the CAMAC crate. Activates the inhibit! |
| N(28)F(26)A(9) | I | "C". Clears the CAMAC crate. |
| N(30)F(26)A(9) | "I". Enables the CAMAC crate inhibit. | |
| N(30)F(24)A(9) | I | Disables the CAMAC crate inhibit. Very important! |
| F(0)A(0-7) | R | Read the contents of channels 0-7 (no clear). |
| F(2)A(0-7) | R | "Read and clear" the contents of channels 0-7. (The module is cleared only after F(2)A(7)!) |
| F(9)A(0) | I | Clear the module. |
| F(10)A(0) | I | Clear the LAM flag. (Not really necessary?) |
| F(17)A(0-7) data | I | Writes upper limit for individual channels 0-7. |
| F(17)A(8-15) data | I | Writes lower limit for individual channels 0-7. |
| F(20)A(0-7) data | I | Writes offsets for individual channels 0-7. |
| F(20)A(9) data | I | Writes a common threshold. (4418/V only) |
| F(20)A(14) data | I | Writes the status word. Should always be 2560 (0xa00)! |
| F(0)A(0-7) | R | Read the contents of channels 0-7 (no clear). |
| F(2)A(0-7) | R | "Read and clear" the contents of channels 0-7. (The module is cleared only after F(2)A(7)!) |
| F(24)A(12) | I | Clear the module. (Note: not a standard function!) |
| F(0)A(0-3) | R | Reads the contents of channels 0-3. |
| F(9)A(0-3) | I | Clears the module and the contents of channels 0-3. |
| F(0)A(0-7) | R | Read the contents of channels 0-7 (no clear). |
| F(2)A(0-7) | R | "Read and clear" the contents of channels 0-7. (The module is cleared only after F(2)A(7)!) |
| F(9)A(0) | I | Clear the module. |
| F(10)A(0) | I | Clear the LAM flag. (Not really necessary?) |
| F(0)A(0-7) | R | Read the contents of the channel indicated by the internal address pointer and increment the pointer by one. 32-bit data! |
| F(2)A(0-7) | R | Clear the module. |
| F(9)A(0) | I | Clear the LAM flag. (Not really necessary?) |
| F(16)A(i) | R | Sets the internal address pointer to channel i. |
| F(2)A(0-7) | R | "Read and clear" the contents of channels 0-7. |
| F(9)A(0) | I | Clear the module. |
| F(10)A(0) | I | Clear the LAM flag. (Not really necessary?) |
| N(1)F(0)A(3) | Reads the programmed maximum conversion time CVT (CVT = (65535 - data)/100 ns). | |
| N(1)F(0)A(2) | Reads the programmed fast clear time FCT (FCT = (65535 - data)/100 ns). |
NOTE: For many CAMAC modules, e.g., the Silena 4418 series, internal test functions have been defined by which the module performance can be checked. Refer to the module manuals for details!
This concludes the "MBS Reference" part of the FRS data acquisition manual. If you want more detailed information, please refer to the suggested reference litterature or contact the "experts"! If you have suggestions for improvements and/or updates, please contact Klaus Sümmerer or the author.