Astronomers' perspective
by Andy Adamson
The OracDR data reduction system is intended to be a pipeline reducer for incoming data. In the longer term, it will be integrated with the ORAC observatory control system (and thus with the ORAC Observing tool and procedures developed therein). It will also be the online reduction system for all new instruments. There are a number of differences between the OracDR method of reduction and the other systems currently in use at UKIRT, and observers should not expect OracDR to behave or be used in the same way as those systems.
- Firstly, OracDR aims to reduce data to a point where its quality can be assessed; it will not generally produce publication-quality results (though in certain circumstances it may do).
- Secondly, although OracDR also works offline, it is expected that most observers will use their own preferred data reduction package to work interactively work on their data. The rest of this document summarizes and accounts for the operational differences between the pipeline and existing packages.
It pipes, therefore it is
This is crucial. Everything else about the package is clear once this is grasped. This is not a reduction package a la CGS4DR; it is a reduction black box which knows the incoming data types (by their headers) and transparently applies a reduction recipe to them. There is nothing preventing you from running three simultaneous instances of the pipeline, for example to (i) reduce the incoming data in real time, (ii) re-reduce a previous group of files using a different reduction recipe and (iii) reduce and file a single previous observation as a dark. You do this by running three versions of oracdr, using the command-line switches to alter their behaviour (recipe, start and end observation numbers to process, graphics options, etc.). Each instance of the pipeline will go through the required files (existing ones or files just arriving on disk as specified on the command line) and reduce them. Once its remit of reduction is complete, it will exit.
Control
The behaviour of OracDR is entirely controlled by the command line options entered at startup. From that point on, the system either takes its reduction recipe instructions from the file headers (this is the default) or uses a hardwired recipe given on the command line itself. The recipe is, in fact, the only allowed parameter on the command line - all the rest are options. Once you start an instance of oracdr up, there is no further control over it; this is a considerable change from the situation with CGS4DR, for example, where the same package remains up once you start it, and changes are made within the package. Corollary: if the pipeline fails to find a required calibration frame, for example, the only logical thing for it to do is exit cleanly, telling you why it did so. There is no control from within the pipeline. Note that once the full ORAC system is available, there will be plenty of pre-checking that your calibration frames will indeed exist and be appropriate. Until that point, the behaviour of the system is logical if you understand the underlying philosophy and do not expect to be able to control the pipeline in real time.
The command usage is as follows:
oracdr [-options] RECIPE_NAME
note again that the only parameter to the command is the recipe name. These are all given on the web page linked to from the ufti pages. The recipes are applied to the individual frames by an algorithm engine, which is currently the STARLINK system (KAPPA, CCDPACK).
Options
The rest of the behaviour of the command is controlled by the option switches. These options may be abbreviated to a unique substring. It's via command line switches that you (for example) control the range of file numbers to be reduced, force the system to use a particular calibration file when reducing (e.g. to try a different flat exposure). This list needs to be read thoroughly by anyone wanting to use the system.
-help this help text
-version print the version number
-verbose print engine messages (else just have oracdr messages)
-debug log debug messages to ORACDR.DEBUG
-nodisplay do not launch display
-from number of first observation
-to number of last observation
-list comma separated list of object *numbers*
Colons indicate a range (1,2,3:5,10 means 1,2,3,4,5,10)
-ut UT date of observations (defaults to current yyyymmdd)
-calib comma seperated key=value pairs (eg. -cal dark=file1 or
-cal dark=file1,bias=file2) of calibration *files*
-loop Type of data detection loop: list, inf, or wait
-loop list is equivalent to not using a loop at all
(only data specified is reduced)
-loop wait waits for data to appear before timing out
(data is reduced and we wait for new data)
-loop inf does not wait for data
(data is reduced and new data is assumed to be available)
-loop flag waits for completion files to appear before
procesing (data is reduced and we wait for new data
via the flag)
-batch Run in batch mode. Precalculate groups before proceesing
data. 'wait' loop mode should not be used with this option.
-skip Allow the data detection loop to skip missing observations.
Default is to stop the loop when a expected file
can not be found.
-noeng Do not start algorithm engines. Note: this will cause
the vast majority of recipes to fail.
-log Control the location of the session logging. Default
is to log messages to the screen and file (log=sf)
when DISPLAY environment variable is not set. If
the DISPLAY env variable is set default is to use log=fx.
Options are:
s - log to screen (Xterm=STDOUT)
f - log to file. The logfile is called .oracdr_12922.log
where 12922 is the current process ID - it is written
to ORAC_DATA_OUT (it is a hidden file)
x - log to an X widget. Has the advantage that warnings
and errors do not scroll off the screen
The options can be combined (ie log=sf will write log
messages to screen and to a file).
-warn Turn on warning messages (perl -w). This should be used
for debugging only. If -verbose is also true then full
diagnostics are turned on (see 'perldoc diagnostics')