Advice to Observers - 20050930 - Lessons Learned from the Data Processing Trenches (DPC)

September 30, 2005 - Notes by DPC

Recommendations for Mimir Observers for the Fall Observing Quarter

I have been processing my huge May/June Mimir run and writing extensive software to make the processing both possible and fairly easy. Along the way, I have learned a great deal about our Mimir data and how improvements to observing strategy can insure capture of high quality science data.

  1. Start up Mimir at least an hour or two before you plan to start collecting data
    1. There are some drifts in the array control electronics that appear temperature related
    2. The muffin fans on the back of the telescope have the greatest effect in changing the array control electronics temperature
      1. It is probably not wise to leave them on during the day
      2. It *is* wise to turn them on 1-2 hours before starting Mimir observations, to allow the fans to equilibrate the electronics temp
      3. We *are* installing a new temperature-controlled fan vent system for the electronics, and this *may* cure the drifts, but for now, assume not
    3. Start LOIS and take at least two Mimir test images
      1. This leaves the electronics and array in a continuous readout mode, which will also help both reach temperature stability
  2. Pay strict attention to saturation limits
    1. The lower half of the array has the lowest saturation limits, with some pixels going into saturation by 2200-2400 counts
    2. Keep the brightest portions of the brightest star centers & spectra under about half of the saturation value, so that if the seeing disk decreases the stars do not go into saturation
    3. My linearity correction code is very unforgiving about saturation. If the pixel value exceeds the saturation level found from the linearity run, the pixel is deleted (set to -1e6) in the science frame.
    4. We will try to boost the saturation level during the November Engineering run, but don't assume we will succeed. Every array tweak we have done to improve stability and reduce hot pixel counts has resulted in *lower* saturation limits so far.
    5. Bright backgrounds reduce the allowed saturation levels even more, since some of the allowed count "range" is used up between the release of the array reset and the first read. For L-band, this may significantly reduce the dynamic range.
  3. Save all the "raw1_nnn" files in addition to the "nnn.FITS" files.
    1. The "raw1_nnn" files are the first readouts of each image. These are *required* for the linearity correction data processing step.
    2. The "raw2_nnn" (second read) files may be deleted
  4. Use exotic modes only if you have to, or plan to use your own reduction software
    1. "COADDS" are OK - the software on this site supports processing of COADDed images
    2. "subframes" are currently a problem - none of my data processing code is set up to deal with images smaller than full size.
      1. this is on my list to fix, but will not happen until after Jan 1, I suspect
    3. Fowler sampling (NFS) is not yet supported with my software. Again, probably after Jan 1
  5. Take "Bias" frames often. Bias frames are zero exposure darks, taken as a 32 image set. The dark filter in FW1 is fine, and an yggdrasil command to move the filter wheels into "dark" positions is working
    1. It is vital to take multiple blocks of 32 Bias images *many* times during the night, to insure electronics drifts are detected and removed
    2. Taking bias sets *before* and *after* long sets of observations of an object is ideal
    3. Taking biases *before* and *after* flats and linearity runs is essential
    4. Don't assume biases on one night will work on another night
  6. Be sure to take linearity data, as per the How-To-Guide instruction sheet
    1. Do this at night, *NOT* during the day, as the light settings needed are very low and sun illumination changes will kill the trends sought
    2. If only doing linearity, you only need the "lights on" set. You do not need the "lights off" set.
    3. If you want to measure read noise and conversion gain, you *will* need to collect the "lights off" set
  7. Take lots of images of your targets
    1. The array controller communication seems to still drop about 3% of the images, so if you really need that image or spectrum, divide the integration time in half or thirds and take two or three times as many images to insure you don't miss crucial science information
  8. Polarimetrists - I have not yet mapped out the instrumental polarization across the instrument field of view, but it could be significant
    1. Be sure to polarimetrically observe at least one globular cluster during each polarimetry run
    2. It is best if you can put the globular in the center of the array, then in the center of each quadrant to help map out the instrumental polarization