Post Cryogenic (NEOWISE) Calibrations

1. Notes

o Below we show W1,W2 darks and flats generated using on-orbit data at
  different times throughout the post-cryogenic mission. A self-consistent 
  calibration method that tracks the change in response of each pixel relative
  to the overall background was used. More calibrations will be added to
  this page as the mission proceeds.

o Note that the absolute level of the darks using the self-cal method is
  is very difficult to get right. There is no guarantee that they're physical,
  although the same method applied to cryogenic data did show stability
  in the dark level vs time. The "dark" counts (in DEB DN) are actually what  
  you see in the images below minus 128 (the detector bias for W1,W2).
  The absolute variation is about +/-4 DN. The (relative) spatial
  variations however are very accurate. 

o Approximately 2000 frames (~8 scans worth) were used for each run below,
  although we will probabably use ~6000 frames for pipeline calibrations.
  It's a challenge to tease out a systematically varying (and evenly time-
  sampled) background in W1, W2, especially since the detectors are no
  longer in a thermally controlled environment.

o The flat calibrations have an accuracy of ~1%/pixel, where the uncertainty
  was estimated using a-posteriori methods (hence more believable). The darks  
  have a relative accuracy of ~5%/pixel (w.r.t. their median level). 
  Absolute accuracy is much lower and not quantifiable at this time.

o Moon-glow is suspected to have contaminated the Oct 18 W2 flat product.
  The closest moon-to-frame separation was ~30 degrees. Other calibrations
  had the moon at >~ 60 degrees. The Oct 18 W1 flat appears unaffected,
  as expected from cryo-mission analyses. The W2 flat calibration appears 
  stable after Oct 18.

o Images in each panel [across all dates] are at the same stretch.
  The color-bar gives an idea of the variation and level.

o Note that the darks and flats below bear no resemblance to the cryogenic
  (nominal) calibrations. To view these, see: provisonal cals and v3.5 cals. 

2. W1 Flats: Oct 18, 21, 24, 26, 31, Nov 4, 15, 22

Dates go from left to right.


3. W2 Flats: Oct 18, 21, 24, 26, 31, Nov 4, 15, 22


4. W1 Darks: Oct 18, 21, 24, 26, 31, Nov 4, 15, 22


5. W2 Darks: Oct 18, 21, 24, 26, 31, Nov 4, 15, 22


6. Bad Pixel Trending

The trend plots below were made from ancillary products associated with the dark and flat generation process. The specific bad-pixel conditions and thresholds assumed for W1 and W2 are:

  (1) LO/HI DARK: median stack pixel value < -20 OR > 500 DN;
  (2) HI NOISE: robust stack RMS value > 50 DN; 
  (3) LO/HI RESPONSE: flat value < 0.1 OR > 1.5;

Note: these conditions are not mutually exclusive, e.g., a noisy pixel could be due to the high Poisson noise from a high dark current or hot pixel.

W1

W2


7. Performance

7.1 Old Vs New calibrations on single frames

The top row in following figures (left-to-right) shows a frame processed using old (cryogenic) calibrations before subtraction of the sky-offset from dynacal, with subtraction of the sky-offset, and the skyoffset calibration itself. Note how the sky-offset picks up the residuals due to the assumption of a bad dark and flat. This calibration is only designed to pick up variations in the relative offset, not gain variations.

The bottom row represents the same image sequence but using the new flat and dark (specifically generated usng data from Oct 26). Note how the residual offset variations in the skyoffset are more clustered around zero (see color bar).

W1:

W2:


7.2 Pixel RMS Trends

We searched for significant differences in pixel (spatial RMS) fluctuations per frame assuming the old and new calibrations. If the new calibrations are removing residual systematic structure, then a drop in the pixel RMS is expected relative to the old calibrations. A consistent drop of ~2% was indeed seen in W1 and W2 for a run of 50 frames. We examined three different robust metrics for the RMS deviation per frame:

SigMED16Ptile: the median (50th percentile) - 16th percentile of the pixel distribution.

SigLTMADMED: the low-tail median absolute deviation from the median.

SigLTMADFM: the low-tail median absolute deviation from the mode.




Last update - 23 November 2010
F. Masci - IPAC/Caltech