--- bck_vs_mjdW1.pdf bck_vs_mjdW2.pdf => horizontal line is median value from end of post-cryo, the red dots are binned medians. --- hotpixtrend.pdf => change in hot/hi-dark pixels relative to end of post-cryo, derived from stacking normalized-L0 frames, computing a trimmed average, then thresholding the resulting map above the zero baseline. Note that this flavor of "bad" pixels are relatively static and can be subtracted out with the static dark. --- noisypixtrend.pdf => #noisy pixels derived from stacking normalized-L0 frames, computing a trimmed standard-deviation then threshold above a fixed std-dev of 30 (arbitrary thres) for all cases. Numbers are then normalized by the number of pixels in the active region of each array. Note that these fractions are close to the actual fractions of _unusable_ pixels on each array. They vary on frame-to-frame timescales and cannot be corrected using calibrations. --- rms_vs_mjdW1.pdf rms_vs_mjdW2.pdf => spatial rms in a 200x200 box in a "clean" region (w/ minimal bad pixels) in L0 frames. Horizontal black lines are median values from the end of post-cryo. The red dots are binned medians. --- origrms_vs_mjdW1.pdf origrms_vs_mjdW2.pdf => same as the rms_vs_mjdW?.pdf except: here, filtered L0 frames from 40 scans from the end of original post-cryo were measured instead of 4 scans in the earlier plots (where the median was the horizontal line). These are compared to frames from 30+ scans acquired on Jan 21, 2014. --- *** origL1rms_vs_mjdW1.pdf *** origL1rms_vs_mjdW2.pdf => same as the origrms_vs_mjdW?.pdf (for L0) but now for the L1b frames. Notice the constant RMS when computed on L1b frames. This means the application of calibrations (particularly flat-fielding) to the L0 significantly reduces the RMS. For details, see email below. --- L1rms_vs_mjdW1.pdf L1rms_vs_mjdW2.pdf => same as rms_vs_mjdW1.pdf but not for L1b frames (same pixel region) and using all frames in every 10-th scan from neowiser start until Jan 27, 2014. --- sat_vs_mjdW1.pdf sat_vs_mjdW2.pdf => self-explanatory (I hope). The horizontal blue/red lines are medians from the end of post-cryo. --- L0pixhist_x.y_396.375W1.pdf L0pixhist_x.y_332.470W2.pdf L0pixhist_x.y.W1_comp.pdf => history of a single L0 frame pixel for each band. This pixel was suspected of becoming noisy all of sudden and this is confirmed towards the end of this timeline. The time-range shown is ~ Dec 13, 2013 to Apr 15, 2014. --- F. Masci, 04.15.2014 --------------------------------------------------------------------------- Random emails: More info on figs: *** origL1rms_vs_mjdW?.pdf *** From: Frank Masci Subject: W2 sensitivity issue resolved Date: January 25, 2014 at 4:23:46 PM PST If you recall, last week I recomputed the robust rms estimates on _L0_ frames using a large number of scans from the end of original post-cryo, then compared to more recent frames from neowiser. Here was the result: http://wise2.ipac.caltech.edu/proj/fmasci/neowisertrending/origrms_vs_mjdW2.pdf This time, I used the calibrated _L1b_ frames (same data and pixel region). Here’s the result: http://wise2.ipac.caltech.edu/proj/fmasci/neowisertrending/origL1rms_vs_mjdW2.pdf The L1b rms trend is pretty flat and maybe even lower for neowiser! The reason why I initially used L0 frames was to be closer to the “true” raw detector state and not be affected by the changing calibrations. This thinking is correct for monitoring “absolute” background levels; but it is not correct when deriving noise-related metrics like the rms. The pixel calibrations are the very thing that correct for all the instrumental/pixel systematics (e.g., flat-fielding) and hence that’s where our sensitivity estimates (via source extraction) are derived. So, there is no decrease in sensitivity and w2sigP1 is consistent with this. Nonetheless, it’s interesting to compare the local pixel-rms going from L0 to L1b - testament that the calibrations are actually doing something. Flat-fielding (and bias variations) play a big role here, so the L0 RMS is only telling us that the spatial variation in detector responsivity (or bias level) changed dramatically. Sometimes, all that’s needed (usually) is to think outside the box. Regards, Frank ------------------------- From: Frank Masci Subject: life of a neowiser pixel Date: April 16, 2014 at 9:28:15 AM PDT To: Roc Cutri , Tim Conrow , "J.Davy Kirkpatrick" , Sergio Fajardo-Acosta , Chris Gelino , Sherry Wheelock , Howard McCallon , Howard McCallon , Carl Grillmair Here are plots of the history of a single L0 pixel in W1, W2 from ~Dec 5, 2013 to ~Apr 3, 2014. That’s 811,988 framesets! There’s a reason why these specific pixels where targeted (one in W1 and another in W2). It’s because while creating the time-dependent calibrations (rms-maps in particular), some previously “normal" pixels started to behave erratically. Therefore, their histories were examined. Sure enough, these pixels became noisier towards the end of this period. This is disturbing since it says some pixels decide to go berserk without notice and behave erratically for some time, perhaps for 10 to 15 days. The hope was that a “static” bad-pixel mask could be used (for the last big time window) into the foreseeable future. It appears this is not appropriate. Something more dynamic is needed. I’ll run more tests to explore the stability of noisy pixels. http://wise2.ipac.caltech.edu/proj/fmasci/neowisertrending/L0pixhist_x.y_396.375W1.jpg http://wise2.ipac.caltech.edu/proj/fmasci/neowisertrending/L0pixhist_x.y_332.470W2.jpg Regards, Frank -------------------------- From: Frank Masci Subject: Re: life of a neowiser pixel Date: April 16, 2014 at 12:39:17 PM PDT To: Chris Gelino , Roc Cutri , Tim Conrow , "J.Davy Kirkpatrick" , Sergio Fajardo-Acosta , Sherry Wheelock , Howard McCallon , Howard McCallon , Carl Grillmair The attached plot may clarify things better. The vertical lines are cal-window boundaries (used to make the RMS-stack maps). Aside from intrusion by the galactic plan, the end of the timeline sent earlier has all the action. You can see a sudden jump in the value of this single pixel. Looking at the stack maps, there’s about 120 other pixels that behaved the same way. It remains to be seen if these get restored to their normal state. BTW, the history for a “happy” neighboring pixel is still being generated (to match the same ~12 day span). Will overlay and send later. Regards, Frank --------------------------- From: Frank Masci Subject: Re: L0 and L1b (v7.5) trending update Date: January 14, 2015 at 4:24:59 PM PST To: Roc Cutri Cc: Roc Cutri Here are the L0 frame levels for all frames from Jan 7 - 13. The vertical dashed line is at the location of the first frameset observed after the survey parameter change (=56089a001): http://wise2.ipac.caltech.edu/proj/fmasci/neowisertrending/bck_vs_mjdW1_34.pdf http://wise2.ipac.caltech.edu/proj/fmasci/neowisertrending/bck_vs_mjdW2_34.pdf The horizontal lines are medians from the end of original post-cryo (relics from early survey trending). And here the L0 pixel RMSs (in a “clean" region): http://wise2.ipac.caltech.edu/proj/fmasci/neowisertrending/rms_vs_mjdW1_34.pdf http://wise2.ipac.caltech.edu/proj/fmasci/neowisertrending/rms_vs_mjdW2_34.pdf And the number of hard/soft saturated pixels: http://wise2.ipac.caltech.edu/proj/fmasci/neowisertrending/sat_vs_mjdW1_34.pdf http://wise2.ipac.caltech.edu/proj/fmasci/neowisertrending/sat_vs_mjdW2_34.pdf In conclusion, there’s no discernible impact to any of the above metrics. Regards, Frank