=================================================================
GENERATING REGULAR (WEEKLY) L0 and L1b PIXEL METRICS FOR NEOWISER
=================================================================

F. Masci, 1/20/2016

ssh caustic
newcfg dev

-------------------------------------------------------------------------
(1)
mkcalprod:

Generate either dark, flat or both and compute metrics therefrom for
trending. Use two cal-windows per week (3.5 days each = 302400 sec). 

example:
ssh wcnode42
newcfg dev
cd 
./mkcalprod 2016-01-05T12:00:00.000 302400 dark >& LOG_mkcal1.txt &

ssh wcnode43
newcfg dev
cd 
./mkcalprod 2016-01-02T00:00:00.000 302400 dark >& LOG_mkcal2.txt &

ensure time-interval (second argument) is <~3.5 days or <~ 302400 sec
and execute each of the above on separate machines (wcnode42 and wcnode43).

at end of run, metrics to trend will appear at the end of LOG_mkcal?.txt:
e.g:
1001 frms : avg B/S temperature = 73.972 K
=> temperature from averaging B/S temperatures of ~middle 1000 frames
   in time-window (~3.5 days).
AND
66473a102-66580a009 187690  308250  24819  25738
=> bad pixel metrics: number of hot pix w1; number of hot pix w2;
                      number of noisy pix w1; number of noisy pix w2;

The above metrics can be appended to my latest growing table: BadPixStats.txt
Columns: counter; mid-window date; #hot pixW1; #hot pix W2; #noisy pix W1;
         #noisy pix W2; mid-window BSA temperature.
This table is used to generate the following four trends: 
  - hotpixtemp.pdf
  - hotpixtrend.pdf
  - noisypixtemp.pdf
  - noisypixtrend.pdf

These plots are also linked at the bottom of the following page:
http://wise2.ipac.caltech.edu/proj/fmasci/neowisertrendingcal/readme.html

-------------------------------------------------------------------------
(2)
checkL0satandbck (L0-frame trending):

Compute L0 frame metrics for trending.
Filter out moonlight, saa, and galactic plane frames:
keep: ($ms>35)&&(abs($gs)>15)&&($saa>5)

Output files (L0satbckstatsW?.txt; see below) contain:
L0 frame filename; mjd; sat enabling flag; number of sat pixels from
encoding; number of hard-sat (bare-bias) pixels in active region;
modal background level - 128 in small special region; robust RMS
in this same region.

example:
ssh wcnode47
newcfg dev
cd 
./checkL0satandbck 2016-01-05T12:00:00.000 604800 >& LOG_checkL0.txt &

where 604800 sec => go back 7 days.

The above output files can be appended to my latest growing tables:
L0satbckstatsW1.txt and L0satbckstatsW2.txt for plotting these six trends:
  - bck_vs_mjdW1.pdf
  - bck_vs_mjdW2.pdf
  - sat_vs_mjdW1.pdf
  - sat_vs_mjdW2.pdf
  - rms_vs_mjdW1.pdf
  - rms_vs_mjdW2.pdf

These plots are also linked on the following page:
http://wise2.ipac.caltech.edu/proj/fmasci/neowisertrendingcal/readme.html

-------------------------------------------------------------------------
(3)
modebckgndchiV2 (L1b frame cal-performance trending):

Compute median of all frame pixel modes, chi's, rms's, skyNumLo/Hi 
over individual scans from meta-ical tables for w1 and w2.
Frame filtering is also applied in this version:
($glat[$i]>20) && ($moonsep[$i]>30) && ($saasep[$i]>5)

Output file (bckgndrmschiL1b.txt; see below) contains in this order:
index scanId MJD w1bck w2bck w1uncscal w2uncscal w1rms w2rms w1SkyLo
w2SkyLo w1SkyHi w2SkyHi w1medSpikeHi w2medSpikeHi w1maxSpikeHi
w2maxSpikeHi

example:
ssh caustic
newcfg dev
cd 
./modebckgndchiV2 2016-01-05T12:00:00.000 604800 >& LOG_modebck.txt &

where 604800 sec => go back 7 days.

The above output file can be appended to my latest growing table:
bckgndrmschiL1b_v7.5.txt for plotting the following: 
  - medchi_vs_mjdL1b_v7.5.pdf
  - medskyLoHi_vs_mjd_v7.5.pdf
  - medrms_vs_mjdL1b_v7.5.pdf
  - medspikeHi_vs_mjdL1b_v7.5.pdf
  - maxspikeHi_vs_mjdL1b_v7.5.pdf

These plots are linked on the following page: 
http://wise2.ipac.caltech.edu/proj/fmasci/neowisertrending/readme_v7.5.html

-------------------------------------------------------------------------
(4)
lownuflat:

Make low frequency flats by stacking binned 'NEOWISER - AllWISE' residual
photometry maps for W1, W2.

Outputs:
  convlownuflatW?_*.fits
  convsigmalownuflatW?_*.fits

example:
ssh caustic
newcfg dev
cd 
./lownuflat 2016-01-05T12:00:00.000 604800 >& LOG_lonu.txt &

where 604800 sec => go back 7 days.

The two images convlownuflatW?_*.fits can be displayed side-by-side using
DS9 with a common stretch of 0.01-0.99 (+/- 1%) and color map "SLS", e.g:
convlownuflat_67333b-67546a.jpg