IV. WISE Data Processing

4. WISE Moving Object Pipeline Subsystem (MOPS)


Contents

a. Introduction
b. Subsystem Overview
i. Operational Description
ii. Caveats
iii. Finding WMOPS Objects
c. Summary

a. Introduction

The WISE Moving Object Pipeline Subsystem (WMOPS) was designed to detect Solar system objects in the WISE scan data as these data were processed in real-time, i.e. without delay, with particular emphasis on the detection of Near-Earth objects (NEOs). To facilitate ground-based recovery efforts and thus extend the observing arc of 8-12 detections beyond the span of a few days (on average), the detections and astrometry were to be reported to the community as soon as possible. Hence, WMOPS reported the tracks of candidate object detections within a maximum of 10 days, but usually within a few days, of their detection times. Reports of these detections were made to the Minor Planet Center, or MPC, the IAU-designated clearing-house for the detections of small bodies in the Solar system.

The WISE spacecraft and instrument functioning parameters presented their own challenges unique to IR observations from spacecraft platforms. Parallax motion was minimized by the terminator-following orbit of WISE, making the distances to moving objects more difficult to determine, and objects with low projected sky-velocities more difficult to detect, than from ground-based observatories. Designed to be sensitive to both slow and fast moving objects, WMOPS operated on the extracted source lists produced by the pipeline processing of each WISE pole-to-pole scan. These source lists had been astrometrically and photometrically calibrated, spurious detections of image artifacts were identified, and many of the previously known Solar system objects in the field-of-view of each WISE exposure were tentatively associated with sources.

WMOPS processing began by first selecting sources with a signal-to-noise ratio (SNR) above a 4-σ threshold that were not identified as artifacts. Detections with profile fits resulting in rchi^2 > 4 were also rejected as these were likely cosmic rays or other artifacts. WMOPS then identified and filtered out "stationary" objects that repeated in position between scans. Linkages between pairs of non-stationary detections were then made using the FindTracklets routine (courtesy of the PanSTARRS project; see Kubica et al. 2007 Icarus 189, 151-168.), and then detection-pair linkage was performed using the CollapseTracklets routine (courtesy of the LSST project; Myers et al. 2008, BAAS 40, 493) to generate lists of candidate moving object tracks. This step was run several times, over adjacent overlapping regions of the sky. A database of detection pairs was kept during the processing, as well as a "legacy" database that was updated post-processing, which served as the WMOPS "memory" between runs.

After each run, the candidate object tracklets were then vetted using a combination of automated and manual quality assessment processes. Confirmed tracklets and visual magnitude estimates were then reported to the MPC. WMOPS was run at 3 to 4 day intervals, so that sufficiently long tracklets could be generated. Tracklets with fewer than 5 detections were not considered valid detections. Owing to the success of the WISE processing up to the point of the source-list extraction, WMOPS was able to push to lower signal-to-noise thresholds and lower the minimum required detections in a tracklet, allowing for an increased detection rate. Note that WMOPS was an extremely large and complicated subsystem. This section provides a working overview of the subsystem design.

b. Subsystem Overview

i. Operational Description

WMOPS fulfilled one of the tasks of the NEOWISE project, an augmentation of the original WISE spacecraft science funded by the NASA NEOO program to discover and characterize new and existing NEO's in the WISE images. In order to facilitate ground-based follow up, WMOPS reported the tracklets to the MPC within 10 days of the central detection time of each new-object track. These reports were made in the prescribed MPC-format for astrometric observations from a satellite-based observing platform. Because it is not part of the original proposal for the mission, WMOPS was not allowed to impact the other WSDS systems.

WMOPS was run at 3 and 4 day intervals, requiring WMOPS to complete processing in under 3 to 4 cluster-CPU hours. It was the goal of WMOPS to detect 80% of the objects that fulfilled the requirements for detections, namely exceeding an SNR threshold of 7 and having a minimum of 6 detections. Before version 3.5 implementation, we achieved 90% detection on the automated processing (pre-eyes-on QA see below), exceeding our original goal, which allowed us to lower the WMOPS internal SNR threshold to 4, i.e. to accept nearly all detections with sufficient signal to pass the MDET subsystem's threshold (a multiband SNR of >5). With V3.5 we increased to 92% completeness for the objects that met the original criteria. Our desired goal was also to achieve 90% reliability in the tracklets we report to the MPC. As follow-up with any set of tracklets can be spotty, and especially with the thousands of tracklets reported by WMOPS to the MPC each run, our only means of registering tracklet reliability was by feedback from the Minor Planet Center. Each complete WMOPS run had fewer than 1% bad tracks on average.

To minimize processing time within the WSDS requirements, WMOPS worked with detections, eliminating artifacts and stationary objects from the processing instead of subtracting a static sky from the images and generating a new source list from there. This, the detector characteristics, and the WISE spacecraft's survey cadence made WMOPS a uniquely designed subsystem. Beyond the linear-motion search algorithms themselves, it was these specifics, i.e. the way the surrounding software sifts through and filters input detections and output tracklets, that often determine the efficacy of any search software for a moving object survey. Figure 1 summarizes the basic flow of the WMOPS subsystem. WMOPS ran off of the products of the WSDS scan/frame and multi-scan pipelines. In particular, there are dependencies with the ARTID and ICal modules additional to the originally proposed single exposure products.

A scan list was usually provided for each run. Before frame processing could commence, WMOPS needed to assess which frames had complete coverage to determine the stationary objects. HEALPix, a package specifically designed and tested to conduct computations in spherical coordinate systems, was used to find neighboring frames, and corner positions of nearby frames were converted to the pixel coordinates of the central frame. A recursive algorithm was employed to determine if and by what frames complete coverage of the central frames area was obtained. The covered frames (nearly all of the frames over the course of the survey had complete coverage) in each of the scans were then read, the detections filtered according to several criteria, and the detections read into the database and organized into overlapping regions of the sky, or globs, where they were processed in batches. For the full-cryogenic mission dat,a WMOPS only functioned in W3 and W4. The broad criteria for rejection of detections on ingest were as follows: too low of signal-to-noise values in both W3 and W4, identified artifacts of specified nature, specified extremely pathological PSF chi-squared fit variance, and excessively large numbers of non-stationary object detections-per-frame (see below). Each glob was processed when the mops-mdex tables from each frame in the glob were successfully read and the detections placed into the SQLite wmops-database. A critical step in the frame processing was the stationary-object-rejection (SOR), which initiated the spatial processing of the detections on a per-frame basis. The idea behind SOR was simple; detections from separate scans that did not move more than, or were co-located within, a minimum distance were rejected. The operational rejection limit for minimum moving-object motion was about 2 arc-seconds, or a little less than the WISE pixel size. To avoid wrap-around problems for the SOR step, WMOPS converted all positions (e.g. on the overlap frames) into the coordinates of the central frame being processed.

After SOR, the remaining moving object candidate detections were processed by FindTracklets. The FindTracklets routine created pairs of detections (called "touples") within a glob, and correlated velocity and direction of motion with each touple, along with the individual on-sky location with each of the detections. These lists of touples were stored in the database, and were then fed through the CollapseTracklets routine, which created chains of matched the touples output by FindTracklets, matching the touples according to velocity and location, within a specified uncertainty at each point. The output tracklets were then filtered for length (a minimum of 5 detections from different scans). The tracklets were prepared for output and converted into the standard MPC reporting format and two files were generated and placed in a results directory: last.mpc and last.mpc.sids. These files contain the MPC-formatted output (see the MPC web site) and the source ID list for each track. The legacy database was updated to include the tracklets of length 5 or greater. Orbit fits then were generated using the MPC Initial Orbit Determination (MPCIOD) software provided by the Minor Planet Center in executable form for use by WMOPS, and the orbital elements and fit residuals of those tracklets that converged to physical orbits were placed in the last.els file. This completed the activities of the wmops routine, which as its last act, kicked off the wmopsqa routine.

Figure 1 - Summary flow of the WMOPS subsystem. The ingesting, glob processing, and formatting were done primarily by the wmops perl-based software. The quality assurance was done in a partly-automated fashion by the wmopsQA routine and by eyes-on vetting of tracklets by the QA team. In the last QA filtering step, final products were generated using the wmopsMPCGen routine and a final vetting on individual tracks were done by-hand for tracks with poor residuals from the Digest2 routine provided by the MPC.

The wmopsQA routine did the final automated filtering and generated the cgi-script files, lists, plots and thumbnail products for the eyes-on QA steps. There was a run-summary page as well as individual tracklet pages (Figure 2) generated for the QA. These pages were generated using the first.cgi and track.cgi scripts. Every tracklet that was not automatically rejected by the automated filters had the full set of eyes-on QA data products generated in its own separate QA subdirectory. Plots were generated using gnu-plot scripts and output in jpeg format. Thumbnails were generated with a scaling of +7σ, -3σ scaling. The list of rejected and accepted tracklets were kept in the qa_state.txt file, which was modified by the submit.cgi script which in turn was initiated by a button on the tracklet-level QA page. The qa_state.txt file was essentially a modification of a .sids (as in last.mpc.sids) file with the vetting status label ("Accept", "Artifact Reject", "Review", or "Reject") of the tracklet pre-pending the .sids line entry. After the eyes-on tracklet review was completed, in a QA filtering step, products were generated using the wmopsMPCGen routine and a final vetting on individual tracks were done by-hand for tracks with poor residuals from the Digest2 routine provided by the MPC. The wmopsMPCGen routine created the qa-vetted list of MPC-formatted tracks, and made crude estimates of R-band magnitude for the objects to aid follow-up by the general community. Two lists of "clean" and corrected tracklets in MPC format, one list for candidate NEOs and one for other candidate objects, were submitted to the MPC.

Figure 2 - Discovery of cometary activity in P/2009 WJ50 (La Sagra) in the WMOPS QA page raw image arrays. Sky-plane motion charts and text-based data are not shown here, but were generated as wmopsQA products. Note the examples of an image latent and cosmic ray in the second column. Thumbnail images were critical to the successful identification of new objects.

ii. Caveats

The 90% completeness value referred strictly to those objects with 5 or more detections that:

Finally, the visual wavelength magnitudes reported to the MPC, which may remain in their MPCCAT-OBS data (see below), were estimated from IR fluxes and colors and are likely not accurate to better than 1 magnitude. The standard deviation of these values relative to the true magnitude is expected to be in excess of 0.5 magnitudes.

iii. Finding WMOPS Objects

The fastest means of finding WMOPS observed objects, beyond the available IRSA search engines, is to search through the MPCCAT-OBS archive or using the MPC-OBS service to extract particular objects using the WISE observer code, C51. These will return the track that was reported by WMOPS to the MPC. A description of the tracks and how to obtain them is provided in the moving object tracklet section of the User's Guide.

Newly discovered objects of interest, like NEOS, comets, and Centaurs, were issued Minor Planet Electronic Circulars (MPECS). These often reported observations from follow-up observers, in addition to the spacecraft observations.

c. Summary

As of February 2011, WISE observed more than 157,056 moving Solar system objects, including ~ 33,335 new discoveries, attributable mostly to WMOPS. The dataset contains at least 584 NEOs, more than 1,500 Jupiter Trojans, 18 Centaurs and scattered-disk objects, and more than 120 comets. During the time of the spacecraft's operation, WMOPS enabled WISE to be the most prolific observer of Solar system bodies.


Last update: 2011 July 5


Previous page    Next page
Return to Explanatory Supplement TOC