previous page   back to table of contents   next page

IV. NEOWISE Data Processing

IV.3. WISE Moving Object Pipeline Subsystem (WMOPS)


Contents

a. Introduction
b. NEOWISE-Reactivation adaptations
c. Subsystem Overview
i. Operational Description
ii. Caveats
iii. Finding WMOPS Objects

IV.3.a. Introduction

The WISE Moving Object Pipeline Subsystem (WMOPS) was designed to detect Solar system objects in the WISE single-exposure data as quickly as possible during data processing, with particular emphasis on the detection of Near-Earth objects (NEOs). To facilitate ground-based recovery efforts and thus extend the observing arc of 8–12 detections beyond the span of a few days (on average), the detections and astrometry are reported to the community as soon as possible. In practice, WMOPS reported the tracklets of candidate object detections within approximately three days of their detection times to the Minor Planet Center (MPC), the IAU-designated clearinghouse for the detections of small bodies in the Solar system.

The NEOWISE spacecraft and instrument functioning parameters presented their own challenges unique to IR observations from spacecraft platforms. Parallax motion between sequential observations was minimized by the terminator-following orbit of NEOWISE, which made the distances to moving objects more difficult to determine, and objects with low-projected sky velocities more difficult to detect than from ground-based observatories. Designed to be sensitive to both slow- and fast-moving objects, WMOPS operated on the extracted source lists produced by the Scan/Frame Pipeline processing of each NEOWISE pole-to-pole scan. These source lists were astrometrically and photometrically calibrated, spurious detections of image artifacts were identified, and many of the previously known Solar system objects in the field of view of each NEOWISE exposure were associated with source detections.

WMOPS processing began by first selecting sources with a flux measurement signal-to-noise ratio (SNR) above a 4.5-σ threshold that were not flagged as artifacts. Profile-fit measurements resulting in rchi2>4 were also rejected as these are likely cosmic rays or other artifacts, except in the case of detections with SNR>15-σ, where the rchi2 requirement is ignored to facilitate identification of objects that showed cometary activity. WMOPS then identified and filtered out "stationary" objects that repeated in position between scans. Linkages between pairs of non-stationary detections were then made using the FindTracklets routine (courtesy of the Pan-STARRS project; see Kubica et al. 2007 Icarus 189, 151-168), and then detection-pair linkage was performed using the CollapseTracklets routine (courtesy of the Vera C. Rubin Observatory/LSST project; Myers et al. 2008, DPS 40, 52.06) to generate lists of candidate moving object tracks. These steps were run several times, over adjacent overlapping regions of the sky. A database of detection pairs was kept during the processing, as well as a "legacy" database that was updated post-processing, which served as the WMOPS "memory" between runs.

After each run, the candidate object tracklets were vetted using a combination of automated and manual quality assessment processes. Confirmed tracklets and visual magnitude estimates were then reported to the MPC. More than a critical and effective repository of reported observations, the MPC provides multiple services to the community for the verification and confirmation of tracklets. These services include vetting of candidate tracklet astrometry for realistic orbital solutions, posting of object ephemerides of high-priority objects for follow-up to the community via the NEO Confirmation page (along with multiple orbital solutions), linking of reported objects with prior discoveries, and continually updating object brightness estimates and object orbital elements based on newly linked observations. Regular reports are also made by the MPC to the small body observing community in the form of electronic circulars. The above list of services that the MPC provides is not exhaustive, and it should be noted that the MPC plays a critical role in most Solar system object search programs.

Tracklets with fewer than 5 detections were not considered valid detections for the automated WMOPS search. Owing to the success of the NEOWISE processing up to the point of the source-list extraction, WMOPS was able to push to lower signal-to-noise thresholds and lower the minimum required detections in a tracklet, allowing for an increased detection rate. Note that WMOPS was an extremely large and complicated subsystem. This section provides a working overview of the subsystem design.

IV.3.b. NEOWISE-Reactivation Adaptations

The general architecture of WMOPS was nearly identical to what was run during the WISE prime mission, and is described in Section IV.3 of the WISE All-Sky Release Explanatory Supplement with the following important modifications:

IV.3.c. Subsystem Overview

IV.3.c.i. Operational Description

WMOPS fulfilled one of the tasks of the original NEOWISE project, an augmentation of the WISE program funded by the NASA NEOO program to discover and characterize new and existing NEOs in the WISE images. In order to facilitate ground-based follow-up, WMOPS reported tracklets to the MPC within 10 days of the central detection time of each new-object track. These reports were made in the prescribed MPC format for astrometric observations from a satellite-based observing platform.

WMOPS was run at 2- to 4-day intervals, requiring WMOPS to complete processing in under 3 to 4 cluster-CPU hours. It was the goal of WMOPS to detect 80% of the objects that fulfilled the requirements for detections, namely exceeding an SNR threshold of 7 and having a minimum of 6 detections. Before WSDS v3.5 implementation during the prime mission, WMOPS achieved 90% detection on the automated processing (pre-eyes-on QA; see below), surpassing the original goal. With v3.5 WMOPS increased to 92% completeness. The desired goal was also to achieve 90% reliability in the tracklets reported to the MPC. As follow-up with any set of tracklets can be spotty, and especially with the thousands of tracklets reported by WMOPS to the MPC each run, the only means of registering tracklet reliability was by feedback from the Minor Planet Center. Each complete WMOPS run had fewer than 1% bad tracks on average.

To minimize processing time within the WSDS requirements, WMOPS worked with detections, eliminating artifacts and stationary objects from the processing instead of subtracting a static sky from the images and generating a new source list from there. This, the detector characteristics, and the WISE spacecraft's survey cadence made WMOPS a uniquely designed subsystem. Beyond the linear-motion search algorithms themselves, it was these specifics, i.e. the way the surrounding software sifts through and filters input detections and output tracklets, that often determines the efficacy of any search software for a moving object survey. Figure 1 summarizes the basic flow of the WMOPS subsystem. WMOPS ran using the products of the Scan/Frame Pipeline. In particular, there were dependencies with the Artifact Identification and Instrumental Calibration in addition to Single-exposure detection lists.

A scan list was provided for each WMOPS run. Before frame processing could commence, WMOPS needed to assess which frames had complete coverage to determine the stationary objects. HEALPix, a package specifically designed and tested to conduct computations in spherical coordinate systems, was used to find neighboring frames, and corner positions of nearby frames were converted to the pixel coordinates of the central frame. A recursive algorithm was employed to determine if and by what frames complete coverage of the central frames area was obtained. The covered frames (nearly all of the frames over the course of the survey had complete coverage) in each of the scans were then read and the detections filtered according to several criteria. For NEOWISE data, WMOPS only functioned in W1 and W2. The broad criteria for rejection of detections on ingest were as follows: too low of signal-to-noise values in both W1 and W2, flagged artifacts of specified nature, specified extremely pathological PSF chi-squared fit variance, and excessively large numbers of non-stationary object detections-per-frame (see above). A critical step in the frame processing was the stationary-object-rejection (SOR), which initiated the spatial processing of the detections on a per-frame basis. The idea behind SOR was simple; detections from separate scans that did not move more than, or were co-located within, a minimum distance were rejected. The operational rejection limit for minimum moving-object motion was about 2 arcseconds, or a little less than the NEOWISE pixel size. To avoid wrap-around problems for the SOR step, WMOPS converted all positions (e.g. on the overlap frames) into the coordinates of the central frame being processed.

After SOR, the remaining detections were paired with each other according to a distance and time separation so that pairs representing hypothetical objects, which would have velocities greater than the fastest object that could have 5 detections in the NEOWISE survey, were excluded. These pairs were then fed through the CollapseTracklets routine, which connected pairs into tuples of 3 or more detections based on speed, direction of travel and position. These tuples were then stored in a collection of flat files based on the frame that spawned the tuple. For a tuple to be included in the output of the per-frame processing it must have reached a length of at least 3.

Tuples were subsequently grown by repeated runs of CollapseTracklets on tuples from overlapping regions of sky called globs. For a tuple to be included in the output of a glob it must have been of length not less than 5 detections. To permit an object to move from one glob to an adjacent, overlapping glob, no glob was ever processed at the same time as an overlapping glob, and output from a glob was used as input to all overlapping globs run after the former glob. At the conclusion of the processing of a glob, meta information about the glob was stored in the WMOPS legacy database, which then informed future WMOPS runs about previously processed globs whose output could be used as inputs to globs in said future run (as appropriately overlapped). Through this database, tuples could span not only spatial regions of processing in a single WMOPS run, but they could also grow to span multiple WMOPS runs.

The output tuples were then filtered to remove subset and duplicate tuples converted into tracklets (tracklet and tuple are sometimes used interchangeably) in the standard MPC reporting format and two files are generated and placed in a results directory: last.mpc and last.mpc.sids. These files contained the MPC-formatted output (see the MPC website) and the source_id list for each track. Orbit fits then were generated using the MPC Initial Orbit Determination (MPCIOD) and Digest2 (which performed great-circle orbit fits) software provided by the Minor Planet Center in executable form for use by WMOPS, and the orbital elements and fit residuals of those tracklets that converge to physical orbits were placed in the last.els file. This completed the activities of the WMOPS routine, which as its last act, kicked off the wmopsqa routine.

Figure 1 - Summary flow of the WMOPS subsystem. The ingesting, glob processing, and formatting were done primarily by the WMOPS Perl-based software. The quality assurance was done in a partly automated fashion by the wmopsQA routine and by eyes-on vetting of tracklets by the QA team. In the next-to-last QA filtering step, final vetting on individual tracks was done on the QA web pages for tracks with poor residuals from the Digest2 routine provided by the MPC. Final products were generated using the wmopsMPCGen routine and the duplicate-rejection routine was run on the final MPC-formatted list before sending the astrometry to the MPC.

The wmopsQA routine did the final automated filtering and generated the CGI-script files, lists, plots and thumbnail products for the eyes-on QA steps. There was a run-summary page as well as individual tracklet pages (Figure 2) generated for the QA. These pages were generated using the first.cgi and track.cgi scripts. Every tracklet that was not automatically rejected by the automated filters had the full set of eyes-on QA data products generated in its own separate QA sub-directory. Plots were generated using gnu-plot scripts and output in JPEG format. Thumbnails were generated with a scaling of +7σ, -3σ scaling.

The list of rejected and accepted tracklets were kept in the qa_state.txt file, which was modified by the submit.cgi script, which, in turn, was initiated by a button on the tracklet-level QA page. The qa_state.txt file was essentially a modification of a .sids (as in last.mpc.sids) file with the vetting status label ("Accept," "Artifact Reject," "Review," or "Reject") of the tracklet pre-pending the .sids line entry. After the eyes-on tracklet review was completed, in a QA filtering step, products were generated using the wmopsMPCGen routine and a final vetting on individual tracks was done via the web products for tracks with poor residuals from the Digest2 routine provided by the MPC. The wmopsMPCGen routine created the QA-vetted list of MPC-formatted tracks, and made crude estimates of R-band magnitude for the objects to aid follow-up by the general community. A routine to eliminate duplicate usage of detections in different tracklets was then run on the final report. At the request of the MPC, the final tracklet list was split into two reports: one for tracklets with Digest2 NEO probability scores greater than 65% and one containing all other tracklets. These two reports were then submitted to the MPC.

Figure 2 - Discovery data of 2014 HQ124, a PHA that came within 3 lunar distances of the Earth, in the WMOPS QA page raw image arrays. Sky-plane motion charts and text-based data are not shown here, but were generated as wmopsQA products. Note the stacked image in the final column of the top two rows, and the individual detection vetting buttons. Thumbnail images were critical to the successful identification of new objects.

IV.3.c.ii. Caveats

The 90% completeness value from the prime mission referred strictly to those objects with 5 or more detections that:

The minimum tracklet length validation requirement and the minimum sky-plane projected velocity imposed by the SOR defined the range of motion to which WMOPS was sensitive. For survey operations, the parallax motions below 5 arcseconds over 90 minutes (approximate time between WISE coverages near the ecliptic) corresponded to a distance of roughly 28 AU. The limit of five detections imposed an upper limit in that objects could have outpaced the survey cadence before the minimum number of detections could be obtained. The upper limit to the rate of motion perpendicular to the scan direction imposed by this requirement was roughly 3.3°/day, considering both an irregular cadence pattern as well as an evenly progressing survey. The maximum speed of a confirmed object detected by WMOPS was 4.3°/day (Figure 3), but the component perpendicular to the scan direction was 3.22°/day. The issue of completeness for the particular population of Near Earth Objects is discussed in Mainzer et al. 2011. ApJ 743, 156.

Figure 3 - Histograms of the sky-plane velocities of the objects reported to the MPC during the NEOWISE Reactivation mission. The velocities are median values across the detected tracklet and the right plot is shown in log-space to highlight the fastest-moving objects. The object with the highest sky-plane velocity was the NEO 2020 TK3, which in October of 2020 had a sky-plane velocity of 70.5°/day, and had a fortuitous geometry that allowed two detections to be recovered manually by Masiero et al. (2023, PSJ, 4,225) as it was passing through the overlap region between consecutive survey images. The fastest object detected by WMOPS was 2018 QT1, with an average projected sky motion of 4.3°/day, modestly exceeding the WMOPS theoretical "speed limit" of 3.3°/day.

Finally, the estimated visual wavelength magnitudes reported to the MPC, which may remain in their MPCAT-OBS data (see below), were estimated from IR fluxes and colors and are likely not accurate to better than 2 magnitudes. The standard deviation of these values relative to the true magnitude is expected to be in excess of 0.5 magnitudes.

IV.3.c.iii. Finding WMOPS Objects

The fastest means of finding WMOPS observed objects, beyond the available IRSA search engines, is to search through the MPCAT-OBS archive to extract particular objects using the WISE observatory code, C51. These will return the track that was reported to the MPC. A description of the tracks and how to obtain them is provided in section II.4.e of WISE All-Sky Release Explanatory Supplement.

Note that in addition to objects identified by WMOPS, the MPCAT-OBS archive also contains detections of objects found in the NEOWISE single-exposure images via manual recovery. The process of identifying these objects and the list of objects found via this method are described in Masiero et al. 2018, AJ, 156, 60.

To facilitate future analysis of the efficiency of WMOPS, including to support survey debiasing, each tracklet delivery to the MPC has also been archived in the form it was submitted. Note that the MPC performed data validation on all submitted tracklets, and so detections in the submitted files may have been removed as spurious prior to publication by the MPC. In a small number of cases, reported tracklets were later identified as entirely spurious, and so were removed from the MPC observation catalog at the request of the mission. It should be noted by users that because WMOPS performed a rolling analysis of the NEOWISE data, tracklets in a given delivery to the MPC often contained partial repeats of previously submitted tracklets, but under new tracklet names.

Newly discovered objects of interest, like NEOs, comets, and Centaurs, were issued Minor Planet Electronic Circulars (MPECs). These often reported observations from follow-up observers, in addition to the spacecraft observations.



Last update: 12 September 2024


previous page   back to table of contents   next page