[updated 14 July 2009]
[note: click on any figure to see it full size]

Completeness (14 July 2009)

Update: the cumulative plots have changed direction in the sense that they are now summed from bright to faint. Also, flux bins were made with fine resolution at low fluxes and progressively coarser resolution at high flux levels. The functional form for the flux bin boundaries is an exponential of the form constant * exp(x/2), where x is a running index. The whole range of flux densities is covered in these plots, for each WISE band.

The first set of images shows the flux-binned completeness (top row) and total number of matches (bottom row) as a function of the truth flux. There is very little difference between the simulations with and without the pattern noise.

Completeness W1 Completeness W2 Completeness W3 Completeness W4

These plots show the cumulative completeness and match totals as a function of truth flux. While the results between the no noise pattern and noise pattern simulations are similar, there does appear to be a subtle increase in the total number of matches in the simulation with the noise pattern.

complete_cumul W1 complete_cumul W2 complete_cumul W3 complete_cumul W4

Reliability (30 June 2009)

Update: the cumulative plots have changed direction in the sense that they are now summed from bright to faint. No other changes have been made from the 27 June posting.

The following sets of plots show the cumulative reliability summed up from bright to faint sources.
W4 is most impacted by the noise pattern in this simulation. The overall reliability drops by approximately 2% with the noise.
W3 shows an unexpected increase in the reliability relative to the no pattern noise set of images. While there are fewer total sources that are detected, a larger fraction of them are being matched with objects in the truth list. This is most important for sources fainter than about mag=10. The most likely cause is that we are finding more chance matches in the noise pattern sim than in the no pattern noise sim. This will be investigated in the future.

Cumulative Histogram W1 Cumulative Histogram W2 Cumulative Histogram W3 Cumulative Histogram W4

Reliability output for W1
 No Pattern NoiseWith Pattern Noise
Mag binmatchestotal sourcesratiomatches
(cumul.)		total sources
(cumul.)		ratio
(cumul.)		 matchestotal sourcesratiomatches
(cumul.)		total sources
(cumul.)		ratio
(cumul.)
5.700 7 7 1.000 7 7 1.000 4 4 1.000 4 4 1.000
6.200 2 2 1.000 9 9 1.000 5 5 1.000 9 9 1.000
6.700 8 8 1.000 17 17 1.000 6 6 1.000 15 15 1.000
7.200 12 12 1.000 29 29 1.000 6 7 0.857 21 22 0.955
7.700 19 19 1.000 48 48 1.000 14 15 0.933 35 37 0.946
8.200 16 16 1.000 64 64 1.000 18 18 1.000 53 55 0.964
8.700 22 22 1.000 86 86 1.000 24 24 1.000 77 79 0.975
9.200 27 27 1.000 113 113 1.000 24 24 1.000 101 103 0.981
9.700 49 49 1.000 162 162 1.000 50 51 0.980 151 154 0.981
10.200 54 54 1.000 216 216 1.000 49 49 1.000 200 203 0.985
10.700 67 67 1.000 283 283 1.000 69 69 1.000 269 272 0.989
11.200 138 144 0.958 421 427 0.986 136 141 0.965 405 413 0.981
11.700 259 264 0.981 680 691 0.984 251 257 0.977 656 670 0.979
12.200 381 410 0.929 1061 1101 0.964 385 404 0.953 1041 1074 0.969
12.700 558 623 0.896 1619 1724 0.939 584 637 0.917 1625 1711 0.950
13.200 940 1104 0.851 2559 2828 0.905 934 1093 0.855 2559 2804 0.913
13.700 1725 2343 0.736 4284 5171 0.828 1660 2271 0.731 4219 5075 0.831
14.200 2914 4187 0.696 7198 9358 0.769 2942 4208 0.699 7161 9283 0.771
14.700 5021 7232 0.6941221916590 0.737 4792 7023 0.6821195316306 0.733
15.200 821911822 0.6952043828412 0.719 829011811 0.7022024328117 0.720
15.7001357420007 0.6783401248419 0.7021357120091 0.6753381448208 0.701
16.2001067215619 0.6834468464038 0.6981063015596 0.6824444463804 0.697
16.700 3122 5216 0.5994780669254 0.690 3183 5217 0.6104762769021 0.690
17.200 177 630 0.2814798369884 0.687 141 579 0.2444776869600 0.686
17.700 6 149 0.0404798970033 0.685 7 153 0.0464777569753 0.685
18.200 0 3 0.0004798970036 0.685 0 5 0.0004777569758 0.685
18.700 0 0 -NaN4798970036 0.685 0 0 -NaN4777569758 0.685
19.200 0 0 -NaN4798970036 0.685 0 0 -NaN4777569758 0.685

Reliability output for W2
 No Pattern NoiseWith Pattern Noise
Mag binmatchestotal sourcesratiomatches
(cumul.)		total sources
(cumul.)		ratio
(cumul.)		 matchestotal sourcesratiomatches
(cumul.)		total sources
(cumul.)		ratio
(cumul.)
5.700 8 8 1.000 8 8 1.000 6 6 1.000 6 6 1.000
6.200 3 3 1.000 11 11 1.000 3 3 1.000 9 9 1.000
6.700 5 7 0.714 16 18 0.889 6 6 1.000 15 15 1.000
7.200 14 14 1.000 30 32 0.938 8 9 0.889 23 24 0.958
7.700 17 17 1.000 47 49 0.959 13 14 0.929 36 38 0.947
8.200 21 21 1.000 68 70 0.971 22 22 1.000 58 60 0.967
8.700 20 20 1.000 88 90 0.978 22 22 1.000 80 82 0.976
9.200 33 33 1.000 121 123 0.984 34 34 1.000 114 116 0.983
9.700 46 46 1.000 167 169 0.988 44 45 0.978 158 161 0.981
10.200 67 68 0.985 234 237 0.987 57 57 1.000 215 218 0.986
10.700 72 76 0.947 306 313 0.978 80 81 0.988 295 299 0.987
11.200 171 178 0.961 477 491 0.971 161 168 0.958 456 467 0.976
11.700 279 290 0.962 756 781 0.968 272 279 0.975 728 746 0.976
12.200 435 462 0.942 1191 1243 0.958 448 475 0.943 1176 1221 0.963
12.700 709 822 0.863 1900 2065 0.920 750 849 0.883 1926 2070 0.930
13.200 1527 1969 0.776 3427 4034 0.850 1495 1897 0.788 3421 3967 0.862
13.700 3044 4397 0.692 6471 8431 0.768 3071 4392 0.699 6492 8359 0.777
14.200 5046 7501 0.6731151715932 0.723 4935 7450 0.6621142715809 0.723
14.700 701010644 0.6591852726576 0.697 680610437 0.6521823326246 0.695
15.200 879313828 0.6362732040404 0.676 877213722 0.6392700539968 0.676
15.700 4586 7262 0.6323190647666 0.669 4534 7169 0.6323153947137 0.669
16.200 1027 1762 0.5833293349428 0.666 1111 1886 0.5893265049023 0.666
16.700 155 385 0.4033308849813 0.664 163 390 0.4183281349413 0.664
17.200 3 12 0.2503309149825 0.664 5 12 0.4173281849425 0.664
17.700 0 0 -NaN3309149825 0.664 0 0 -NaN3281849425 0.664
18.200 0 0 -NaN3309149825 0.664 0 0 -NaN3281849425 0.664
18.700 0 0 -NaN3309149825 0.664 0 0 -NaN3281849425 0.664
19.200 0 0 -NaN3309149825 0.664 0 0 -NaN3281849425 0.664

Reliability output for W3
 No Pattern NoiseWith Pattern Noise
Mag binmatchestotal sourcesratiomatches
(cumul.)		total sources
(cumul.)		ratio
(cumul.)		 matchestotal sourcesratiomatches
(cumul.)		total sources
(cumul.)		ratio
(cumul.)
5.000 0 0 -NaN 0 0 -NaN 0 0 -NaN 0 0 -NaN
5.500 8 8 1.000 8 8 1.000 7 7 1.000 7 7 1.000
6.000 2 2 1.000 10 10 1.000 2 2 1.000 9 9 1.000
6.500 4 4 1.000 14 14 1.000 3 3 1.000 12 12 1.000
7.000 14 14 1.000 28 28 1.000 11 12 0.917 23 24 0.958
7.500 16 16 1.000 44 44 1.000 12 13 0.923 35 37 0.946
8.000 19 19 1.000 63 63 1.000 18 18 1.000 53 55 0.964
8.500 30 33 0.909 93 96 0.969 28 29 0.966 81 84 0.964
9.000 48 48 1.000 141 144 0.979 48 48 1.000 129 132 0.977
9.500 93 94 0.989 234 238 0.983 91 94 0.968 220 226 0.973
10.000 143 152 0.941 377 390 0.967 141 152 0.928 361 378 0.955
10.500 322 375 0.859 699 765 0.914 337 387 0.871 698 765 0.912
11.000 700 942 0.743 1399 1707 0.820 695 928 0.749 1393 1693 0.823
11.500 794 1272 0.624 2193 2979 0.736 757 1194 0.634 2150 2887 0.745
12.000 489 885 0.553 2682 3864 0.694 435 847 0.514 2585 3734 0.692
12.500 179 343 0.522 2861 4207 0.680 177 311 0.569 2762 4045 0.683
13.000 2 4 0.500 2863 4211 0.680 0 1 0.000 2762 4046 0.683
13.500 0 0 -NaN 2863 4211 0.680 0 0 -NaN 2762 4046 0.683

Reliability output for W4
 No Pattern NoiseWith Pattern Noise
Mag binmatchestotal sourcesratiomatches
(cumul.)		total sources
(cumul.)		ratio
(cumul.)		 matchestotal sourcesratiomatches
(cumul.)		total sources
(cumul.)		ratio
(cumul.)
5.000 4 4 1.000 4 4 1.000 5 5 1.000 5 5 1.000
5.500 9 9 1.000 13 13 1.000 8 8 1.000 13 13 1.000
6.000 4 4 1.000 17 17 1.000 3 3 1.000 16 16 1.000
6.500 17 18 0.944 34 35 0.971 14 15 0.933 30 31 0.968
7.000 43 44 0.977 77 79 0.975 32 34 0.941 62 65 0.954
7.500 107 129 0.829 184 208 0.885 104 127 0.819 166 192 0.865
8.000 208 320 0.650 392 528 0.742 195 292 0.668 361 484 0.746
8.500 233 407 0.572 625 935 0.668 232 389 0.596 593 873 0.679
9.000 195 392 0.497 820 1327 0.618 188 398 0.472 781 1271 0.614
9.500 146 305 0.479 966 1632 0.592 140 317 0.442 921 1588 0.580
10.000 16 41 0.390 982 1673 0.587 23 48 0.479 944 1636 0.577
10.500 0 0 -NaN 982 1673 0.587 0 0 -NaN 944 1636 0.577
11.000 0 0 -NaN 982 1673 0.587 0 0 -NaN 944 1636 0.577


Completeness (27 June 2009)

For the completeness analysis objects in the truth list were searched for in the coadd detected source list using a 1 arcsec search radius. No SNR, blending flag, or upper limit cuts were made to the coadd detected source list in this analysis. The completeness is defined as the ratio of the ratio of truth sources that matched MDEX sources to the total number of truth sources (NTruth-MDEX matches/NTotal truth).

The first set of images shows the flux-binned completeness (top row) and total number of matches (bottom row) as a function of the truth flux. There is very little difference between the simulations with and without the pattern noise.

Completeness W1 Completeness W2 Completeness W3 Completeness W4

These plots show the cumulative completeness and match totals as a function of truth flux. While the results between the no noise pattern and noise pattern simulations are similar, there does appear to be a subtle increase in the total number of matches in the simulation with the noise pattern.

complete_cumul W1 complete_cumul W2 complete_cumul W3 complete_cumul W4

Reliability (27 June 2009)

The reliability was determined by taking the ratio of the coadd sources matched to truth sources to the total number of coadd sources (NMDEX-Truth matches/Ntotal MDEX sources). A search radius of 1 arcsec was used. The MDEX source list was cut according to the following parameters:


The first two cuts ensure that the presented magnitudes are not upper limits; the second cuts make sure the sources are isolated singles.

There are subtle differences in the reliabilities between the two simulations. The differences for the brightest sources and faintest sources are due to small number statistics (see the tables below).

Discrete Histogram W1 Discrete Histogram W2 Discrete Histogram W3 Discrete Histogram W4

The following sets of plots show the cumulative reliability summed up from faint to bright sources.
W4 is most impacted by the noise pattern in this simulation. The overall reliability drops by approximately 2% with the noise.
W3 shows an unexpected increase in the reliability relative to the no pattern noise set of images. While there are fewer total sources that are detected, a larger fraction of them are being matched with objects in the truth list. This is most important for sources fainter than about mag=10. The most likely cause is that we are finding more chance matches in the noise pattern sim than in the no pattern noise sim. This will be investigated in the future.

Cumulative Histogram W1 Cumulative Histogram W2 Cumulative Histogram W3 Cumulative Histogram W4

Reliability output for W1
 No Pattern NoiseWith Pattern Noise
Mag binmatchestotal sourcesratiomatches
(cumul.)		total sources
(cumul.)		ratio
(cumul.)		 matchestotal sourcesratiomatches
(cumul.)		total sources
(cumul.)		ratio
(cumul.)
19.200 0 0 -NaN 0 0 -NaN 0 0 -NaN 0 0 -NaN
18.700 0 0 -NaN 0 0 -NaN 0 0 -NaN 0 0 -NaN
18.200 0 3 0.000 0 3 0.000 0 5 0.000 0 5 0.000
17.700 6 149 0.040 6 152 0.039 7 153 0.046 7 158 0.044
17.200 177 630 0.281 183 782 0.234 141 579 0.244 148 737 0.201
16.700 3122 5216 0.599 3305 5998 0.551 3183 5217 0.610 3331 5954 0.559
16.2001067215619 0.6831397721617 0.6471063015596 0.6821396121550 0.648
15.7001357420007 0.6782755141624 0.6621357120091 0.6752753241641 0.661
15.200 821911822 0.6953577053446 0.669 829011811 0.7023582253452 0.670
14.700 5022 7232 0.6944079260678 0.672 4792 7023 0.6824061460475 0.672
14.200 2934 4187 0.7014372664865 0.674 2963 4208 0.7044357764683 0.674
13.700 1779 2343 0.7594550567208 0.677 1725 2271 0.7604530266954 0.677
13.200 959 1104 0.8694646468312 0.680 956 1093 0.8754625868047 0.680
12.700 564 623 0.9054702868935 0.682 588 637 0.9234684668684 0.682
12.200 382 410 0.9324741069345 0.684 386 404 0.9554723269088 0.684
11.700 259 264 0.9814766969609 0.685 251 257 0.9774748369345 0.685
11.200 138 144 0.9584780769753 0.685 137 141 0.9724762069486 0.685
10.700 67 67 1.0004787469820 0.686 69 69 1.0004768969555 0.686
10.200 54 54 1.0004792869874 0.686 49 49 1.0004773869604 0.686
9.700 49 49 1.0004797769923 0.686 50 51 0.9804778869655 0.686
9.200 27 27 1.0004800469950 0.686 24 24 1.0004781269679 0.686
8.700 22 22 1.0004802669972 0.686 24 24 1.0004783669703 0.686
8.200 16 16 1.0004804269988 0.686 18 18 1.0004785469721 0.686
7.700 19 19 1.0004806170007 0.687 14 15 0.9334786869736 0.686
7.200 12 12 1.0004807370019 0.687 6 7 0.8574787469743 0.686
6.700 8 8 1.0004808170027 0.687 6 6 1.0004788069749 0.686
6.200 2 2 1.0004808370029 0.687 5 5 1.0004788569754 0.686
5.700 7 7 1.0004809070036 0.687 4 4 1.0004788969758 0.687

Reliability output for W2
 No Pattern NoiseWith Pattern Noise
Mag binmatchestotal sourcesratiomatches
(cumul.)		total sources
(cumul.)		ratio
(cumul.)		 matchestotal sourcesratiomatches
(cumul.)		total sources
(cumul.)		ratio
(cumul.)
19.200 0 0 -NaN 0 0 -NaN 0 0 -NaN 0 0 -NaN
18.700 0 0 -NaN 0 0 -NaN 0 0 -NaN 0 0 -NaN
18.200 0 0 -NaN 0 0 -NaN 0 0 -NaN 0 0 -NaN
17.700 0 0 -NaN 0 0 -NaN 0 0 -NaN 0 0 -NaN
17.200 3 12 0.250 3 12 0.250 5 12 0.417 5 12 0.417
16.700 155 385 0.403 158 397 0.398 163 390 0.418 168 402 0.418
16.200 1027 1762 0.583 1185 2159 0.549 1111 1886 0.589 1279 2288 0.559
15.700 4586 7262 0.632 5771 9421 0.613 4534 7169 0.632 5813 9457 0.615
15.200 879313828 0.6361456423249 0.626 877213722 0.6391458523179 0.629
14.700 701010644 0.6592157433893 0.637 680610437 0.6522139133616 0.636
14.200 5046 7501 0.6732662041394 0.643 4935 7450 0.6622632641066 0.641
13.700 3045 4397 0.6932966545791 0.648 3072 4392 0.6992939845458 0.647
13.200 1552 1969 0.7883121747760 0.654 1520 1897 0.8013091847355 0.653
12.700 721 822 0.8773193848582 0.657 763 849 0.8993168148204 0.657
12.200 436 462 0.9443237449044 0.660 448 475 0.9433212948679 0.660
11.700 279 290 0.9623265349334 0.662 272 279 0.9753240148958 0.662
11.200 171 178 0.9613282449512 0.663 161 168 0.9583256249126 0.663
10.700 72 76 0.9473289649588 0.663 80 81 0.9883264249207 0.663
10.200 67 68 0.9853296349656 0.664 57 57 1.0003269949264 0.664
9.700 46 46 1.0003300949702 0.664 44 45 0.9783274349309 0.664
9.200 33 33 1.0003304249735 0.664 34 34 1.0003277749343 0.664
8.700 20 20 1.0003306249755 0.664 22 22 1.0003279949365 0.664
8.200 21 21 1.0003308349776 0.665 22 22 1.0003282149387 0.665
7.700 17 17 1.0003310049793 0.665 13 14 0.9293283449401 0.665
7.200 14 14 1.0003311449807 0.665 8 9 0.8893284249410 0.665
6.700 7 7 1.0003312149814 0.665 6 6 1.0003284849416 0.665
6.200 3 3 1.0003312449817 0.665 3 3 1.0003285149419 0.665
5.700 8 8 1.0003313249825 0.665 6 6 1.0003285749425 0.665

Reliability output for W3
 No Pattern NoiseWith Pattern Noise
Mag binmatchestotal sourcesratiomatches
(cumul.)		total sources
(cumul.)		ratio
(cumul.)		 matchestotal sourcesratiomatches
(cumul.)		total sources
(cumul.)		ratio
(cumul.)
13.500 0 0 -NaN 0 0 -NaN 0 0 -NaN 0 0 -NaN
13.000 2 4 0.500 2 4 0.500 0 1 0.000 0 1 0.000
12.500 179 343 0.522 181 347 0.522 177 311 0.569 177 312 0.567
12.000 489 885 0.553 670 1232 0.544 435 847 0.514 612 1159 0.528
11.500 794 1272 0.624 1464 2504 0.585 757 1194 0.634 1369 2353 0.582
11.000 700 942 0.743 2164 3446 0.628 695 928 0.749 2064 3281 0.629
10.500 322 375 0.859 2486 3821 0.651 337 387 0.871 2401 3668 0.655
10.000 143 152 0.941 2629 3973 0.662 141 152 0.928 2542 3820 0.665
9.500 93 94 0.989 2722 4067 0.669 91 94 0.968 2633 3914 0.673
9.000 48 48 1.000 2770 4115 0.673 48 48 1.000 2681 3962 0.677
8.500 31 33 0.939 2801 4148 0.675 28 29 0.966 2709 3991 0.679
8.000 19 19 1.000 2820 4167 0.677 18 18 1.000 2727 4009 0.680
7.500 16 16 1.000 2836 4183 0.678 12 13 0.923 2739 4022 0.681
7.000 14 14 1.000 2850 4197 0.679 11 12 0.917 2750 4034 0.682
6.500 4 4 1.000 2854 4201 0.679 3 3 1.000 2753 4037 0.682
6.000 2 2 1.000 2856 4203 0.680 2 2 1.000 2755 4039 0.682
5.500 8 8 1.000 2864 4211 0.680 7 7 1.000 2762 4046 0.683
5.000 0 0 -NaN 2864 4211 0.680 0 0 -NaN 2762 4046 0.683

Reliability output for W4
 No Pattern NoiseWith Pattern Noise
Mag binmatchestotal sourcesratiomatches
(cumul.)		total sources
(cumul.)		ratio
(cumul.)		 matchestotal sourcesratiomatches
(cumul.)		total sources
(cumul.)		ratio
(cumul.)
11.000 0 0 -NaN 0 0 -NaN 0 0 -NaN 0 0 -NaN
10.500 0 0 -NaN 0 0 -NaN 0 0 -NaN 0 0 -NaN
10.000 16 41 0.390 16 41 0.390 23 48 0.479 23 48 0.479
9.500 146 305 0.479 162 346 0.468 140 317 0.442 163 365 0.447
9.000 195 392 0.497 357 738 0.484 188 398 0.472 351 763 0.460
8.500 233 407 0.572 590 1145 0.515 232 389 0.596 583 1152 0.506
8.000 208 320 0.650 798 1465 0.545 195 292 0.668 778 1444 0.539
7.500 108 129 0.837 906 1594 0.568 104 127 0.819 882 1571 0.561
7.000 43 44 0.977 949 1638 0.579 32 34 0.941 914 1605 0.569
6.500 17 18 0.944 966 1656 0.583 14 15 0.933 928 1620 0.573
6.000 4 4 1.000 970 1660 0.584 3 3 1.000 931 1623 0.574
5.500 9 9 1.000 979 1669 0.587 8 8 1.000 939 1631 0.576
5.000 4 4 1.000 983 1673 0.588 5 5 1.000 944 1636 0.577

This analysis is a joint effort by Chris Gelino and Sergio Fajardo-Acosta. Chris headed the Reliability analysis and Sergio headed the Completeness analysis.
27 June 2009