9.2.2 Data accountability

Author(s): Alex Hutton and Enrique Utrilla

The integration process takes sources from different datasets and combines them based on their source ID. During this process, using information provided by the IDU crossmatch, previously existing sources can be deleted e.g. if they were based on observations that have been identified as spurious. Other sources might have been tagged as superseded if they were split into two or more new sources, or if several old sources were merged into a single one.

Traceability of the sources

The result of the previously described operations is summarized in the following table:

Table 9.1: Traceability of the sources
Type Initial count Deleted Removed by Split Removed by Merge Remaining
IGSL Source 987 693 514 7 907 472 11 749 990 364 997 278 603 038 774
New source (IDU-01) 2 062 191 820 182 815 939 18 622 238 526 949 198 1 333 804 445
HPM Addendum Source 3 058 1 506 8 1344 200
New source (IDU-02) 645 771 010 0 0 0 645 771 010
Total 3 695 659 402 190 724 917 30 372 236 891 947 820 2 582 614 429

Combining the initial and cycle 1 data, the new sources from IDU-02 and discarding the deleted and the superseded sources, the output dataset has 2,582,614,429 sources.

Of these sources, most were assigned observations from the cycle 02 data by IDU. A small number were not observed, but remain included in the dataset (principally IGSL sources brighter than 6th magnitude):

Table 9.2: Sources observed / not observed in cycle 2
Type Count Observed Count Not Observed Total Count
IGSL Source 603 028 880 9 894 603 038 774
New source from IDU-01 1 333 804 445 0 1 333 804 445
HPM Addendum source 168 32 200
New source from IDU-02 645 771 010 0 645 771 010
Total 2 582 604 503 9 926 2 582 614 429

Updates from AGIS-02

The outputs of the AGIS run for the cycle 2 (AGIS-02) were integrated into the dataset described in the previous subsection, which included sources with positions and proper motions supplied by different systems - including AGIS-01, IDU-01 and IDU-02, as well as data from the IGSL catalogue.

The distribution of the AGIS-02 updates against the types of source records is shown in the table below:

Table 9.3: Distribution of updates from AGIS-02
Source Type Origin 2-params 5-params Not Converged Not Processed Total
NEW IDU-02 174 865 789 1 310 193 34 616 513 25 853 584 236 646 079
NEW (merge) IDU-02 53 194 546 282 141 441 8 986 512 3 113 939 347 436 438
NEW (split) IDU-02 31 302 497 22 258 676 5 610 630 2 516 690 61 688 493
PERSISTENT HPM-Addendum 41 119 6 34 200
IDU-01 648 914 536 534 839 990 98 967 682 51 082 237 1 333 804 445
IGSL 103 971 247 494 849 188 3 544 902 673 437 603 038 774
Total 1 012 248 656 1 335 399 607 151 726 245 83 239 921 2 582 614 429
Notes. In this context, ‘Not Processed’ means that AGIS did not find any suitable observations matching the required quality criteria for the given source.

Of the sources retained in the integration, that is, those which have not been deleted or superseded by IDU-02, most received AGIS-02 updates. In the few cases where they did not, their previous values for positions and proper motions are maintained.

On a per-source level there is no mixing between AGIS-01 and AGIS-02 solutions. This means that a 2-parameter solution from AGIS-02 will remove any proper motions that were applied to a source from AGIS-01. Nevertheless proper motions from IDU and the IGSL are retained to provide support for future cross-match processing.

Distribution of position and proper motion originators

The following two tables summarise the origin of the positions and proper motions for the sources in the dataset generated for DR2.

The distribution of the positions of the sources is as follows:

Table 9.4: Distribution of position origins
Position Origin Count Percentage
AGIS-02 2 347 648 263 90.90%
IDU-02 80 697 868 3.12%
HPM-Addendum Source 40 0.00%
AGIS-01 64 301 751 2.49%
IDU-01 87 922 600 3.40%
IGSL 2 043 907 0.08%
Total 2,582,614,429 100.00%

The originators of the proper motions of the integrated sources is the following:

Table 9.5: Distribution of proper motion origins
Proper Motion Origin Count Percentage
AGIS-02 1 335 399 607 51.71%
IDU-02 4 915 627 0.19%
HPM-Addendum Source 81 0.00%
AGIS-01 4 645 0.00%
IGSL 41 200 632 1.60%
None 1 201 093 837 46.51%
Total 2 582 614 429 100.00%