skip to main content

gaia early data release 3 documentation

7.2 Data Consolidation

7.2.2 Data accountability

Author(s): Alex Hutton and Enrique Utrilla

The integration process takes sources from different datasets and combines them based on their source ID. During this process, using information provided by the IDU cross-match, previously existing sources can be deleted e.g. if they were based on observations that have been identified as spurious. Other sources might have been tagged as superseded if they were split into two or more new sources, or if several old sources were merged into a single one.

Traceability of the sources

The result of the previously described operations is summarised in Table 7.1.

Table 7.1: Traceability of the sources
Type Input Deleted Removed by Split Removed by Merge Added Output
IGSL Source 603 038 774 3 732 663 1 815 549 16 118 938 581 371 624
New source (IDU-01) 1 333 804 445 105 501 144 5 597 468 41 726 981 1 180 978 852
HPM Addendum 200 50 0 27 123
New source (IDU-02) 645 771 010 81 006 080 4 348 988 35 842 395 524 573 547
New source (IDU-03)
    as a new source 194 550 000 194 550 000
    as result of merge 45 887 691 45 887 691
    as result of split 25 260 950 25 260 950
Total 2 582 614 429 190 239 937 11 762 005 93 688 341 265 698 641 2 552 622 787

Combining the input data from data reduction cycle 02, the new sources from IDU-03 and discarding the deleted and the superseded sources, the output dataset has 2 552 622 787 sources. All sources were assigned at least one observation by the IDU cross match.

Updates from AGIS-03

The outputs of the second AGIS run for the cycle 3 (AGIS-03.2) were integrated into the dataset described in the previous subsection, which included sources with positions and proper motions supplied by different systems - including AGIS-01, AGIS-02, IDU-01, IDU-02, IDU-03 as well as data from the IGSL catalogue.

The distribution of the AGIS-03 updates against the types of source records is shown in Table 7.2.

Table 7.2: Distribution of updates from AGIS-03
Not Not
Source Type Origin 2-params 5-params 6-params Converged Processed Total
NEW IDU-03 179 039 547 768 236 13 017 569 1 724 648 194 550 000
NEW (merge) IDU-03 12 472 283 33 051 532 339 443 24 433 45 887 691
NEW (split) IDU-03 22 849 878 1 688 465 673 951 48 656 25 260 950
PERSISTENT IDU-02 215 419 351 211 457 213 92 521 229 4 289 332 886 422 524 573 547
HPM-Addendum 13 104 3 3 123
IDU-01 531 101 741 95 298 720 519 412 259 33 613 607 1 552 525 1 180 978 852
IGSL 66 253 555 278 706 238 235 588 193 784 981 38 657 581 371 624
Total 1 027 136 368 585 462 275 883 029 917 52 718 886 4 275 341 2 552 622 787
Notes. In this context, ‘Not Processed’ means that AGIS did not find any suitable observations matching the required quality criteria for the given source.

Of the sources retained in the integration, that is, those which have not been deleted or superseded by IDU-03, most received AGIS-03 updates. In the few cases where they did not, their previous values for positions and proper motions are maintained.

On a per-source level there is no mixing between AGIS solutions from different cycles. This means that a 2-parameter solution from AGIS-03 will remove any proper motions that were applied to a source from AGIS-02. Nevertheless proper motions from IDU and the IGSL are retained to provide support for future cross-match processing.

Distribution of position and proper motion originators

The following two tables summarise the origin of the positions and proper motions for the sources in the dataset generated prior to filtering in the archive for Gaia EDR3.

The distribution of the positions of the sources is shown in Table 7.3.

Table 7.3: Distribution of position origins
Position Origin Count Percentage
AGIS-03 2 495 628 560 97.77%
IDU-03 15 828 700 0.62%
AGIS-02 6 261 410 0.25%
IDU-02 3 249 943 0.13%
HPM-Addendum 1 0.00%
AGIS-01 4 027 522 0.16%
IDU-01 27 205 498 1.07%
IGSL 421 153 0.02%
Total 2 552 622 787 100.00%

The originators of the proper motions of the integrated sources is shown in Table 7.4.

Table 7.4: Distribution of proper motion origins
Proper Motion Origin Count Percentage
AGIS-03.2 1 468 492 192 57.53%
IDU-03 14 182 341 0.56%
AGIS-02 360 423 0.01%
IDU-02 3 015 135 0.12%
HPM Addendum 12 0.00%
AGIS-01 108 0.00%
IDU-01 0 0.00%
IGSL 21 569 221 0.84%
None 1 045 003 366 40.94%
Total 2 552 622 798 100.00%