3.4.2 Daily and cyclic processing

IDT includes several major tasks. It must establish a first on-ground attitude (see Section 3.4.5), to know where the telescopes have been pointing at any moment; it must calibrate the CCD bias (see Section 3.3.2), and it must calibrate the sky background (see Section 3.3.4). Only with those pieces in place, one can start thinking of processing actual observations.

For the observations, the first thing is to reconstruct all relevant circumstances of the data acquisition, as explained in Section 3.4.3. From the $G_{\mathrm{BP}}$ and $G_{\mathrm{RP}}$ fluxes, one can determine a source colour, and then proceed to determine the image parameters (see Section 3.4.11).

The final step of IDT is the cross-match between the on-board detections and a catalogue of astronomical sources, having filtered detections deemed spurious (see Section 3.4.13). One catalogue source is assigned to each detection, and if no one is found, a new source is added.

The Intermediate Data Updating (IDU) is one of the most demanding systems across DPAC in terms of data volume and processing power. IDU includes some of the most challenging Gaia calibration tasks and aims to provide:

• •

  updated on-ground attitude (see Section 3.4.5),

• •

  updated cross-match table using the latest attitude, geometric calibration, and source catalogue available (see Section 3.4.13),

• •

  updated calibrations for the bias, CCD features and astrophysical background (see Section 3.3.2 to Section 3.3.4),

• •

  updated instrument LSF/PSF model (see Section 3.3.5),

• •

  updated image parameters: locations and fluxes (see Section 3.4.11).

All these tasks have been integrated in the same system due to the strong relation between them. They are also run in the same environment, at the Marenostrum supercomputer hosted by the Barcelona Supercomputing Centre (BSC, Spain; Section 1.3.5). This symbiosis facilitates the delivery of suitable observations to the calibrations, and of calibration data to IDU tasks.

As anticipated in Section 3.1.1, IDU plays an essential role in the iterative data reduction; the successive iterations between IDU, AGIS, and PhotPipe (as shown in Figure 3.11) enable achieving the high accuracies envisaged for (the intermediate and) final Gaia catalogue(s).

IDU incorporates the astrometric solution from AGIS, resulting in an improved cross-match, but also incorporates the photometric solution from PhotPipe within the LSF/PSF calibration, resulting in improved image parameters. These improved results are the starting point for the subsequent iterative reduction loop.

In Gaia DR1, the image parameters came from IDT. However, from Gaia DR2 onward all image parameters result from IDU, thereby providing an improved completeness and consistency.