1.2.3 Overview of the data processing and tools

DPAC responsibilities include preparatory simulations, spanning the three levels from pixels in the focal plane (GIBIS; Babusiaux et al. 2011), through simulated telemetry (GASS; Masana et al. 2008), to simulated DPAC data products (GOG and GUMS; Antiche et al. 2014). Simulations are being used both internally for testing, training, and validation and externally to support the astronomical community in preparing itself for the Gaia mission catalogues (Robin et al. 2012; Luri et al. 2014). The most recent versions of simulation products can be retrieved from the Gaia ESA Archive and are discussed in Chapter 2.

A common set of (Java) tools are in place for DPAC data processing. These ensure consistency of the data processing and avoid multiple, independent developments of similar concepts. The most important common tools are:

Configuration database (CDB)

this provides data related to the configuration of the spacecraft that could impact on data processing. Included are information such as the along-scan phasing offset of each CCD row and strip; the charge-injection configuration for each CCD (Section 1.3.4); the CCD window geometry (Table 1.2 and Figure 1.3); TDI-gate information for each CCD; etc.

Gaia parameter database (GPDB)

this provides useful mathematical, physical, and astronomical constants (for instance the value of $\pi$, the Solar-system ephemeris, and the speed of light in a vacuum to a significant number of digits), as well as mission-specific information and nominal, pre-flight spacecraft and payload design parameters. See Section 4.1.8 for details.

‘GaiaTools’ (GT)

this is the main common software library that provides utilities such as reading / writing data in accordance to the common data model (see Section 1.2.3) and related database querying facilities; plotting tools; mathematical tools (vector / array manipulation and quaternion operations); fitting algorithms; scanning law integration; timescale and calendaric transformations; and so on.

There is also a common acronym list, available on https://gaia.esac.esa.int/gpdb/glossary.txt.

The Gaia data-processing centres (DPCs) exchange data through the main database (MDB) located at DPCE (Section 1.3.5) following a hub-and-spokes topology. The MDB also houses the DPAC end data products from which the public data releases are produced. All DPCs as well as the MDB share a common data model. For more details, see Gaia Collaboration et al. (2016b).

The system architecture of the Gaia science ground segment is documented in O’Mullane et al. (2011).

The data flow in the Gaia science ground segment is described in Hernandez and Hutton (2015) and Mignard et al. (2008) and schematically visualised in Figure 1.11.

The daily processing of Gaia data includes the initial data treatment (IDT; Fabricius et al. 2016, see also Section 3.4.2) and first look (FL; Fabricius et al. 2016, see also Section 3.5.2), the astrometric verification unit (AVU; Section 3.5.1) processing, the RVS daily processing (Sartoretti et al. 2018), the Gaia photometric science alerts (GSA), and the Solar-system science alerts. More details are provided in Gaia Collaboration et al. (2016b).

The cyclic processing, encompassing the Intermediate Data Update (IDU; Section 3.4.2), the Astrometric Global Iterative Solution (AGIS; Section 4.4.2), the Global Sphere Reconstruction (AVU-GSR; Section 3.5.1), the Photometric Pipeline (PhotPipe; Chapter 5), the RVS pipeline (Chapter 6), the variable-star pipeline (Chapter 10), and the astrophysical-parameter pipeline (Apsis; Chapter 11), is detailed in Gaia Collaboration et al. (2016b).