1.3 Release framework

For the time interval covering Gaia DR1, the spacecraft subsystem configuration is composed of nominal units (A-chain) with the exception of the Micro Propulsion Subsystem (MPS) which was switched to the redundant B-chain after a Safe Mode recovery triggered by a spurious performance change of MPS Thruster 3A (MT3A) on 17 July 2014. After testing of both MPS chains (which both performed nominally), the decision was made to use the B-side due to the MT3A performance issues. See Gaia Collaboration et al. (2016b) for more details.

Numerous Video Processing Unit (VPU) resets have been experienced during the early part of the mission. The thermal effect of having the associated row of CCDs automatically switched off was mitigated to a large extent with the introduction (on 18 September 2014) of an automatic recovery action on-board. Since the deployment of the VPU software version 2.8 in April 2015, no VPU resets have occurred.

A small movement of the M2 mirrors was performed to improve the focus of the following field-of-view in October 2014 and the preceding field-of-view in August 2015. For more details, see Section 1.3.3 and Gaia Collaboration et al. (2016b).

Telescope decontamination operations were performed in September 2014 and June 2015, scheduled after ice build-up had progressed beyond acceptable levels. For more details, see Section 1.3.3 and Gaia Collaboration et al. (2016b).

During December 2014, the MOC introduced open-loop ground automation (i.e., periods during which Gaia transmits science data to the ground stations without any intervention from MOC staff) and extra ground-station bookings to allow for increased science-data down link. The science-data return to ground was thereby increased along with the minimisation of data latency for the photometric and Solar-system alert projects.

A Lunar transit with approximately 0.35% of the Sun’s disk obscured occurred on 11 November 2014. The event was visible in multiple subsystems on the spacecraft (i.e., temperatures, MPS reaction to Solar radiation pressure, etc.). A similar, but larger (5%) event took place in November 2015 (which is not covered by Gaia DR1).

The necessity for a monthly Station-Keeping Manoeuvre (SKM, i.e. the maintenance of the desired orbit around L2) has diminished throughout the mission, the cadence now being between 3 and 4 months.

For the time interval covering Gaia DR1, all elements on the focal plane were operating correctly and as expected. There have been no major issues with the PEM-CCD couples and all 106 devices are functioning within, or close to, specification (only one device currently slightly exceeds the readout-noise specification). In the absence of hardware issues, the most important parameter affecting the characteristics and performances of the focal-plane elements is the temperature.

Amongst the many different temperature sensors on-board the satellite, there are three placed close to the detector array. They are distributed across the focal plane with one placed near the SM1 CCD on row 1, one placed near the AF5 CCD on row 4, and the third one near the RVS3 CCD on row 5. Shown in Figure 1.2 is a plot of the readings obtained from these three sensors after convolving the data through a one-hour-wide running-average smoothing kernel. One can immediately note two temperature increases. These correspond to controlled heating events which were carried out in order to decontaminate the mirrors. Any future heating events will display a signature similar to these two heating events (note that the RVS heaters were switched off as part of a one-off test before the second decontamination in this time range, so, under nominal circumstances, this temperature drop will not be repeated). The top axis on Figure 1.2 displays the number of days after Gaia launch, while shown on the bottom axis are the equivalent values but displayed in a unit of six-hour spacecraft revolutions.

When thermal equilibrium is reached after each heating event, it can be observed that the focal-plane temperature is extremely stable. Large temperature changes can dramatically affect detector characteristics (such as CTI or gain), so long-term stability is an important aspect of the performance. There is an obvious gradient of a few degrees over the focal plane (from sensor-to-sensor), but this gradient remains stable, and the operating temperatures of all devices are close to the target value of $-{110}^{\circ }$ Celsius. There is a very slow and gradual long-term evolution of the temperature readings due to the variable distance of the spacecraft to the Sun and, obviously, this changes with a period of a year, but this is a very small effect.

The L${}_2$ environment offers many advantages for the operation of astronomical observatories, such as high thermal stability. Due to the large distance of L${}_2$ from Earth ($\sim$1.5 million km), the effect of geomagnetically trapped charged particles is not a concern. However, it follows that L${}_2$ lacks the shielding of the Earth’s magnetosphere and is thus vulnerable to impacts from ionised particles from other sources. Indeed, the L${}_2$ radiation environment is dominated by particles (mostly protons) from the effectively isotropic cosmic rays and the more directional Solar eruptive events. The cosmic-ray component is expected and observed to be rather steady throughout the mission, with the number of impacts expected and observed to vary smoothly and to be anti-correlated with the Solar cycle (see Section 1.3.3). In contrast, the impacts of particles from Earth-directed Solar events are sporadic with more events expected and observed around the time of Solar maximum.

The most worrying effects of radiation on the Gaia science performances are expected to be due to performance degradations of the detectors over time. The most important effects are due to two main sources: 1) ionising radiation damage, and 2) non-ionising radiation damage.

The effects of ionising radiation on the detectors are cumulative and occur when charge builds up on parts of some electronic devices which can result in threshold shifts, charge leakage, etc. A special calibration procedure is periodically run on-board to monitor the accumulation of charge in the oxide layers of the CCD detectors; this is further described in Section 1.3.3.

The lattice displacement damage to the CCD silicon layers is the biggest radiation threat to the achievement of the Gaia science performance goals. The generation of new energy levels between the valence and conduction bands in the silicon produces crystal defect sites that trap electrons from traversing charge packets, to be released from the trap at some later moment in time. This results in a smearing of the images that are readout from the CCD that cannot be calibrated in a simple manner. Indeed, Charge Transfer Inefficiency (CTI) of the CCDs was identified as a potential threat to the mission at an early stage. CTI in the transfer through the serial (readout) register is discussed in Section 1.3.3; however, Gaia is most sensitive to CTI in the image area (along-scan direction).

The most important hardware mitigation for along-scan CTI effects is the periodic injection of charge into the devices. This is carried out every $\sim 2$ seconds for AF1 and AF2-9 devices and $\sim 5$ seconds for BP and RP devices. This scheme sees four contiguous lines of charge injected into the CCD and clocked out through the $4500$ TDI lines. This serves to periodically fill trapping sites with injected electrons, thus preventing the trapping of photo-electrons. Indeed, for those traps with characteristic release time scales on the order of seconds, these traps will be kept filled over the period of the charge injection, thus keeping them effectively permanently filled and thus rendered effectively inactive. The regular presence of the injection features in the data stream also means that the CTI effects on the injections can be used to monitor the evolution of the trapping and release effects on the detector over time. One such diagnostic is what is known as the First Pixel Response (FPR) and is measured by computing the number of electrons removed from the first injected pixel line through trapping (in practise, we also need to account for trapping from subsequent lines).

Shown in Figure 1.3 are the FPR results (normalised by the charge-injection level) for the AF7 CCD on row 4. The steady increase in CTI is apparent and is thought to be due to the low-energy tail of the Galactic cosmic-ray impacts. The two step increases are correlated with two Earth-directed proton events.

The almost linear degradation in the transfer efficiency is noticeable and rather similar for all devices. However, the effects of the Solar proton events are heavily affected by the differential shielding across the focal plane. Therefore, the net CTI observed in each device is a convolution of the (low-level) initial pre-flight CTI due to traps generated during the manufacturing process, the effect of the cosmic-ray impacts, and the effects of the Solar protons. Shown in Figure 1.4 is the result of the extrapolation of in-flight FPR measurements to the nominal end-of-mission time for Gaia. The higher CTI FPR values for the RP CCDs are apparent and caused by the higher inherent along-scan transfer inefficiences for these devices that were already measured pre-launch. Also apparent in this figure is a pattern showing higher FPR values towards the centre of the focal plane, which is a result of the shielding. However, the most important result that can be derived from these data is that the now-predicted end-of-mission CTI values are approximately an order of magnitude less than was feared before launch.

Furthermore, offline studies of the image shapes as a function of the time since charge injection have confirmed that the effects of CTI in the along-scan direction are low. A full analysis of the effects of CTI on the science data will become possible with time as more calibrations are carried out in the iterative data processing. Note that no periodic charge injection occurs in the SM and RVS devices, however. The CTI in the along-scan direction in these (and the other) devices is monitored through special calibration activities that take place every three to four months. The results are consistent with those presented here for the AF, BP, and RP CCDs. See Crowley et al. (2016) for a full discussion of the effects of the radiation environment on the Gaia detectors over the first two years of the mission.

A common technique for monitoring the CTI in the serial (readout) registers of CCDs is to generate charge in the image area and then monitor any trailing into the post-scan samples after the charge is transferred through the serial register. Since the Gaia VPUs are not capable of generating nominal packets containing post-scan samples, a special calibration is run periodically where five different levels of charge are injected into each science device (apart from SM devices, where the definition of the CCD operating mode precludes the acquisition of post-scan pixels). At the same time, ‘Virtual Objects’ (VOs) are placed over the end of the image area (in order to monitor the injected charge level) and are also used to acquire post-scan pixels. To circumvent the problem where post-scan samples cannot be stored in nominal packets, the engineering functionality of SIF is used to gather the raw PEM data which is subsequently down linked in a special packet with high priority. Since radiation damage has been lower than expected, it is currently sufficient to run this activity with a cadence of somewhere between three to four months.

At time of writing (mid 2016), there have been seven runs of the serial-CTI calibration in-flight. These runs allow the tracking of the evolution of the CTI in the serial register. However, it is found that the CTI is still dominated by the traps that have been present since device manufacture. Shown in Figure 1.5 are data derived from each run of the activity so far for the AF6 device on CCD row 6, and for the highest injection level. The co-added, high S/N cuts in the across-scan direction show the two pre-scan samples, followed by image-area samples showing the injected charge, and then the trailing into the post-scan samples due to the release of the injected electrons from traps in the serial register. In Figure 1.6, the very slow evolution of the serial CTI curve from run-to-run is readily apparent.

This slow evolution of the serial CTI (the continued domination by pre-flight, manufacturing traps) is apparent all across the focal plane. In summary, CTI in the serial register is still dominated by the traps inherent to the manufacturing process and the radiation-induced degradation in CTI, whilst clearly measurable, is of the order of a few percent of the pre-flight CTI ($\sim$$1$–$3$% for AF/BP devices and $\sim$$3$–$7$% for the thicker red devices, which had much lower native CTI to begin with). So, similar to the situation for the CTI in the along-scan direction, the CTI effects of radiation damage in the serial register are low.

As discussed in detail in Section 2.3.5, the electronic readout of the Gaia CCDs suffers from variations within a TDI line which have a non-trivial dependency on the exact readout scheme. Since most CCDs are operated in a windowing mode, where the exact locations of the windows depend on the sky, it follows that a constant reconfiguration of the readout scheme is applied to most devices. There is a model of the offset variation as a function of various readout parameters that needs to be calibrated periodically throughout the mission in order to properly correct these offsets. Some details on the various effects are discussed in Section 2.3.5; here we discuss only the status of the special on-board calibration that is carried out to calibrate the model.

During the non-uniformity calibration activity, the VPUs are, one-by-one, taken out of operational mode and artificial ‘Virtual Object’ (VO) detections are placed into the VPU algorithms. These detections result in empty windows to be readout from each device as the objects are propagated across the various science CCDs in the row controlled by that VPU. The VO patterns are designed to sample different readout configuration timings and are used to calibrate the offset non-uniformity model.

The nominal cadence of each run has settled to be between three to four months and, to date, the activity has been performed on-board nine times. The data gathering has mostly been successful. However, some issues related to the CCD gates and an on-chip cross-talk effect during gate-activation commands resulted in some optimisations being performed for specific devices in order to get the highest-quality data.

All CCDs are manufactured with a number of non-perfect, or defected, pixels. These can be manifest in a number of ways, but typically there will be a number of pixels which are under-responsive in comparison to the surrounding population, and a number that exhibit elevated signal levels, even in darkness. Due to the TDI operation of the Gaia CCDs, such pixel defects will be smeared out in some form due to the fact that the signal has been integrated over all $4500$ CCD image-area pixels in the column before reaching the serial register. Of course, for gated observations, the effective integration distance is shortened, so it may be that a defect pixel far from the serial register produces an effect in the non-gated data, but not in (all or some of) the gated data. In addition to these fabrication cosmetics, there are radiation-induced defects created and evolving in-orbit.

For Gaia, apart from the standard need to characterise CCD defects for the processing of the science data, it is also important to take them into account in order to ensure correct operation of the on-board detection (the SM CCDs) and confirmation (the AF1 CCDs) chains. Defects and evolution in dark-signal non-uniformity (DSNU) need to be tracked and updated on-board.

The locations and characteristics of these defects were diagnosed pre-flight and stored in the VPU memories for the on-board processing of the SM, AF1, and RP windows (the RP-window data is used on-board to provide a red magnitude for each object which is used as input when deciding whether an RVS window should be assigned for an object or not). So far, only one on-board change has been required in order to update the DSNU table for one SM CCD which contains a weak ($\sim$15 ADU), radiation-induced hot column.

When analysing the whole focal plane using the analysis of defect pixel evolution as described in Section 2.3.4, it is found that the number of radiation-induced defects is low. The sensitivity of the analysis to weak defects will increase during later data-processing cycles and more detailed statistics will become available with progressing mission duration (for further detail on defect pixel evolution, see Crowley et al. 2016).

Radiation-induced generation of electron-hole pairs in the CCDs cause a gradual accumulation of positive charge on the CCD gate oxide layers over time. The end result is an effective change in the potentials under the gates of each of the CCDs, which is termed a ‘flat-band voltage shift’. The CCD operating point is set to tolerate the maximum expected end-of-mission flat-band voltage shift of $0.5$ V, but the trend over time needs to be monitored in case steps need to be taken to avoid any degradation in the performances of the devices. In principle, any effect can be compensated for by changing the operating voltages of the devices by an appropriate amount. A periodic run of a special calibration activity on-board is required to monitor the evolution.

During this activity, charge-injection data is acquired from every CCD and the potential under the injection drain ($V_{\mathrm{ID}}$) is successively increased from a low value of $12.5$ V in order to find the $V_{\mathrm{ID}}$ value at which charge stops being injected (this point, the turn-off voltage, corresponds to the point at which the $V_{\mathrm{ID}}$ equals the potential under the injection gate). The potential under the gate will evolve with increased charge accumulation in the oxide, but the potential under the drain should remain unaltered since it is connected directly to the silicon. Thus, the change in operating point over time diagnoses the flat-band voltage shift. The activity is being run throughout the mission with a cadence of $\sim$6–12 months; so far, it has been run on-board three times, including a dry-run of the activity shortly after launch.

The average and standard deviation for the on-ground-measured turn-off voltages for the 106 flight devices is $15.574\pm 0.258$ V. A comparison with in-orbit data so far shows no clear evidence for a measurable flat-band voltage shift beyond the measurement noise for any one device. However, the mean shift over all devices between the on-ground measurement and the June 2015 on-board measurements is $+0.005\pm 0.027$ V. Therefore, after extrapolation, it is currently expected that flat-band voltage shifts will not cause detector performance issues before the end-of-mission. It should be noted that for future runs of this activity, the $V_{\mathrm{ID}}$ sampling will be optimised for each CCD in order to reduce measurement noise.

The object detection algorithms running on-board Gaia scan the images coming from each Sky-Mapper (SM) CCD in search for local flux maxima. For each local maximum, spatial-frequency filters are applied over the flux distribution within a $3\times 3$ samples window centred around the local flux maximum. Objects failing to meet star-like PSF criteria (too sharp or too smooth PSF) are rejected. For full details on the object rejection strategy, see de Bruijne et al. (2015).

This filtering mechanism takes place at the SM CCDs but is followed by a confirmation stage in the AF1 CCDs that further removes those objects that, having been detected in SM, do not re-appear in AF1, typically cosmic rays or Solar protons. Detection-process statistics (e.g., the rejected number of cosmics) are telemetered to ground in the form of auxiliary-science-data packets. These are only counters (in fact a ‘by-product’ of the detection chain) and neither energy nor nature of the impacting radiation particles are provided.

Figure 1.8 shows the typical prompt-particle-event rates extracted from the auxiliary science telemetry. The prompt-particle-event time lines contain two main features, namely (1) ‘peaks’ of counts (expectedly) concentrated around times of increased radiation activity, and (2) a ‘background’ count rate (at $\sim$40 counts s${}^{-1}$) which is present at all times, on top of which Solar activity shows up.

The predicted background rate measured at each CCD (SM or AF) produced by an incident isotropic flux of cosmics can be estimated using Sullivan (1971) and is given by:

where $A$ is the effective detection area of the given detector in cm${}^2$ ($A\simeq 17.1$ cm${}^2$ for SM CCDs) and $F$ is measured in particles cm${}^{-2}$ s${}^{-1}$. For a typical (Barth et al. 1999; Catalano et al. 2014) particle background at L${}_2$ of 5 protons cm${}^{-2}$ s${}^{-1}$, the expected prompt-particle-event rate measured by Gaia at L${}_2$ should be $\simeq$42.75 counts s${}^{-1}$, which is in good agreement (10% difference) with actual measured rates. The peaks in the prompt-particle-event counters are clearly correlated with increased Solar activity. We thus compared Gaia prompt-particle-event counters against other spacecraft’s radiation monitoring instruments data (ACE: http://www.srl.caltech.edu/ACE/; and GOES: http://www.swpc.noaa.gov/products/goes-proton-flux) over the same time scales to perform an external consistency check. Figure 1.9 shows a couple of such qualitative comparisons for ‘strong’ Solar events. We found systematically that Gaia prompt-particle-event counters are following the increase in the Solar radiation particles (protons) measured by the other dedicated radiation monitoring instruments. The Gaia focal plane being shielded inside the thermal tent structure of the spacecraft should systematically measure lower particle fluxes than the other instruments, as a good fraction of the incident particles on the spacecraft will be blocked by the thermal-tent materials. Both analyses give confidence on the robustness of the Gaia on-board autonomous detection/rejection algorithms.

Cosmics impacting on a CCD produce (displacement) radiation damage. The traps created in this process degrade the nominal charge-transfer times of the photo-electrons during the readout in TDI mode as charge is trapped and released. This degradation in the charge-transfer efficiency – or CTI effect ($\mathrm{CTE}=1-\mathrm{CTI}$) – is known to introduce a bias in the star’s centroid position estimation (Prod’homme et al. 2012). On-board measures are taken to mitigate this effect, e.g., activation of regular charge injections that help fill the existing traps.

Gaia was launched in December 2013, just after the maximum of Solar cycle 24. Solar activity is therefore expected to progressively decrease towards Solar minimum (see Figure 1.10), resulting in a ‘benign’ (a-priori) situation from a CCD radiation damage perspective.

The Gaia prompt-particle-event counters are regularly used to monitor the instantaneous radiation environment experienced by Gaia at L${}_2$ and its possible implications for radiation damage. For a quantitative assessment, however, first-pixel-response analysis is used to study the level of radiation damage accumulated by the Gaia CCDs.

Before launch, it was expected that the Gaia telescopes focus would be very stable. In this way, the best focus would be obtained during commissioning and kept constant throughout the mission. The M2 mirror mechanisms (M2MMs) were not envisaged to be used during the nominal mission, except in case of a major anomaly. However, the focus has continuously evolved during the mission. Due to its direct impact on the mission performance (astrometric precision and spectral resolution), the First Look pipeline provides a suite of diagnostics, monitoring its behaviour on a daily basis. The astrometric precision is the most sensitive parameter, and can be estimated via the Cramér-Rao metric, as explained in Lindegren (1978, 2010):

where $\sigma$ is the maximum along-scan centroid location precision for a given star and CCD transit (that is, the along-scan astrometric information contained in an individual sample image), $N_{e^{-}}$ is the total number of electrons for an infinite focal plane, $S_k$ is the normalised PSF multiplied by $N_{e^{-}}$, that is the number of electrons collected from the star for each pixel $k$, $S_k^{\prime }$ is the derivative of $S_k$ with respect to the along-scan pixel coordinate, $r$ the readout noise (in electrons), and $b$ the homogeneous sky background (in electrons). This equation can be used to estimate how close any given PSF or LSF centroiding method is to the maximum theoretical performance. However, a very good knowledge of the payload and each star is needed to provide accurate estimates, including oversampled average PSFs with colour and across-scan dependence. However, relative measurements are much simpler:

where ${\sigma }_0$ is a normalised metric independent of the actual stellar flux for bright stars where the Poisson noise dominates. For these objects, ${\sigma }_0$ can be estimated using the actual sample values for $S_k$ and finite differences for $S_k^{\prime }$. An average value can thus be provided and used for quick look payload health checks on a timely basis. The approximate method thus provides relative and not absolute measurements, and is affected by additional effects such as stellar colour, across-scan position, or sub-pixel position. However, its simplicity provides a very useful proxy to determine the overall focus evolution trend, and whether a refocus is needed.

Figure 1.11 shows the history of the AF image quality for each field of view averaged over all CCDs throughout the mission, estimated using all unsaturated, un-gated, class-0 observations. Several things are apparent. First, the values are always close to and mostly below one, as expected for good-quality, near-diffraction-limited telescopes. Second, the focus has never been completely stable. Third, there are sharp discontinuities introduced by payload decontamination. Some optical quality is always recovered as a result. However, fine adjustment (refocus) has sometimes been applied afterwards. Fourth, the degradation slope is becoming smaller as the mission evolves, which points to an increasingly stable payload. Table 1.3 lists the payload decontamination and refocusing events that have happened during the interval covered by Gaia DR1 (commissioning has been explicitly excluded).

More detailed information is provided by the daily First Look diagnostics. Figure 1.12 shows the average relative AF image quality for each CCD and field of view, as estimated during a typical interval (OBMT: 3536.78056 – 3540.50363 [revs], UTC: 2016-03-31T00:54:21 – 2016-03-31T23:14:39). The field-of-view dependence of the astrometric quality is apparent. The optimum values are typically obtained for CCD strips AF5 to AF8. These variations are expected for the large field-of-view covered by Gaia.

It became clear in the early commissioning phase that the response in AF was degrading rapidly, and especially for observations in the following field of view. In addition, the degradation had a clear colour dependence.

After heating of the mirrors and the focal plane, the response was fully recovered, showing that the extinction was due to ice deposits at least on the mirrors. The temperature at which the recovery took place was consistent with a contaminant of water ice, and this was and still is considered the most likely explanation.

After decontamination, the response degradation soon returned, and a total of five de-contaminations have been carried out so far (mid 2016). Fortunately, the build up of new ice layers is slowing down substantially, as demonstrated in Figure 1.13, and only a very small number of de-contaminations will still be needed in the future.

The response has been monitored closely since the early commissioning, using the photometry from the Tycho-2 catalogue (Høg et al. 2000). Tycho-2 was chosen because it contains a sufficient number of sources all over the sky, easy to identify, and with good-quality astrometry and photometry. At the same time, average relations between Tycho photometry and the Gaia broad-band $G$, $G_{\mathrm{BP}}$, and $G_{\mathrm{RP}}$ magnitudes had already been established before launch, using methods explained in Jordi et al. (2010).

The precise photometric relations depend not only on the spectral type of the star, but also on the amount of reddening. The reddest sources were therefore not used in the response monitoring. Even so, Figure 1.13 shows artificial response variations that occur in step with changes in the average colour of the stars, especially for the $G_{\mathrm{RP}}$ magnitudes. This, however, does not affect the overall trend, which is the objective of the monitoring.

As a consequence of the response variation, there has been a corresponding variation in the $G$ magnitude limit for the Gaia observations. This is so because the detection limit is defined as a certain, fixed flux level, so when contamination builds up, the limit is (up to) a few tenths of a magnitude brighter.

The Video Processing Units (VPUs; Section 1.1.3) assign, among others, the window size, sampling class, and observation priority to each object that is confirmed for observation. The VPU algorithms (VPAs) are highly parametrised. The configuration of the VPU/VPA parameters, which is carefully tracked in the CDB, determines which objects are observed. During the time interval covering Gaia DR1, the VPU application software (VPAs) was updated on 24–28 April 2015 from version 2.7 to 2.8. Version 2.8 allows enhanced flexibility in the areas of the RVS across-scan window size and RVS faint limit, the AF2-9 faint-star across-scan window size, the AF1 coarse-confirmation parametrisation, and suppression of SM double detections.

Down-link priority settings and data-deletion settings are user defined and tracked in the CDB. In general, bright stars have higher priority for being telemetered to ground first and are protected on-board from being deleted. During the time interval covered by Gaia DR1, a few changes to these settings were made: the non-unity multiplexing ratios were multiplied by a factor 10 on 28 October 2014, the faint-end granularity for SP2 packets was improved on 10 November 2014, and the position of the ASD4 and BAM File_IDs in the PDHU down link priority table was swapped on 16 September 2015 so that the on-board procedure that stops the down link in case link problems are detected affects BAM data rather than ASD4 data.

DPCE is part of the Gaia Science Operations Centre (SOC) at ESAC, Madrid, Spain. DPCE operates software from:

• •

  CU1:

  • –

    Main DataBase (MDB);

  • –

    MOC Interface Task (MIT);

  • –

    De-compression and Calibration Services (DCS);

  • –

    Payload Operations System (POS);

  • –

    Gaia Transfer System (GTS);

• •

  CU3:

  • –

    Initial Data Treatment (IDT);

  • –

    First Look (FL);

  • –

    Astrometric Global Iterative Solution (AGIS);

• •

  DPCE:

  • –

    IDT/FL DataBase (IDTFLDB);

  • –

    Daily PipeLine (DPL);

  • –

    Gaia Observing Schedule Tool (GOST).

As the ‘hub’ in the Gaia Science Ground Segment, DPCE is the interface between MOC and other DPAC DPCs (Section 1.2.4). This includes data retrieval from MOC and distribution to DPCs after IDT/FL processing and distribution of the MDB, i.e., receiving the processed data from the other DPCs and assembly of the subsequent version after DPAC processing. DPCE also has responsibility for interactions with MOC with respect to payload calibration and operations.

DPCE performs both daily and cyclical processing.

DPCT operates the AVU pipelines in the Cycle-01 processing, including the daily pipelines of the AVU/AIM and AVU/BAM systems and the data-reduction-cycle pipeline of the GSR system. DPCT processed 800 daily runs of AIM and BAM and completed the first GSR data-reduction processing. The number of work flows executed is about $\mathrm{100\hspace{0.17em}000}$, and the number of jobs processed is about $10.5$ million. The size of the received input data is about 70 TB while the largest Oracle database at DPCT has a size of 150 TB.

The daily pipelines of the AVU/AIM and AVU/BAM systems are stable after the relevant upgrade implemented at the end of the commissioning phase, but they continue to evolve in order to improve their results and add new modules needed to enrich the analysis. The AVU/AIM and AVU/BAM software systems have run with versions 16.0, 17.0, and 18.0 (including a number of patch releases to fix specific issues found during operations). The AVU/AIM pipeline is running with the following modules: Ingestion, Raw Data Processing, Monitoring, Daily Calibration, Report and Monthly Diagnostics. The AVU/AIM processing strategy is based on time, with each AVU/AIM run being defined on 24 hours of observed data. The AIM pipeline starts with the aim to select AstroObservation having gclass $\le$ 2. The Raw Data Processing processes AstroObservations with gclass equal to 0, 1, or 2 and estimates the image parameters. In processing Cycle 01, the AVU/AIM system processed with a PSF/LSF bootstrapping library including specific image profile templates for each CCD, spectral-type bin, and gclass. The AVU/AIM system cannot process defined runs when IDT runs in RAW-only mode. The Monitoring module is a collection of software modules dedicated to extract information on the instrument health, astrometric-instrument calibration parameters, image quality during in-flight operations, and comparison among AVU/AIM and IDT outputs. The Daily Calibration module is devoted to the Gaia signal-profile reconstruction on a daily basis. Its work flow also includes diagnostics and validation functions. The calibration-related diagnostics include the image-moment variations over the focal plane. An automatic tool performs validation of the reconstructed image profiles before using them within the AIM chain. The computing performance depends strongly on the number of AstroObservations characterising the AIM run. The AIM pipeline can manage runs with different sizes. The observed range in Cycle-01 processing is between 2 and 11 million AstroObservations. A filter is activated in runs with more than 5 million AstroObservations in order to process the minimum number of data in each bin defined on several instrument and observation parameters and time intervals without losing quality in the AVU/AIM result. The filter is usually activated when Gaia is scanning the galactic plane.

The AVU/BAM pipeline is running with the following modules: Ingestion, Pre-Processing, RDP, Monitoring, Weekly Analysis, Calibration, Extraction and Report. In the Raw Data Processing (RDP) module, the following algorithms are running: Raw Data Processing, Gaiometro, Gaiometro2D, DFT, Chi Square, BAMBin, and comparison with IDT BamElementary. The AVU/BAM system has two run strategies: IDT and H24. In the IDT strategy, used from commissioning to December 2015 (covering Gaia DR1), a BAM run is defined when a transfer containing the BAM data is received at DPCT. The processing is started automatically without any check on data. In the other strategy, the H24 strategy, a BAM run is defined based on 24 hours of data and the processing starts automatically when the data availability reaches a threshold defined by the BAM payload expert (e.g., 98%–99%). The AVU/BAM system has been processing with the H24 strategy since December 2015 to have BAM analyses at regular intervals. The AVU/BAM pipeline has been sending the BAM output to the end of each run since the commissioning phase. After the last operational BAM software release, BAM 19.0.0, the pipeline takes about 60 minutes to execute all modules.

In order to ensure that the automatic data reception and ingestion processes are executed without data losses, DPCT has implemented and executed a set of procedures to guarantee the consistency of the data inside the DPCT database. Data-consistency checks are executed on all DPCT data stores and at different times, e.g., before and after data are used in the data-reduction pipelines. The DPCT data-consistency checks are working as expected, i.e., the data-management pipelines are reliable.

The DPCT data stores are in good shape and they are used to provide data services to all AVU data processing and analysis activities. In particular the DPCT database repository, implemented with Oracle technology, is collecting all data received and generated from/to the DPCT pipeline. This configuration allows having data online to conduct additional analyses not implemented in the delivered pipelines. At the end of processing Cycle 01, the size of this database repository is about 150 TB.

Moving to the data-reduction-cycle pipelines, the GSR system started operations in processing Cycle 01 running software version 18.1.0 (including two patch releases to fix specific issues found during operations). The first GSR ran on data of data Segment 00 and data Segment 01 with a Tycho-2 star selection of the Gaia catalogue table and the usage of the IDT and IDU attitude data set. The GSR pipeline was executed with the following modules: Ingestion, System Coefficient Generation, Solver, Solution Analysis, De-Rotation and Comparison, Extraction and Report. The ingestion step is the most complex data-handling module as billions of AstroElementaries are read and matched to sources to populate the GSR data store. The System Coefficient Generation module is devoted to calculate the coefficient parameter of the system of linearised equations to be solved to produce the GSR sphere solution. The Solver module consists of the implementation of the LSQR algorithm for solving the system of linearised equations and it is the only module running on the Fermi supercomputer at CINECA. The Solver finds a GSR solution and the analysis module checks the exit status of the solution algorithm and provides an alert in case of problems revealed by the stopping conditions implemented in the LSQR algorithm. The next module in the pipeline is the De-Rotation and Comparison module which converts the GSR solution into a format compatible with that of the AGIS one. It also de-rotates the AGIS solution back into its internal reference frame. The GSR results are collected in the final report.

The first GSR solution was for astrometric position and attitude parameters only. The instrument was added as correction to the known terms and the attitude-knot interval was 180 s. The operational run has been completed and ended as expected with the successful completion of all the pipeline modules. The results are not scientifically viable because of the pending improvement of the GSR software for what regards the implementation of the attitude constraints and the implementation of missing parts of the instrument model. For this reason, the output of GSR-01 was not sent to the MDB to be included in the MDB-01 release. From a technical point of view, the full GSR pipeline, including the Solver module running on the CINECA supercomputer, has been executed.

The following table shows the main data types produced at DPCT during the data processing prior to Gaia DR1:

• •

  BamElementaryT;

• •

  Bav;

• •

  CalibratedBav.

As explained above, the GSR output is not provided to the MDB. The output and findings of AIM, as well those of BAM, provided in the daily and periodic reports have been used by the payload-expert group to check the instrument health by performing cross checks with other DPAC systems providing the same instrument measurements.