2.3 Calibration models

Author(s): Claus Fabricius

Below, we present the more important models applied for the calibration of the SM and AF CCDs. Some are operated on longer timescales and others on shorter scales, but they are all operated on shorter timescales than anticipated before launch. For an overview on the detector performance during the first couple of years in space see Crowley et al. (2016).

The CCD cosmetics (see Section 2.3.4) deal with questions related to individual CCD columns, like saturation level and abnormal response.

The CCD bias and bias non-uniformity (see Section 2.3.5) exposes the difficulties involved in reading CCDs in window mode, where most potential samples are merely flushed. As a consequence, the precise timing for reading a particular column within the short (less than one ms) time available for reading a line of pixels, varies from time to time that column is read, and so does the bias. The model must therefore take the exact readout details into account.

The astrophysical background model (see Section 2.3.3) includes in fact several elements. It must of course describe the complexities of the two sky areas that overlap in the focal plane, but the dominating background source is the stray light which varies strongly with the spin phase of the spacecraft with respect to the Sun, and produces a complex, intermittent pattern on the focal plane. In addition, the background model must also incorporate the charge release trails following the regular charge injections on each CCD (except SM).

The PSF/LSF model (see Section 2.3.2) must encompass many complex effects, but for the first data release, several have been waived. The optical PSF depends on colour, the field of view, and the position in the focal plane, but it also changes with time. On top comes effects induced by the scanning law, by the way the CCDs are operated, and by complex inefficiencies of the charge transfer within the CCDs. A final complexity is that the chromatic image shifts are included in the PSF model as shifts of the PSF origin.

Author(s): Michael Davidson, Lennart Lindegren

Two of the key Gaia calibrations are the Line Spread Functions (LSF) and Point Spread Functions (PSF). These are the profiles used to determine the image parameters for each window in a maximum-likelihood estimation (see Section 2.4.8), specifically the along-scan image location (observation time), source flux, plus across-scan location in the case of 2D windows. Of these the observation time is of greatest importance in the astrometric solution, and this is reflected in the higher requirements on the AL locations when compared to the AC locations (Lindegren et al. 2012). Indeed, to increase the signal-to-noise the majority of windows are binned in the across-scan direction and are observed as 1D profiles. A Line Spread Function is thus more useful than a 2D Point Spread Function in these cases. The PSF is often understood as the response of the optical system to a point impulse, however in practise for Gaia it is more useful to include also effects such as the finite pixel size, TDI smearing and charge diffusion. This leads to the concept of the effective PSF as introduced in Anderson and King (2000). Pixelisation and other effects are thereby included directly within the PSF profiles. Calibration of the LSF/PSF is among the most challenging tasks in the overall Gaia data processing, due to the dependence on other calibrations, such as the background and CCD health, and due to uncertainties in crucially measured inputs like colour. This calibration will also become more difficult as radiation damage to the detectors increases through the mission, causing a non-linear distortion. Discussion of these CTI effects can be found in Fabricius et al. (2016). Here we will focus on the LSF/PSF of the astrometric instrument, which in the pre-processing step is assumed to be linear, allowing a more straight-forward modelling and application. The LSF/PSF varies over the relatively wide field of view of each telescope (1.7${}^{\circ }$ by 0.7${}^{\circ }$) and with the spectral energy distribution of an observed source. As previously discussed, the observation time of a source depends on the gate used and, since the LSF/PSF profile can vary along even a single CCD, all gate configurations must be calibrated independently. This can be difficult for the shortest gates due to the relatively low number of observations available. An LSF/PSF library contains a calibration for each combination of telescope, CCD and gate.

Several aspects must be considered when defining a model to represent the LSF/PSF. Firstly, the LSF profiles must be continuous in value and derivative, and they must be non-negative. By definition the full integral in the AL direction is 1, thus neglecting the flux lost above and below the binned AC window. This AC flux loss is calibrated as part of the photometric system. The LSF $L$ is normalised as

where $u_0$ is the LSF origin. The origin should be chosen to be achromatic (the centroid of a symmetrical LSF is aligned with the origin but this is not true in general), and since image locations are measured relative to it, it should be tied to a physically well-defined celestial direction. However, it is not possible to separate geometric calibration from chromaticity effects within the daily pipeline; this requires the global astrometric solution from the cyclic processing. The origin is therefore fixed as $x_0=0$ and consequently there will be a colour-dependent bias in this internal LSF calibration. The LSF profile is used to model the expected de-biased photo-electron flux $N(k)$ of a single stellar source, including noise, by

where $\beta$, $\alpha$ and $\kappa$ are the background level, the flux of the source, and the along-scan image location. The index $k$ is the along-scan location of the CCD sample under consideration. The actual photo-electron counts will include a random noise component, in addition.

For the practical application, the LSF can be modelled as a linear combination of basis components

where $N$ basis functions are used. The value $B_m$ of each basis function $m$ at coordinate $u$ is scaled by a weight $w_m$ appropriate for the given observation. A set of basis functions can be derived through Principal Component Analysis (PCA) of a collection of LSF profiles chosen to represent the actual spread of observations, i.e. covering all devices and a wide range of source colours and smearing rates. An advantage of PCA is that the basis functions are ranked by significance, allowing selection of the minimum number of components required to reach a particular level of residuals. These basis functions can in turn be chosen in a variety of ways; we have used a S-spline model (see Section 2.3.2) with a smooth transition to Lorentzian profiles at the LSF wings. Further optimisation can be achieved to assure the correct normalisation by transforming these bases, although this is beyond the scope of the current document. A set of 51 basis functions were determined from pre-flight simulation data, each represented using 75 coefficients The 20 most significant functions have been found to adequately represent real LSFs, although further improvements are possible.

With a given set of basis functions $B_m$ the task of LSF calibration becomes the determination of the basis weights $w_m$. These weights depend on the observation parameters including AC position within the CCD, effective wavelength, AC smearing and others. In general, the observation parameters can be written as a vector $𝒑$ and the weights thus as $w_m(𝒑)$. To allow smooth interpolation, each basis weight can be represented as a spline surface where each dimension corresponds to an observation parameter. In the actually implemented calibration system, each dimension can be configured separately with sufficient flexibility to accommodate the actual structure in the weight surface, i.e. via choice of the spline order and knots. In practise, the number of observation parameters has been restricted to two: AC position and effective wavelength (i.e. source colour) for AL LSF, and AC smearing and effective wavelength for the AC LSF. The coefficients of the weight surface are formed from the outer product of two splines with $k$ and $l$ coefficients respectively. There are therefore $k\times l$ weight parameters per basis function which must be fitted.

The rectangular telescope apertures in Gaia led to a simple model to approximate the PSF in the daily pipeline. The PSF is formed by the cross product of the AL and AC LSFs. This model has a relatively small number of parameters to fit at the price of being unable to represent all the structure in the PSF. A more sophisticated full 2D model shall be available for the cyclic processing systems (described elsewhere), where there are fewer processing constraints than in the daily pipeline. We have confirmed that the AC$\times$AL approximation does not introduce significant bias into the measured observation times for 2D windows. Experiments to compare the fitted observation times using the AC$\times$AL versus a full 2D PSF indicate a systematic bias of $2.3\times {10}^{-4}$ pixels. An example of the PSF and its reconstruction via the AC$\times$AL model is presented in Figure 2.3 and Figure 2.4.

LSF calibrations are obtained by selecting calibrator observations. These are chosen to be healthy (e.g. nominal gate, regular window shape) and not affected by charge injections or rapid charge release. Image parameter and colour estimation for the observation must be successful; good bias and background information must be available. With these data Equation 2.12) can be used to provide an LSF measure per unmasked sample. In the case of 2D windows, the observation is binned in the AL or AC direction as appropriate to give LSF calibrations. The quantity of data available varies with calibration unit. Note that a ‘calibration unit’ in the Gaia pipeline is a given combination of window class (e.g. 1D, 2D), gate, CCD, telescope, possibly time interval (days, weeks, months), and possibly other parameters, e.g. AC coordinate interval on a CCD. For the most common configurations (faint un-gated windows) there are many more eligible observations than can be handled, thus a thinned-out selection of calibrators is used.

A least-squares method is used with the LSF calibrations to fit the basis weight coefficients. The Householder least-squares technique is very useful here as it allows calibrators to be processed in separate time batches and for their solutions to be merged. The merger can also be weighted to enable a running solution to track changes in the LSFs over time (see van Leeuwen 2007). Various automated and manual validations are performed on an updated LSF library before approval is given for it to be used by the daily pipeline for subsequent image parameter determination. There are checks on each solution to ensure that the goodness-of-fit is within the expected range, that the number of degrees-of-freedom is sufficiently positive, and that the reconstructed LSFs are well-behaved over the necessary range of $u$. Individual solutions may be rejected and the corresponding existing ‘best-available’ solution be carried forward. In this way an operational LSF library always has a full complement of solutions for all devices and nominal configurations.

The LSF solutions are updated daily within the real-time system, although they are not approved for use at that frequency. Indeed, a single library generated during commissioning has been used throughout the period covered by this first Gaia data release, in order to provide stability in the system during the early mission. This library has a limited set of dependencies including field-of-view and CCD, but it does not include important parameters such as colour, AC smearing or AC position within a CCD. As such it is essentially a library of mean LSFs and AC$\times$AL PSFs. This will change for future Gaia data releases.

Author(s): Nigel Hambly

The astrophysical background incident on the focal plane of Gaia is dominated by stray light (Mignard 2014), in part because the compact and folded design (Safa et al. 2004) of the various optical instruments leaves very little opportunity for stray light path baffling. The lower rows (1–4) of the focal plane have a diffuse background signal that is dominated by sunlight scattered around the DSA while in the higher rows (5, 6 and 7) it is the diffuse optical background from the Milky Way that dominates. Hence the background consists of a high amplitude, rapidly changing photoelectric component that repeats on the satellite spin period. Furthermore, this component evolves slowly in both amplitude with the L2 elliptical orbital solar distance and in phase as the scanning attitude changes with respect to the Ecliptic and Galactic planes. Superposed on this are transient spikes in background due to very bright stars and bright Solar system objects transiting across or near the focal plane. Moreover, there are two non-photoelectric components to the background signal. These are a charge release signal associated with artificial charge injections which are used for on-board radiation damage mitigation (Prod’homme et al. 2011) and the dark current signal. The photoelectric background signal varies routinely over three orders of magnitude depending on instrument and spin phase with values as low as $\sim 0.1$ and up to $\sim 100$ electrons per pixel per second. Regarding the non-photoelectric signals, the charge release signal currently varies between 1 and 10 electrons per pixel per second in the first TDI line immediately after the last injection line but rapidly diminishing to 1% of this level after the following $\sim 20$ TDI lines, while the dark signal is all but negligible (see Section 2.3.4).

The parametric model for the astrophysical background consists of the outer product of two one-dimensional spline functions (van Leeuwen 2007) which defines a flexible two-dimensional surface model known colloquially as a ‘bispline’. In the along-scan direction the spline function is quadratic with knots evenly spaced at intervals that adapt to the available data density. In regions of high density that generally exhibit higher background fluctuations on smaller spatial scales, and in which there is a higher density of data to constrain the model, knots are more closely spaced. Typically the along-scan knot interval is in the range 1 to 10 arcminutes. In the across-scan direction the spline function is linear, allowing for sudden discontinuities in the gradient of the stray light pattern. Knot positions are placed at fixed, unevenly spaced positions to best follow rapid changes in the stray light pattern in the across-scan direction. The charge release model consists of a simple empirical look-up table (LUT) as a function of TDI line index following the last charge injection line with a power-law scaling as a function of that injection level present at the same column position on the CCD. This enables a single charge release LUT calibration per device when applied in conjunction with the across-scan profile of the charge injection for the same device which is also characterised via a LUT.

Author(s): Michael Davidson

The focal plane of Gaia contains 106 CCDs each with 4494 lines and 1966 light-sensitive columns, leading to it being called the ‘billion pixel camera’. The pre-processing requires calibrations for the majority of these CCDs, including SM, AF and BP/RP, in order to model each window during image parameter determination. Where effects cannot be adequately modelled, the affected CCD samples can be masked and the observations flagged accordingly. The CCDs are affected by the kind of issues familiar from other instruments such as dark current, pixel non-uniformity, non-linearity and saturation (see Janesick 2001). However, due to the operating principles used by Gaia such as TDI, gating and source windowing, the standard calibration techniques need sometimes to be adapted. The use of gating generally demands multiple calibrations of an effect for each CCD. In essence each of the gate configurations must be calibrated as a separate instrument.

An extensive characterisation of the CCDs was performed on ground, and these calibrations have been used in the initial processing. The effects must be monitored and the calibrations redetermined on an on-going basis to identify changes, for instance the appearance of new defects such as hot columns. To minimise disruption of normal spacecraft operations, most of the calibrations must be determined from routine science observations. Only a few calibrations demand a special mode of operation, such as ‘offset non-uniformities’ and serial-CTI measurement (see Section 2.3.5 and Fabricius et al. (2016)). There are two main data streams used in this calibration: 2D science windows and Virtual Objects (VOs). The 2D science windows typically contain bright stars, although a small fraction of faint stars which would otherwise be assigned a 1D window are acquired as 2D (known as Calibration Faint Stars). VOs are ‘empty’ windows which are interleaved with the detected objects, when on-board resources permit. By design the VOs are placed according to a fixed repeating pattern which covers all light-sensitive columns every two hours, ensuring a steady stream of information on the CCD health. The VOs allow monitoring of the faint end of the CCD response while the 2D science windows allow us to probe the bright end.

The dark signal (or dark current) is the charge produced by each column of a CCD when it is in complete darkness. While such condition was achieved during the on-ground testing it is not possible to replicate in flight as there are no shutters on Gaia. The observed VO and science windows must therefore be used to determine the dark signal for each gate setting, although these also contain background, source and contamination signal, bias non-uniformity and CTI effects. A sliding window of 50 revolutions is used to select eligible input observations, for instance those not containing multiple gates or charge injections. The electronic bias (including non-uniformity) is subtracted from each window and a source mask is created via an N-sigma clipping of the de-biased samples. The leading samples in the window are also masked to mitigate CTI effects. A least-squares method is then used to estimate a local background for the window (assumed to be uniform), and this in turn can be subtracted to provide a measure of the dark signal in each CCD column covered by the window. In this manner measures can be accumulated for each column over the 50 revolution interval, and then a median taken to provide a robust dark signal value.

In an ideal device there would be a linear response between the accumulated charge and the output of the Analogue-to-Digital Converters (ADCs) at all signal levels. In reality the response typically becomes non-linear at high input signals for a variety of reasons (see Janesick 2001). Although the linearity has been measured before launch, a calibration has not yet been implemented in the daily pipeline due to the uncertainty in determination of the input signals, which require detailed knowledge of a range of coupled CCD effects. In the meantime a conservative linearity threshold has been used to allow masking of samples which may be within the non-linear regime. A related topic is the pixel non-uniformity which represents the variation in sensitivity across the CCD. In Gaia we observe only the integrated sensitivity of the pixels within a particular gate so this is known as the Column Response Non-Uniformity. Similarly no in-flight calibration has yet been performed apart from the extreme case to identify ‘dead’ columns. These are columns which appear to have zero sensitivity to illumination and can be found using bright-star windows. The accumulated samples for a dead column have a distribution which is consistent with the expected dark signal plus read out noise. The CCDs used on Gaia have been selected for their excellent cosmetic quality.

At the highest signal levels various saturation effects occur on the device and within the ADC. There can be very large differences in the effective saturation level across a single device, or even between neighbouring columns, for example due to variation in the Full Well Capacity. For reasons beyond the scope of this paper the saturation level can oscillate or jump depending on the read-out sequencing. An algorithm has been developed to measure the lowest observed saturation level for each gate and column to allow conservative masking of samples. A Mexican hat filter is applied to the accumulation of samples from bright-star windows to identify over-densities of data at particular signal levels, using analytical significance thresholds. The lowest significant peak is then taken as the saturation level. If no peak is found then the maximum observed sample for that column is used.

The calibrations discussed above are computed daily in the framework of the First-Look system (see Section 2.5.2) and, if they are judged to be satisfactory, the corresponding software libraries are subsequently used in the pipeline.

Author(s): Nigel Hambly

As is usual in imaging systems that employ charge-coupled devices (CCDs) and analogue-to-digital converters (ADCs) the input to the initial amplification stage of the latter is offset by a small constant voltage to prevent thermal noise at low signal levels from causing wrap-around across zero digitised units. The Gaia CCDs and associated electronic controllers and amplifiers are described in detail in Kohley et al. (2012). The readout registers of each Gaia CCD incorporate 14 prescan pixels (i.e. those having no corresponding columns of pixels in the main light-sensitive array). These enable monitoring of the prescan levels, and the video chain noise fluctuations for zero photoelectric signal, at a configurable frequency and for configurable across-scan (AC) hardware sampling. In practise, the acquisition of prescan data is limited to the standard un-binned (1 pixel AC) and fully binned (2, 10 or 12 pixel AC depending on instrument and mode) and to a burst of 1024 one millisecond samples each once every 70 minutes in order that the volume of prescan data handled on board and telemetred to the ground does not impact significantly on the science data telemetry budget.