skip to main content

gaia data release 3 documentation

13.2 Data Consolidation

13.2.5 ECSV generation

As in previous releases, the Gaia DR3 data is also provided for bulk download in case end users may want to process the data locally. For maximum compatibility, these files are plain text files in CSV format, compressed with the estandard GZIP algorithm. These files follow the partitioning mechanism described in the previous section.

New in this release is the use of the Enhanced Character Separated Value format (ECSV), adopted by astropy. These are regular CSV files, with an extra header with metadata about the table itself and its columns in YAML format, preceded with the hashtag comment character. For additional details on this format, please visit
https://github.com/astropy/astropy-APEs/blob/main/APE6.rst

Some of the tables in Gaia DR3 include multidimensional data, e.g. vectors and matrices. As defined by ECSV, these values are encoded into Strings in JSON format, and added to the CSV surrounded by double quotes, e.g. a CSV parser would read them as valid Strings, whereas an ECSV parser would be able to successfully decode the vectors or matrices from that String using the metadata in the header.