Objectives and Focus Areas

The identification of all necessary data, its export from the USA to Germany, and its provision and backup on servers at the University of Stuttgart was and remains an essential prerequisite for the establishment and operation of a SOFIA Data Center. This process has now been completed, and several types of data and software can be distinguished:

SI raw data: The SOFIA raw data comprises a total of approximately 141 TB, of which the scientific instruments account for around 24.2 TB. The raw data also includes the housekeeping data from the telescope and the MCCS (Mission Control and Communication System) as well as the images from all telescope cameras.

Metadata: This refers to the DCS (Data Cycle System), which contains important information about the observation projects. This includes the observation proposals (AORs), the order of the observations, their distribution across different flights, information on technical feasibility, flight routes, etc.

Processed scientific data: This data was processed from the raw data using an earlier version of the data reduction pipeline (REDUX) or an optimized REDUX version (from 2017) and serves as a reference for comparison with further updates to the pipeline developed as part of the SDC project.

For the sake of completeness, it should be mentioned at this point that the scientific data is already available today via NASA's multi-mission archive (IRSA – InfraRed Science Archive), but is no longer being maintained or improved.

Data reduction pipeline: The instrument-specific data reduction pipeline (REDUX) has also been backed up and is now operational at the SDC. No specific optimizations have been made to the pipeline for the US instruments (EXES, FORCAST, HAWC+) to date. Improvements to the pipeline for the FIFI-LS instrument are currently (summer 2025) in full swing, as the SDC has the complete expertise for this instrument. The pipeline for the GREAT instrument is being worked on at the University of Cologne.

The SOFIA telescope had a total of three cameras in the visible light wavelength range, which were in constant use during all scientific flights and captured around 25,000 images per camera during a flight. These were all stored, but were not previously available to researchers. The cameras are:

1. Wide Field Imager (WFI) with a focal length of 136 mm (field of view 360 x 360 arcmin)
2. Fine Field Imager (FFI) with a focal length of 733 mm (field of view 67 x 67 arcmin)
3. Focal Plane Imager plus (FPI+) in the focal plane of the telescope; uses the telescope optics with a focal length of 5,240 mm (field of view 9.4 x 9.4 arcmin)

The two CCD cameras, WFI and FFI, were mounted on the front ring of the telescope and were mainly used by the telescope operator for orientation in the sky and to help identify star patterns. The FPI+ was the telescope's tracking camera and was also used as a scientific instrument for observing stellar occultations.

In order to make the images recorded by the three telescope cameras usable, they must first be converted from ARK format (Level 0 images) to the FITS format (Flexible Image Transport System), which is widely used in astronomy. In the process, the FITS files are enriched in their header with a variety of relevant data extracted from various housekeeping files and stored under corresponding FITS keywords in the header. This also includes the WCS (World Coordinate System) data, which provides information about the sky coordinates of the image (Level 1 Images).

The celestial coordinates correspond to the data commanded to the telescope, which are derived from the position of the observed object. However, the actual celestial coordinates, i.e., the telescope pointing, can be determined even more accurately using plate solving methods. Programs such as Astrometry.net or SCAMP are used for this purpose, which can determine the exact position and orientation of an image based on a star catalog. These are then stored in the WCS header of the FITS image and saved for later use (Level 2 Images). This can lead to a further improvement in the subsequent data reduction.

During its operational phase, the SOFIA observatory had a large number of different scientific instruments (SIs), each with a dedicated functionality in the fields of spectroscopy, photometry, or polarimetry. Some of the instruments could be operated in different operating modes or wavelength ranges. The instruments and their key characteristics are described in a separate section of the SDC website.

The SI data (raw data) recorded during the observation campaign are not directly suitable for data analysis and visualization of astronomical phenomena or even results. Rather, extensive and complex reprocessing of the raw data (data reduction) is necessary to remove all instrumental and atmospheric disturbances and to calibrate the data in order to obtain physically meaningful and scientifically usable images or spectra of the observed astronomical object. Typical disturbance variables that must be corrected by data reduction include, for example

• Intrinsic emission from the atmosphere and the telescope
• Absorption by the Earth's atmosphere
• Detector effects

For the SOFIA data, data reduction for all instruments (except GREAT, see below) took place under a uniform framework called “Redux” or “Redux Pipeline.” The aim of the SDC project is to modify the Redux pipeline in line with the latest findings and using improved algorithms, thereby eliminating instrumental peculiarities, artifacts, and calibration deficits as far as possible. In addition, the pointing and water vapor data recalculated in the SDC project will also be included in the data reduction. Faulty or incomplete data sets will also be identified and flagged if they cannot be repaired. This will allow them to be excluded from analysis and evaluation.

The data from all nine SOFIA observation cycles (OC #1 to OC #9) with all instruments will be reprocessed in this way. This will enable researchers from different astrophysical disciplines to reliably access, use, and evaluate SOFIA data even without in-depth specific knowledge of the instruments and their data reduction. This will also lead to further dissemination of the data obtained with SOFIA.

With regard to the prioritization of instruments, the German Space Agency, in cooperation with the GSSWG, has decided on the following order, which will be applied in particular in the event of capacity bottlenecks during processing: Priority will be given to instruments financed by Germany (GREAT, FIFI-LS, FPI+) ahead of US instruments in the order HAWC+, FORCAST, and EXES. Data from the HIPO and FLITECAM instruments, which were only in use for a very short time at the beginning of SOFIA's operation, will not be reprocessed. Adjustments to this prioritization can be discussed within the SDC User Group and proposed to the space agency.

The GREAT (https://upgreat.uni-koeln.de) instrument also occupies a special position. As was already the case during the active phase of the SOFIA project, the data is reprocessed by the University of Cologne. However, there are plans to transfer the GREAT data to the SDC database, thus making the data from all instruments available under a uniform interface.

The SDC's work is therefore currently (11/2025) focused on the topics of water vapor, pointing, and FIFI-LS.

The further development of the Redux pipeline is carried out on the University of Stuttgart's GitHub Enterprise Server, and the code is made available to users. The server is publicly accessible at:

https://github.com/SOFIA-Data-Center/sofia_redux

and was derived from:

https://github.com/SOFIA-USRA/sofia_redux

A schematic representation of the reprocessing workflow is summarized in this graph.