Request for Comments: 4047 UCO/Lick Observatory
Category: Informational D. Wells
National Radio Astronomy Observatory
April 2005
MIME Sub-type Registrations for
Flexible Image Transport System (FITS)
Status of This Memo
This memo provides information for the Internet community. It does
not specify an Internet standard of any kind. Distribution of this
memo is unlimited.
Copyright Notice
Copyright (C) The Internet Society (2005).
Abstract
This document describes the registration of the Multipurpose Internet
Mail Extensions (MIME) sub-types to be used by the international
astronomical community for the interchange of Flexible Image
Transport System (FITS) files. The encoding is defined by the
published FITS standard documents. The FITS format has been in use
since 1979, and almost all data from astronomical observations are
interchanged by using FITS.
Table of Contents
1. Introduction.................................................. 2
2. Conventions Used in this Document............................. 2
3. Overview...................................................... 2
4. FITS Definition............................................... 3
4.1. FITS Structure.......................................... 3
4.2. History of FITS Features................................ 5
4.3. Stability of the FITS definition........................ 6
4.4. Portability of FITS files............................... 7
4.5. Application Programming Interfaces to FITS.............. 7
4.6. FITS File Conformance Testing........................... 8
4.7. Archives That Distribute FITS Files..................... 8
5. IANA Considerations........................................... 9
5.1. Registration of application/fits........................ 10
5.2. Registration of image/fits.............................. 14
6. References.................................................... 19
6.1. Normative References.................................... 19
6.2. Informative References.................................. 20
7. Security Considerations....................................... 21
8. Contributors.................................................. 21
9. Acknowledgements.............................................. 22
Authors’ Addresses................................................ 22
Full Copyright Statement.......................................... 23
1. Introduction
The FITS file format [FITS] was designed in order to facilitate the
interchange of astronomical image data between observatories. FITS
provides a means of transporting arrays and tables of data and
keyword/value pairs of metadata. FITS is defined by standards
documents that are approved by the International Astronomical Union
(IAU, http://www.iau.org/) and published in refereed journals.
Before the inception of HTTP, astronomers used the Internet to
exchange FITS files. Multiple unofficial media types for FITS files
[ASU] came into use shortly after the inception of the WWW and have
remained in use. Currently (2005) the international astronomical
community is pursuing many cooperative efforts (e.g., [IVOA], [NVO],
[AstroGrid], [AVO]) to produce web services that provide astronomical
data. The exchange of FITS files is a fundamental element of the
prototypes for these web services [SIAP]. The astronomical community
has to agree to use one set of media types for FITS files in order to
promote interoperability of its various services.
In its simplest form, FITS is used as a means of transporting
astronomical image data in a raster form along with coordinate
information and other standard and locally defined metadata. In such
applications FITS is much like the well-known TIFF format [TIFF] with
the addition of the GeoTIFF tags [GeoTIFF]. However, FITS is capable
of describing a much broader range of data than 2-dimensional
rasters. A consensus has developed in the FITS community that two
media types are needed: one for images and one for all other cases.
2. Conventions Used in this Document
The keywords "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in RFC-2119 [Require].
3. Overview
This document describes the registration of the MIME media sub-types
"application/fits" and "image/fits".
In 1988 the International Astronomical Union formed the FITS Working
Group (IAUFWG) to oversee matters pertaining to the evolution of the
FITS data format. The IAUFWG has approved the submission of this
document and the registration of these two MIME types.
4. FITS Definition
FITS is defined by a document approved by the International
Astronomical Union (IAU) and published in the journal Astronomy &
Astrophysics [NOST]. Conventions for additional keywords used in
FITS files are proposed by interested parties and negotiated and
reviewed by ad hoc committees of the FITS community. If such usage
of additional keywords is approved by national committees and the
IAUFWG then they become new reserved keywords in the FITS standard
and are published in an ongoing series of papers (e.g., [WCS1,
WCS2]).
Copies of the standard documents can be found at the following sites:
http://fits.gsfc.nasa.gov/
http://archive.stsci.edu/fits/
http://www.cv.nrao.edu/fits/
http://heasarc.gsfc.nasa.gov/docs/heasarc/fits.html
Although a brief structure and feature description is provided in
this section as background information, the reader is directed to the
FITS standards documents to obtain complete feature and technical
details.
4.1. FITS Structure
A FITS file consists of a sequence of one or more header and data
units (HDUs) optionally followed by special records. The structure
of a FITS file is based on blocks with a length of 2880 8-bit bytes
(23040 bits). This size was chosen because it is evenly divisible by
the byte and word lengths of all known computer systems. All FITS
files have lengths that are integral multiples of this block size.
Each FITS header consists of a sequence of one or more 2880-byte
blocks that hold 36 80-character records (36*80=2880). The records
consist of ASCII keyword/value pairs plus optional comments. The
character set is the 7-bit printing ASCII codes, including the ASCII
space. In particular, the control codes CR, LF, FF, TAB and NUL are
not used in FITS headers. The keywords are up to 8 characters in
length. Some keywords are mandatory, and their meaning is rigidly
prescribed. Among these are keywords that describe the structure and
size of the subsequent data array. The standard reserves other
keywords for the purpose of conveying specifically defined items of
metadata. Keywords that are neither mandatory nor reserved may be
inserted with semantics that are defined by local conventions. (Some
local conventions have later been adopted into standardized
practice.) The end of the header is signified by a block containing
the keyword "END". A simple example of a FITS header for a digital
image, using only keywords that were specified in the initial FITS
Agreement of March 1979 [FITS], is as follows:
1 2 3 4 5 6
1234567890123456789012345678901234567890123456789012345678901234567..
SIMPLE = T / file does conform to FITS standard
BITPIX = 16 / 16-bit twos-complement pixel values
NAXIS = 2 / 2-dimensional image
NAXIS1 = 512 / first axis length
NAXIS2 = 512 / second axis length
COMMENT -----------------------------------------------------
COMMENT FITS (Flexible Image Transport System) format is defined
COMMENT in ’Astronomy and Astrophysics’, volume 376, page 359;
COMMENT bibcode: 2001A&A...376..359H
COMMENT -----------------------------------------------------
ORIGIN = ’Lick Observatory ’ /
DATE = ’2003-11-22T05:23:45’ / when this file was written
END
(These 13 records are followed by 23 80-character records of ASCII
spaces, to pad the header block to the 2880-byte FITS block size.)
The first keyword/value pair, SIMPLE=T, is the signature of FITS; all
FITS files begin with these characters. BITPIX is the second keyword
of all FITS files; legal values for it are 8, 16, 32, -32 and -64.
NAXIS is the third keyword of all FITS files; legal values for it are
0 to 999. NAXIS=0 is legal, and implies that there is no image data
matrix associated with the header (and the NAXISi keywords must not
be present); this value is common for files that will be given the
media type "application/fits" as discussed in section 5.1 of this
document. NAXIS=2 is the value most often used for files that will
be given the media type "image/fits", as discussed in section 5.2 of
this document (NAXIS=3 FITS images are also common). The COMMENT
records will be ignored by FITS reading software, but are used here
to specify the precise journal citation for the FITS standard, an
item of information that is important for archiving on timescales of
decades. The ORIGIN keyword is commonly used in FITS files to encode
the name of the institution where the file was produced. The DATE
keyword is used to convey the timestamp for the file (whereas the
keyword DATE-OBS is used to convey observation start times). FITS
headers commonly contain a vast variety of additional keywords used
to encode metadata. In particular, a digital image header will often
include keywords to specify the precise celestial coordinates of the
pixels of the 2-D matrix, with conventions that became part of the
FITS standard in 2002 (see [WCS2]). Finally, the above example
demonstrates how, in addition to the COMMENT records, 60% of the
bytes in keyword=value records in FITS headers are reserved for
comments. Although the simple header example above is contained in
only one FITS block, multi-block FITS headers are commonly
interchanged.
Following each header is a data unit that consists of a sequence of
zero or more 2880-byte blocks. In accordance with the description in
the keywords of their header, these blocks contain an N-dimensional
data array, optionally followed by other small groups of array data.
In most cases, the data array represents either an N-dimensional
array of image pixel values or a 2-dimensional array of tabular data.
Following the HDUs a FITS file may contain zero or more 2880-byte
blocks of special records. The standard does not specify anything
about their content. (This convention for special records is not
known to have been used for any purpose other than prototyping new
elements of the standard, such as random groups and BINTABLE.)
The initial HDU in a FITS file is known as the primary HDU (PHDU);
any subsequent HDU is known as an extension HDU (XHDU). The keyword
content of the PHDU is distinct (a PHDU requires a slight alteration
before it can become an XHDU, and most types of XHDU cannot become a
PHDU). A PHDU may have a data array consisting of zero elements, and
this will often be the case for FITS files intended to communicate
tables and multiple images. An XHDU may be one of several standard
types, or it may be another conforming type. Standard types of XHDU
include "IMAGE" (containing an N-dimensional data array similar in
most respects to the PHDU), "TABLE" (containing a 2-dimensional table
of ASCII character data), and "BINTABLE" (containing a 2-dimensional
table of binary data whose elements may themselves be multi-
dimensional arrays).
4.2. History of FITS Features
In 1981 the original definition of FITS described a single HDU
containing one multi-dimensional image array [FITS], as illustrated
by the sample header shown in section 4.1. Subsequent agreements
have used the original framework of HDUs and keyword/value pairs to
extend FITS while preserving the validity of all existing files.
FITS now has standard means of describing multiple arrays of image
data and/or multiple tables of numeric and character information.
Brief highlights of the history of FITS
- 1979: Initial FITS Agreement and first interchange of files
- 1980: Random groups convention developed
- 1981: Published original (single HDU) definition [FITS]
- 1981: Published random groups definition [GROUPS]
- 1982: Formally endorsed by the IAU
- 1988: Defined rules for multiple HDUs [XTENSION]
- 1988: FITS Working Group established by IAU [IAUFWG]
- 1988: Extended to include ASCII tables [TABLE]
- 1990: Extended to include IEEE floating-point data
- 1994: Extended to multiple image arrays [IMAGE]
- 1995: Extended to binary tables [BINTABLE]
- 1997: Adopted a Y2K-compliant date format
- 2001: Reiterated existing standard in one paper [NOST]
- 2002: Approved conventions for world coordinates [WCS1, WCS2]
4.3. Stability of the FITS Definition
After the adoption of FITS by the IAU in 1982, some in the emerging
community of FITS users realized that there would be tension between
the need for archival stability and the need for evolution. In order
to satisfy both of these requirements they set up a controlled
parliamentary process. At the time of the Generalized Extensions
Agreement [XTENSION], the meta-agreement that controls the evolution
of FITS, the FITS community adopted the guiding principle "Once FITS,
always FITS". Under this rule, no change may be made to FITS that
invalidates existing files. Changes to the FITS standard occur only
after action by the IAU FITS Working Group (FWG). The FWG acts only
after approval by regional working groups that coordinate FITS
activity in various parts of the world.
FITS has been adopted as the archival format for image data and
interferometric data obtained by many ground-based observatories and
for all data from many spacecraft. Many astrophysical archives store
their data in FITS format, and most astronomical catalog data has
been transcribed into FITS files. The many terabytes of data in
these archives contribute to the stability of the FITS standard.
4.4. Portability of FITS files
Eric Greisen, one of the authors of the original document [FITS],
relates that in 1979
[t]he first FITS files were written by a PL/I program on an IBM
360 under OS/MFT (32-bit, twos-complement numbers and 8-bit EBCDIC
characters) and were read by a Fortran program executing on a CDC
6400 under SCOPE (60-bit, ones-complement numbers and 6-bit
"Display Code" characters). [Remark]
Subsequent evolution of computing hardware and FITS over 25 years has
not degraded this ability to transfer the data content.
The structure of FITS files is extremely general, and this
necessarily complements the nature of astronomical data. FITS is
used to store observations of the entire electromagnetic spectrum
from radio to gamma rays, from ground-based observatories and from
spacecraft. FITS is also used to communicate physical properties
other than radiation intensity; these may be inferred from
observations or calculated by theoretical models. The pedigree of
data in a FITS file typically varies among disciplines; FITS may be
used to store raw and uncalibrated data, completely reduced and
calibrated data, or both. Nevertheless, the FITS standard provides
that the syntactic content of the data and metadata are unambiguously
available to posterity.
Observatories have developed numerous local conventions for the
storage and transfer of data peculiar to their instrumentation and
purview. Application software for handling FITS files from different
regions of the electromagnetic spectrum has been largely disjoint.
For a FITS file that consists of multiple HDUs there are no widely
established conventions governing the meaning of, interrelations
between, and suggested use of the data sets. Recognition of any
local conventions used for FITS data has often been based on
heuristics of the additional (non-standard) keyword/value pairs.
Fully understanding the semantic content of a FITS file usually
requires an external data dictionary.
4.5. Application Programming Interfaces to FITS
Although the definition of FITS is expressed in terms of the bit
content of the files, there are widely supported application
programming interfaces (APIs) which simplify the task of manipulating
FITS files. Interfaces exist for many languages and operating
systems. A partial list of APIs follows:
CFITSIO http://heasarc.gsfc.nasa.gov/fitsio/
fitsTcl http://heasarc.gsfc.nasa.gov/ftools/fv/fitsTcl_home.html
WCSLIB http://www.atnf.csiro.au/people/mcalabre/WCS/
PyFITS http://www.stsci.edu/resources/software_hardware/pyfits
WCSTools http://tdc-www.harvard.edu/software/wcstools/
FUNTOOLS http://hea-www.harvard.edu/RD/funtools/
IDLASTRO http://idlastro.gsfc.nasa.gov/
fitsy http://cfa-www.harvard.edu/~john/fitsy/
IUEDAC http://archive.stsci.edu/iue/iuedacfits.html
Mathematica http://documents.wolfram.com/v5/Built-inFunctions/
GraphicsAndSound/ImportAndExport/
AdditionalInformation/Import.html
MatLab http://www.mathworks.com/access/helpdesk/help/techdoc/
ref/fitsread.shtml?cmdname=fitsread
Current lists of more APIs can be found at
http://fits.gsfc.nasa.gov/fits_libraries.html
List of applications that use FITS are found in the IANA
registrations of the media types.
4.6. FITS File Conformance Testing
FITS files can be tested for conformance to the Definition of FITS
rules [NOST, WCS1, WCS2] with an application named "fitsverify",
which is available at
http://heasarc.gsfc.nasa.gov/docs/software/ftools/fitsverify/
in the form of executable binary files for Solaris, Linux, and
Windows platforms, as well as in source code. Although "fitsverify"
has not been endorsed by the IAUFWG, users should be aware that the
designer of the program was the Secretary of the Technical Panel that
produced the published FITS standard [NOST].
4.7. Archives That Distribute FITS Files
As noted in section 4.3 of this RFC, massive (multi-terabyte) data
archives that contain and/or distribute FITS files contribute to the
stability of the FITS standard. There are numerous publicly
available archives of FITS files derived from both space and ground-
based observations that span the entire range of the electromagnetic
spectrum from radio to gamma-ray wavelengths. The following are
examples of such archives, in no particular order:
Telescope(s) URLs for archive access
------------ ------------------------------------------------------
KPNO,CTIO,.. http://archive.noao.edu/nsa/
VLT,HST,.. http://archive.eso.org/
Subaru http://smoka.nao.ac.jp/
SDSS http://www.sdss.org/dr3/
CFHT http://cadcwww.dao.nrc.ca/cfht/cfht.html
VLA,VLBA,GBT http://e2e.aoc.nrao.edu/archive/archive_describe.html
HST,MAST http://archive.stsci.edu/
HEASARC http://heasarc.gsfc.nasa.gov/w3browse/
Chandra http://cxc.harvard.edu/cda/
LaPalma http://archive.ast.cam.ac.uk/ingarch/
BIMA http://bimaarch.ncsa.uiuc.edu/
Keck-DEIMOS http://archive.deep.ucolick.org/
ComptonGRO http://cossc.gsfc.nasa.gov/archive/index.html
Spitzer,.. http://www.ipac.caltech.edu/
AAT http://www.aao.gov.au/archive/
HIPASS http://www.atnf.csiro.au/research/multibeam/
multibeam.html
JCMT http://salish.dao.nrc.ca:8080/jcmt/intro.html
COBE,WMAP http://lambda.gsfc.nasa.gov/
EVN http://www.jive.nl/archive/scripts/listarch.php
Gemini http://gemini.ast.cam.ac.uk/sciops/data/dataIndex.html
XMM-Newton http://xmm.vilspa.esa.es/external/xmm_data_acc/xsa/
5. IANA Considerations
The general nature of the full FITS standard requires the use of the
media type "application/fits". Nevertheless, the principal intent
for a great many FITS files is to convey a single data array in the
PHDU, and such arrays are very often 2-dimensional images. Several
common image viewing applications already display single-HDU FITS
files, and the prototypes for virtual observatory projects specify
that data provided by web services be conveyed by the data array in
the PHDU. These uses justify the registration of a second media
type, namely "image/fits", for files which use the subset of the
standard described by the original FITS standard paper [FITS].
We note that the media type "image/gif" [MIME2] admits raster images
that are three dimensional, because animated GIF images contain two
spatial dimensions and one temporal dimension. We note that the
media types "image/vnd.dwg" and "image/vnd.dxf" admit data that
include three-dimensional vectors and curves as well as objects
created by using constructive solid geometry. Following these
precedents for the "image" media type, we specify that "image/fits"
MAY be used to describe FITS PHDUs that have other than two
dimensions. We expect that most files described as "image/fits" will
have two-dimensional (NAXIS=2) PHDUs.
5.1. Registration of application/fits
To: ietf-types@iana.org
Subject: Registration of Standard MIME Media type application/fits
MIME media type name: application
MIME subtype name: fits
Required parameters: none
Optional parameters: none
Encoding considerations: binary
FITS files can be quite large. When transferred via HTTP it may be
efficient for the transaction to make use of content-coding or
transfer-coding values such as "gzip", "compress", or "deflate".
Security considerations:
FITS provides a means of transporting arrays and tables of data and
keyword/value pairs of metadata. The standard FITS keywords are
either mandatory or reserved. Mandatory keywords provide information
necessary for correct interpretation of the data; reserved keywords
merely provide standard bits of metadata. As such, the current
standard FITS keywords do not pose security risks.
A FITS file author may insert additional keywords with semantics that
are not described by the standard. Parties exchanging FITS files may
employ locally defined conventions that use various keywords and
their values to induce actions on the part of the recipient. There
are existing local conventions where such keywords are used to
request the reading of other files and/or URIs. There are other
local conventions where such keywords are used to modify the state of
a telescope and/or instrument. The security implications of local
conventions such as these SHOULD be analyzed by the parties employing
them.
Interoperability considerations:
FITS files have been successfully transported between wildly
different computers since 1979. The difficulty most likely to be
encountered by a FITS application is inability to acquire the
computational resources required by a very large FITS file.
Published specification:
The specification for this content type is published as a series of
papers in refereed astronomical journals:
Hanisch, R., et al., "Definition of the Flexible Image Transport
System (FITS)", Astronomy & Astrophysics, 376, p. 359, 2001.
Greisen, E. and M. Calabretta, "Representations of world coordinates
in FITS", Astronomy & Astrophysics, 395, p. 1061, 2002.
Calabretta, M. and E. Greisen, "Representations of celestial
coordinates in FITS", Astronomy & Astrophysics, 395, p. 1077, 2002.
Copies of these specifications can also be found via:
http://fits.gsfc.nasa.gov/