# HDF Overview

Tip: For information on the current HDF version, enter the following at the IDL prompt:

HELP, 'hdf', /DLM

The Hierarchical Data Format (HDF) is a multi-object file format that facilitates the transfer of various types of data between machines and operating systems. HDF is a product of the National Center for Supercomputing Applications (NCSA). HDF is designed to be flexible, portable, self-describing and easily extensible for future enhancements or compatibility with other standard formats. The HDF library contains interfaces for storing and retrieving images and multi-dimensional scientific data.

IDL’s HDF routines all begin with the prefix "HDF_".

Note: For more information on the HDF - EOS tools, please see the "HDF - EOS Tools and Information Center".

## Programming Model

### Creating HDF Files

The following IDL commands should be used to create a new HDF file:

- HDF_OPEN: Call this procedure first to open an HDF file. The CREATE keyword must be set if you want to create a new file instead of opening an existing one.
- HDF_DFAN_ADDFDS: Optionally, add a file description.
- HDF_DFAN_ADDFID: Optionally, add a file annotation.

### Adding Data to an HDF File

The routines used to add data to an HDF file vary based on the interface model being used:

- To add an 8-bit image (with or without a palette), use HDF_DFR8_ADDIMAGE or DFR8_PUTIMAGE.
- To add a palette, use HDF_DFP_ADDPAL or HDF_DFP_PUTPAL.
- To add a 24-bit image, use HDF_DF24_ADDIMAGE or HDF_DF24_PUTIMAGE.
- To add a Multi-File Scientific Dataset, use the following commands:
- HDF_SD_CREATE or HDF_SD_SELECT to create an SDS or select an existing one.
- HDF_SD_DIMSET to set dimension information.
- HDF_SD_ATTRSET to set attribute information.
- HDF_SD_SETINFO to insert optional information about the data.
- HDF_SD_ADDDATA to insert the data.
- HDF_SD_SETEXTFILE to move the data to an external file (optional).
- HDF_SD_ENDACCESS to end access to the SDS.
- To add a Vdata, use the following commands:
- HDF_VD_ATTACH to get a Vdata identifier.
- HDF_VD_SETINFO to write information about the Vdata (optional).
- HDF_VD_FDEFINE to prepare non-trivial fields (optional).
- HDF_VD_WRITE to write the Vdata.
- To add a Vdata to a Vgroup, use the following commands:
- HDF_VG_ATTACH to get a Vgroup identifier.
- HDF_VG_SETINFO to set the Vgroup name and class (optional).
- HDF_VG_INSERT to add the Vdata to a Vgroup.
- HDF_VG_DETACH to close the Vgroup.
- HDF_CLOSE to close the file.

## Code Examples

Two example files that demonstrate the use of the HDF routines can be found in the `examples/doc/sdf` subdirectory of the IDL distribution. The file `hdf_info.pro` prints a summary of basic information about an HDF file. The file `hdf_rdwr.pro` creates a new HDF file and then reads the information back from that file. Run these example procedures by entering `hdf_info` or `hdf_rdwr` at the IDL command prompt or view the files in an IDL Editor window by entering`.EDIT hdf_info.pro` or `.EDIT hdf_rdwr.pro`.

## HDF Interfaces

There are two basic interfaces to HDF files: the single-file application interface and the multiple-file application interface. These interfaces support eight different types (models) of data access. The table below lists the different models and the names of the IDL routines that access those models. Each model is described in more detail after the table.

Model | IDL Routine Name Prefix
--- | ---
24-bit raster | HDF_DF24_
annotation data | HDF_DFAN_
palette data | HDF_DFP_
8-bit raster | HDF_DFR8_
scientific data | HDF_SD_
multi-file scientific data | HDF_SD_
VData | HDF_VD_
VGroup | HDF_VG_

### Single File Application Interfaces

In this mode, access is limited to one file at a time. This interface supports the 8-bit raster, 24-bit raster, palette, scientific data, and annotation models. The interfaces are described in more detail after the table.

- 8-bit Raster Model: The HDF_DFR8_ routines access 8-bit images.
- Palette Model: The HDF_DFP_ routines are used to work with the HDF_DFR8_ routines to manipulate palettes associated with 8-bit HDF images.
- 24-bit Raster Model: The HDF_DFR24_ routines access 24-bit images.
- Scientific Data Models (SDs): Used to manipulate arrays of arbitrary dimension and type. Under this model, an array accompanied by a record of its data type, dimensions and descriptors is called a Scientific Dataset (SD).
- Annotation Model: The annotation model is used to describe the contents of the file through such items as labels, data descriptors, and dimension scales.
- Vdata Model: This interface allows for the creation of customized tables. Each table consists of a series of Vdata records whose values are stored in fixed length fields. As described in more detail in the Vdata example below, a Vdata can contain three kinds of identifying information: a Vdata name, Vdata Class, and multiple Vdata field names. The Vdata model is accessed through the routines that begin with the HDF_VD_ prefix.
- Vgroup Model: A collection of one or more data objects, Vdata sets, or Vgroups is known as a Vgroup. Each Vgroup can be given a Vgroup name and Vgroup class. The Vgroup model is accessed through the routines that begin with the HDF_VG_ prefix.

### Multi-File Application Interface

The HDF_SD_ routines allow operations on more than one file at a time. This multi-file interoperability is achieved through HDF's use of a modified version of the NetCDF library. IDL’s interface to HDF’s multi-file capability is the HDF_SD_SETEXTFILE routine.

## HDF Scientific Dataset ID Numbers

IDL’s HDF_SD_ routines can accept two different types of ID numbers. These ID numbers are referred to in the Help system as the SDinterface_id and SDdataset_id arguments.

The SDinterface_id is the Scientific Dataset interface ID. There is only one SDinterface_id per HDF file. For each actual dataset used, you will also need an SDdataset_id, which is the ID for the particular dataset.

Some routines, such as HDF_SD_ATTRFIND, accept either an SDinterface_id or an SDdataset_id. In these cases, the documentation refers to the ID as an SD_id, meaning that either type of ID is accepted.

### IDL and HDF Data Types

HDF and IDL support many different data types. Many of the HDF routines allow you to perform a data type conversion on the fly by setting keywords such as FLOAT. When the data type desired is not explicitly specified, IDL uses the conversions shown in the following tables. Note that single-precision floating-point is the default data type and that the complex data type is not supported.

When writing IDL data to an HDF file, IDL data types are converted to the HDF data types shown in the following table:

IDL Data Type | HDF Data Type
--- | ---
BYTE | DFNT_UINT8 (IDL bytes are unsigned)
INT | DFNT_INT16
UINT | DFNT_UINT16
LONG | DFNT_INT32
ULONG | DFNT_UINT32
FLOAT | DFNT_FLOAT32
DOUBLE | DFNT_DOUBLE
STRING | DFNT_CHAR8

When reading data from an HDF file, HDF data types are converted to the IDL data types shown in the following table:

HDF Data Type | IDL Data Type
--- | ---
DFNT_CHAR8 | STRING
DFNT_UINT8 | BYTE
DFNT_INT16 | INT
DFNT_UINT16 | UINT
DFNT_INT32 | LONG
DFNT_UINT32 | ULONG
DFNT_INT64 | LONG
DFNT_UINT64 | ULONG
DFNT_FLOAT32 or DFNT_NONE | FLOAT

HDF type codes for the supported HDF data types are shown in the table below:

HDF Data Type | Type Code
--- | ---
DFNT_UCHAR | 3
DFNT_CHAR | 4
DFNT_FLOAT32 | 5
DFNT_FLOAT64 | 6
DFNT_INT8 | 20
DFNT_UINT8 | 21
DFNT_INT16 | 22
DFNT_UINT16 | 23
DFNT_INT32 | 24
DFNT_UINT32 | 25
DFNT_INT64 | 26
DFNT_UINT64 | 27

### Common HDF Tag Numbers

The following table lists common HDF tag numbers and their meanings:

Tag Number | Meaning
--- | ---
030 | Version Identifier
100 | File Identifier
101 | File Description
102 | Tag Identifier
103 | Tag Description
104 | Data Identifier Label
105 | Data Identifier Annotation
106 | Number Type
107 | Machine Type
200 | Obsolete8-Bit Image Dimensions
201 | Obsolete8-Bit Palette
202 | Obsolete8-Bit Raster Image
203 | Obsolete8-Bit Image-Compressed
204 | Obsolete8-Bit Image-INCOMP Compressed
300 | RIG Image Dimension
301 | Raster Image Look Up Table (LUT)
302 | Raster Image
303 | Compressed Raster Image
306 | Raster Image Group (RIG)
307 | RIG LUT Dimension
308 | RIG Matte Dimension
309 | Raster Image Matte Data
310 | Raster Image Color Correction
311 | Raster Image Color Format
312 | Raster Image Aspect Ratio
400 | Composite Image Descriptor
500 | XY Position
602 | Vector Image - Tek4014 Stream
603 | Vector Image - Tek4105 Stream
701 | SD Dimension Record
702 | SD Data
703 | SD Scales
704 | SD Labels
705 | SD Units
706 | SD Formats
707 | SD Max/Min
708 | SD Coordinates
710 | SD Link
720 | SD Descriptor (NDG)
731 | SD Calibration Information
732 | SD Fill Value
1962 | Vdata Description
1963 | Vdata
1965 | Vgroup