The UDF Data File

The UDF Data file is the embodiment of the VIDF definition which describes it. Consisting of one or more fixed length data records, it holds the VIDF defined sensor and ancillary measurements in a series of fixed length data records. Other than the requirement that time either remain constant or advance between successive data records there are few constraints on the data record format. Basically this derives from the dynamic nature of telemetry formats. Data records can contain large or small amounts of data, can span large or small periods of time, and can hold all or just a subset of the defined measurements, and they can do all of this in a single data file.

Record Structure

The basic structure of a UDF data record is represented by the C Structure:

struct DataRecord
{
ByTe_4 Time;
ByTe_4 SpinPeriod;
ByTe_4 ZeroDegTime;
ByTe_4 HeaderRecOffsets[MaxNss];
ByTe_4 NumOfSenSets;
u_ByTe_1 Data[N];
};

where the type definitions ByTe_4, ByTe_2, ByTe_1 and u_ByTe_1 represent a 4 byte signed integer, a 2 byte signed integer, an 1 byte signed integer and a 1 byte unsigned integer respectively. All integers should have the endian definition of the local machine. The UDF software will take care of changing this format when when the data is moved to a system which has a different endian definition.

The structure fields have the following definitions.

Time
The time of day in milliseconds associated with the first measurement found in the records data field. The beginning (and ending) times of all subsequent measurements within the data record are found as offsets from this time. Times are determined from a combination of the timing fields found in the header records representing each sensor set together with various information fields from the VIDF associated with this UDF definition. The timing algorithms are fully described in the VIDF section. (see Time Flow Format , Array Timing and Time Offsets).
SpinPeriod
The spin period in milliseconds or zero if the measurements have no rotational dependence. The value can be positive (counterclockwise rotation) or negative (clockwise rotation).
ZeroDegTime
The most recent time at which a defined 0° line crossed a defined external reference location. For example the reference line could be a satellite mounted sun sensor in which case the reference location would be the sun. Both the 0° line and the external reference point should be clearly laid out in the VIDF. If the SpinPeriond field is 0 then this field holds an offset angle in units of degrees times 10 from an external reference location.
With a non-zero SpinPeriod the beginning or ending phase associated with any measurement is determined from
&phi = (T - ZeroDegTime) * 360.0 / SpinPeriod + Offset
where T is the beginning or ending time of the measurement in milliseconds, SpinPeriod and ZeroDegTime are the values found in their respective data record fields, and Offset is the angular separation of measurement location from the 0° line. The latter value is determined from one of the VIDF constant definitions and if not found is assumed to be 0°. The UDF kernel software always normalizes &phi to be between 0° and 360°.
HeaderRecOffsets
A fixed length array which holds the byte offsets into the UDF Header File to the header records associated with each of the defined sensor sets in the record Data field. The length of the array is the value found in the Maximum Number of Sensor Sets field in the VIDF associated with this UDF definition. Should there be fewer sensor sets defined in the data record than the maximum allowed, the excess elements in the array array are ignored. A HeaderRecOffsets value of -2 is used as an EOF indicator.
NumOfSenSets
The number of sensor sets contained in the record data field. If the value is negative then the absolute value must be taken. A positive value indicates that there is one HeaderRecOffsets entry for each sensor set. A negative indicates that there is a single HeaderRecOffsets which is to be used for each of the defined sensor sets.
Data
A free-form fixed-length array of N bytes holding the UDF sensor and ancillary class data. The array byte size is determined from:
N> = (Len - 4 * (MaxNss + 4)
where Len is the record byte length and MaxNss is the maximum number of sensor sets definable for this UDF.

Data Array Specifics

While free-form, the data array does have an imposed super-structure if only to make be able to make some sense of how data is laid down within it and how to set the VIDF fields and Header record field values that the generic software uses as a road-map when finding and returning data. At the top level the data array is divided into two major blocks of data. The first of these is known as the Preface Ancillary data block and the second as the Sensor Set data block which consists of a series of partitions known as sensor sets. A generalized data array field then has the format:

PREF ANC BLK SENSOR SET BLOCK
PREFACE
ANCILLARY		 SENSOR
SET 0		 SENSOR
SET 1		 SENSOR
SET 2		 ... SENSOR
SET i		 ... SENSOR SET
MaxNss - 1

The Preface Ancillary Data Block

The first data laid down in the data array field is that found within Preface Ancillary data block. This block contains all of the measurements which have been defined in the VIDF as Ancillary measurements and which have been assigned the subclass of Preface. If no data falls into this category within the VIDF then this block of data is omitted in the filling of the data field. The data is entered into the array in the same order in which it is defined in the VIDFS, that is the first measurement defined of this subclass is the first measurement in the array, the second defined measurement occupies the second position and so forth.

All measurements placed within this data block are stored using the same word length which is either 1, 2 or 4 bytes. The size is determined by the size required to hold the largest defined measurement in the subclass. Measurements which require less than the allocated byte size may be packed within this word length and then broken out via a VIDF algorithm. The bit length of each defined ancillary measurement is found in the VIDF Ancillary Bit Lengths field.

Preface Ancillary Data are restricted to be scalar measurements and are considered to be valid for the entire time frame covered by the data record.

The Sensor Set Block

Every data array field has a Sensor Set Block which contains at least one sensor set. Sensor sets are collections of measurements which can be completely described by the contents of a single header file record. Remeniscent of the data record itself, they can hold large or small amounts of data, can span large or small periods of time, and can hold all or just a subset of the defined measurements.

Structurally a sensor set can be imagined to look as below.

SENSOR SET
G
E
N

A
N
C
 A0g S0,0 S1,0 S2,0 ... Si,0 ... Sn,0 S
E
N
S
O
R
A1g S0,1 S1,1 S2,1 ... Si,1 ... Sn,1
A2g S0,2 S1,2 S2,2 ... Si,2 ... Sn,2
A3g ... ... ... ... ... ... ...
A4g S0,m S1,m S2,m ... Si,m ... Sn,m
... A0,a A0,a A0,a ... A0,a ... A0,a S
E
N

A
N
C
Aqg A1,a A1,a A1,a ... A1,a ... A1,a
... ... ... ... ... ... ...
Ar,a Ar,a Ar,a ... Ar,a ... Ar,a

The sensor set is broken into three separate blocks, the General Ancillary Block, the Sensor Block, and the Sensor Ancillary Block. Of these only the Sensor Block must be present. In the 2-D representation similiar measurements are contained within the data columns. Sensor set data is placed into the data array sequentially, that is data from the first sensor set followed by data from the second sensor set, etc.

General Ancillary Block

When adding data from a sensor set to the data array, the first data added is the data found in the General Ancilliary Data block. This block contains all of the measurements which have been defined in the VIDF as Ancillary measurements and which have been assigned the subclass of General. If no data falls into this catagory within the VIDF then this block of data is does not exist in the sensor set. The data is entered into the array in the same order in which it is defined in the VIDFS, that is the first measurement defined of this subclass is the first measurement output, the second defined measurement is the second measurement output and so forth. The Preface Ancillary Data, General Ancillary Data is restricted to be scalar data.

All measurements placed within this data block are stored using the same word length which is either 1, 2 or 4 bytes. The size is determined by the size required to hold the largest defined measurement in the subclass. Measurements which require less than the allocated byte size may be packed within this word length and then broken out via a VIDF algorithm. The bit length of each defined ancillary measurement is found in the VIDF Ancillary Bit Lengths field.

All data in the General Ancillary is valid for the entire period of time spanned by the sensor set. It is not associated with any of the sensors being returned in the Sensor Block.

The Sensor Block

The Sensor Block in the sensor set contains measurements that are defined in the VIDF under the heading Sensor Data . Only the measurements actually returned in the time frame covered by the sensor set are included in the Sensor Block.

The Sensor Block is transferred into the data array field immediately after the General Ancillary Block. The data is transferred column by column. The data is stored in the array using a fixed word length of 1, 2, or 4 bytes. The size is determined by the size required to hold the largest defined sensor measurement. Measurements which require less than the allocated byte size may be packed within this word length and then broken out via a VIDF algorithm. The bit length of each defined sensor measurement is found in the VIDF Sensor Data Length field.

How to interprate the Sensor Block contents depends in part on the class of data it contains. The three separate VIDF data classes are considered separately below. Since a UDF definition is limited to contain a single class of data, the sensors sets will likewise are limited to one class.

Scalar
Each column of data in the Sensor Block contains measurements from one VIDF sensor. Sensors cannot occur more than once in the Sensor Block. Multiple values down a column are repititions of the measurement. The overall size of the sensor block as well as its contents is determined through various fields in the header record associated with the sensor set. The column length is the value found in NumSample field while the number of columns is given in the NumSensors field. The sensor corresponding to each column in the Sensor Block is obtained from the SensorIndex field.
Array
Each column of data in the Sensor Block contains a single array measurememnt from one VIDF sensors. Sensors cannot occur more than once in the Sensor Block. Values down a column are the array values returned in measurement. Any subset of the full array may be returned. Each measurement in the Sensor block must not only have identical array lengths but also identical array indices. The overall size of the sensor block as well as its contents is determined identical to the mannor described for scalar measurements. The column length is the value found in NumSample field while the number of columns is given in the NumSensors field. The sensor corresponding to each column in the Sensor Block is obtained from the SensorIndex field.
Matrix
Each column of data in the Sensor Block contains a single column of data from a returned matrix of data. The overall size of the sensor block as well as its contents and organization is determined from various fields in the header record associated with the sensor set. The column length is is the value returned in the NumSample field. The number of columns in a returned matrix is the value returned in NumColumns field and the number of matrices being returned is given in the NumSensors field. The number of columns in the sensor set is then NumColumns * NumSenors. The sensor corresponding to each matrix in the Sensor Block is obtained from the SensorIndex field. The matrix associated with the first sensor begins at column 0, with the second at column NumColumns, with the third at column 2 * NumSensors, and so forth.

All data within the Sensor Set Sensor Block are stored in a common word length of 8, 16 or 32 bits. Multiple data values may be packed within the chosen base size and then broken out using a VIDF algorithm.

Sensor Ancillary Block

The last data added to the data array from a sensor set is the data found in the Sensor Ancilliary Data block. This block contains all of the measurements which have been defined in the VIDF as Ancillary measurements and which have been assigned the subclass of Sensor. If no data falls into this catagory within the VIDF then this block of data is does not exist in the sensor set.

Sensor Ancillary data are generally sensor specific information which are required to convert the sensor measurement data into meaniful units. A simple example would be a set of gains which may be different for each returned measurement. When a measurement is declared to be a Sensor Ancillary measurement one instance of that measurement is returned with each sensor in the sensor set as shown in the sensor set diagram above. The values attached to each sensor need not be identical. A value is returned even if the Sensor Ancilliary measurement is not applicable to a given sensor or sensors..

All of the Sensor Ancillary measurements listed in the VIDF are added to each of the sensor set columns below the sensor definitions. The measurements are added in the order in which they are defined in the VIDF. Unlike either Preface or General Ancillary Data, Sensor Ancillary data can be array data if the Sensors represent either Array or Matrix data. The size of the Sensor Ancillary arrays cannot exceed either the array or column length of the Sensor data. They can in fact, however, be any length up to that limit. How many elements are associated with any defined Sensor Ancillary data measurement can be determined using information in the VIDF. (see the section on Ancillary Mapping in the VIDF document).

All measurements placed within this data block are stored using the same word length which is either 1, 2 or 4 bytes. The size is determined by the size required to hold the largest defined measurement in the subclass. Measurements which require less than the allocated byte size may be packed within this word length and then broken out via a VIDF algorithm. The bit length of each defined ancillary measurement is found in the VIDF Ancillary Bit Lengths field.

The Sensor Ancillary data is added to the data array in the same manner as was the Sensor Block data, column by column.

&larr Previous TOC HOME Next &rarr
OverView The UDF Header File