ILNumerics - Technical Computing Tools

High Performance Framework for Visualization

and Computing in Industry and Science

 

tgt

Importing & Exporting Arrays

This section deals with functions for importing data into and exporting data from ILNumerics arrays. Import and export of array elements is important for convenient creation of arrays and for interacting with external APIs.

 

Importing System.Array, T[]

In order to import data from system arrays T[], the common creation function ILMath.array(T[], params int[] size) is to be used. It expects an one dimensional system array of at least the length required to create an array of given size. Elements are expected to be stored in column major order. Example:

The system array is directly be used as internal storage for the ILNumerics array! No copy will be made and the original system array is handled over to the ILNumerics memory management. Therefore, it should not be referenced from the outside afterwards!

 

Exporting System.Array, T[]

All arrays in ILNumerics provide the member function ExportValues(ref T[]). It is used to copy all element values from the ILNumerics array into a system array. If the reference to an existing system array of sufficient length is given to the function, that system array is filled with a shallow copy of all elements. If a null reference was given as parameter, a matching system array is provided by ILNumerics. Example:

If a system array is provided by the ILNumerics memory management, the length of the array returned may be larger than the number of elements in the ILNumerics array! Use the property A.Size.NumberOfElements (short: A.S.NumberOfElements) in order to find out the upper limit used.

 

Accessing arrays internal data storage

The use of the functionality described in this section requires some understanding of the internal workings of ILNumerics arrays and should be used with care! In order to alter elements of ILNumerics arrays, the common Subarray and Array Modification functionality is recommended to be used.

ILNumerics utilizes one dimensional system arrays T[] as internal element storage. In certain cases, it is necessary to access that storage array directly. One example are platform invoke calls to native (FORTRAN or C) libraries in conjunction with ILNumerics arrays. Furthermore, handling the internal storage directly, can bring further improvements in performance critical situations.

 

Read access

The A.GetArrayForRead() member of an array A returns a reference to the internal storage system array. The following precautions has to be kept in mind, when handling such a storage array:

  1. The system array may be used for read access only. The reason: the storage may be shared by multiple ILNumerics arrays. Altering elements of a system array acquired via GetArrayForRead() may lead to unexpected side effects.
  2. ILNumerics remains responsible for the internal storage system array. The user must ensure, not to reference the storage after the current operation.
  3. In a mulithreaded context, the user must take care of neccessary thread safety, since subsequent array operations may exchange or alter the storage at anytime.
  4. Be aware, that ILNumerics only garantees the minimal length of the system array returned! It might be longer than really needed! Keep this in mind, f.e. when using the system array in foreach loops. Instead of relying on the length of the system array, one should utilize the Size.NumberOfElements property of ILNumerics arrays for size information.

Example:

 

Write access

A reference for the internal storage array can be acquired for writing purpose also: A.GetArrayForWrite() returns the underlying T[] system array for A and ensures, no side effects exist when writing to its elements. The precautions 2, 3 and 4 from the paragraph Read Access above apply the same.

Interfacing external APIs

A.GetArrayForRead() and A.GetArrayForWrite() nicely fit into an efficient scheme for calling external APIs in conjunction with ILNumerics arrays. If you want to call a native lib or an assembly of some other vendor – they mostly require to provide the data as 1 dimensional array T[]. Not completely unintentionally ILNumerics uses 1 dimensional System.Array as internal storage for all of its ILArrays. An efficient data exchange with those extenal libraries is therefore easily possible - without copying data:

This scheme ensures, that the memory for the data remains under control of the ILNumerics memory management. Since the data are acquired via A.GetArrayForWrite() or A.GetArrayForRead() all requirements and precautions from the above section needs to be kept in mind.

Internal Storage Scheme

ILNumerics stores elements of arrays in column major order. I.e. the first dimension changes most frequently when iterating the elements. This storage scheme is also used by FORTRAN and Matlab and many others. Data interchange with any of those systems/languages therefore is simplyfied. For the exchange with systems storing their data in row major order (last dimension changes most frequently) a dimension flip will be necessary. ILMath.permute will be handy here.

 

*.mat file Storage

Arrays in ILNumerics can be stored into (file-)streams in the common way supported by .NET Serialization. However, the recommended way is to store arrays as Matlab® *.mat files. Currently supported is the storage and retrieval of multiple dense arrays of arbitrary element type into a single *.mat file instance.

 

Data Exchange File Formats

ILNumerics offers many ways to exchange data with other applications and APIs. Next to the common options provided by the .NET framework (text / csv, XML, binary streams, etc.) ILNumerics provides the following options to read/write numeric data:

The Data I/O section documents all these options.