Importing & Exporting Arrays
This section deals with functions for importing data into and exporting data from ILNumerics arrays. Import and export of array elements is important for convenient creation of arrays and for interacting with external APIs.
In order to import data from system arrays
The system array is directly be used as internal storage for the ILNumerics array! No copy will be made and the original system array is handled over to the ILNumerics memory management. Therefore, it should not be referenced from the outside afterwards!
All arrays in ILNumerics provide the member function
If a system array is provided by the ILNumerics memory management, the length of the array returned may be larger than the number of elements in the ILNumerics array! Use the property
Accessing arrays internal data storage
The use of the functionality described in this section requires some understanding of the internal workings of ILNumerics arrays and should be used with care! In order to alter elements of ILNumerics arrays, the common Subarray and Array Modification functionality is recommended to be used.
ILNumerics utilizes one dimensional system arrays
- The system array may be used for read access only. The reason: the storage may be shared by multiple ILNumerics arrays. Altering elements of a system array acquired via
GetArrayForRead()may lead to unexpected side effects.
- ILNumerics remains responsible for the internal storage system array. The user must ensure, not to reference the storage after the current operation.
- In a mulithreaded context, the user must take care of neccessary thread safety, since subsequent array operations may exchange or alter the storage at anytime.
- Be aware, that ILNumerics only garantees the minimal length of the system array returned! It might be longer than really needed! Keep this in mind, f.e. when using the system array in foreach loops. Instead of relying on the length of the system array, one should utilize the
Size.NumberOfElementsproperty of ILNumerics arrays for size information.
A reference for the internal storage array can be acquired for writing purpose also:
Interfacing external APIs
A.GetArrayForRead() and A.GetArrayForWrite() nicely fit into an efficient scheme for calling external APIs in conjunction with ILNumerics arrays. If you want to call a native lib or an assembly of some other vendor – they mostly require to provide the data as 1 dimensional array T. Not completely unintentionally ILNumerics uses 1 dimensional System.Array as internal storage for all of its ILArrays. An efficient data exchange with those extenal libraries is therefore easily possible - without copying data:
This scheme ensures, that the memory for the data remains under control of the ILNumerics memory management. Since the data are acquired via A.GetArrayForWrite() or A.GetArrayForRead() all requirements and precautions from the above section needs to be kept in mind.
Internal Storage Scheme
ILNumerics stores elements of arrays in column major order. I.e. the first dimension changes most frequently when iterating the elements. This storage scheme is also used by FORTRAN and Matlab and many others. Data interchange with any of those systems/languages therefore is simplyfied. For the exchange with systems storing their data in row major order (last dimension changes most frequently) a dimension flip will be necessary. ILMath.permute will be handy here.
*.mat file Storage
Arrays in ILNumerics can be stored into (file-)streams in the common way supported by .NET Serialization. However, the recommended way is to store arrays as Matlab® *.mat files. Currently supported is the storage and retrieval of multiple dense arrays of arbitrary element type into a single *.mat file instance.
Data Exchange File Formats
ILNumerics offers many ways to exchange data with other applications and APIs. Next to the common options provided by the .NET framework (text / csv, XML, binary streams, etc.) ILNumerics provides the following options to read/write numeric data:
- ILNumerics.ILMatFile class for handling Matlab *.mat files
- HDF5 as general data exchange format
The Data I/O section documents all these options.