The ILNumerics Memory Management

ILNumerics’ memory management is optimized for numerical algorithms. Unlike the.NET Garbage Collector (GC) it reuses memory of large numerical objects. This reuse saves time in the GC, improves processor-cache-performance, and prevents heap fragmentation. Together with other optimizations in ILNumerics, the memory management is an important factor to the high performance of numerical algorithms.

In order to be able to reuse memory of old objects on time, arrays that aren’t needed anymore must be identified early enough. However, one characteristic of managed software environments like .NET is that they hide this kind of information from developers. How is it possible for ILNumerics to access this information anyway?

Intelligent Arrays

ILNumerics takes advantage of modern principles of object-orientated programming. All numerical algorithms are formulated in an expressive mathematical syntax based on numerical objects. These objects carry all common information about the underlying numerical data like the shape and the values of their elements. Additionally, intelligent arrays hold information about how long the object is needed by parts of the algorithm.

ILNumerics’ intelligent arrays come in two forms:

1. In mathematical algorithms most objects are temporary arrays. They are created automatically during the execution of an algorithm and carry interim results. After the first use, there is no way for a programmer to use the object further. Therefore, the memory of such objects can be reclaimed immediately.
2. Arrays which are used several times within the algorithm are declared as variables by the programmer; all variables are of a long living type.

ILNumerics’ intelligent arrays are smart enough to clean up after themselves. By default all objects are created as temporary arrays. Whether transferred to another function as a parameter or by accessing a member of the array – no matter how they are used: after the first use, their memory is immediately collected into the ILNumerics memory pool.

The only way for an array in ILNumerics to survive the first usage is to get converted into a long-living type. The conversion is done automatically, simply by assigning to any variable. ILNumerics tracks the number of references to a memory block and copies memory only when absolutely necessary. This is often refered to as 'reference counting' and 'lazy copy on write'.

For the programmer the memory management is completely transparent. ILNumerics’ intelligent arrays are able to realize a language very similar to Matlab®. In contrast to other math software, this convenience is not achieved at the expense of performance. Rather, intelligent arrays are the foundation for an efficient reuse of memory.