Intelligent Memory Management

It is well known that memory speed has not been able to catch up with processor speed improvements over the last 40 years. Today, memory access is the typical bottleneck in demanding computations. Furthermore, .NET advertises its much simpler memory handling for good reasons: automatic memory management removes whole categories of serious bugs and cuts development cycles.

Because neither the garbage collector (GC) nor the unmanaged heap are optimized for large arrays being created and disposed frequently, we invented a new approach to handling the memory for ILNumerics' array objects.

ILNumerics collects all memory which is not used anymore into an efficient memory pool and reuses it for subsequent operations. This brings the following advantages:

• The GC is completely prevented.
• Heap fragmentation is prevented.
• Processor cache behavior is improved.

High Performance: Prepared for Tomorrow's Hardware

Once the memory problem is solved one can start reaching for truely high performance. This is mainly a matter of efficient utilization of all available computing resources. Traditionally, such computations target a CPU. But today there is so much more we have to deal with: Several CPUs share multiple cores, each one with multiple SIMD vector extensions, potentially a vast amount of computational tasks can be done in parallel on GPUs or accelerating units....

Today, the ILNumerics Computing Engine already supports multiple cores - transparent to the user. This is one of the reasons for the higher speed in the previous example, as you can see from these performance indicators. The red line indicates the percentage of processor utilization (higher is better):

The green line shows a high number of soft page faults for the C version and much less processing capacity is spend on the computation. ILNumerics reuses the same memory over and over (leeding to almost no page faults) and is busy computing the results instead: on multiple cores in parallel.

Managing all available resources optimally is still a topic of scientific research. However, the level of abstraction and the mature runtime environment give ILNumerics a clear advantage over established native languages. Our team of engineers make more and more improvements available with each release. Your advantage: you will not be required to modify existing code in order to profit from future speed-up!