EDIT 2014-10-02: The information in this article relates to ILNumerics Professional Edition, which is no longer maintained. However, the deployment scheme similarly holds for its successor: ILNumeris Ultimate VS. The goal is to make platform specific native binaries available at runtime. For ILNumerics Ultimate VS the number of such native libs has increased: next to mkl_custom.dll there will be several native HDF5 related dlls needing to be deployed with ILNumerics. All libs are found in the distribution package of any ILNumerics Ultimate VS Edition which allows distribution.

One issue escorted ILNumerics for just a long enough time. It is an issue which prevented ILNumerics to deploy to multiple
platform targets seamlessly. It completely prevented designer support for visualization applications when targeting 64 bit. It prevented a developer from easily switching between 32 bit and 64 bit targets in Visual Studio for testing purposes. And it – no wonder – caused a whole bunch of confusion among our users and a correspondingly huge amount of support requests: native dependencies.

AnyCPU in the Wild

It’s been a sad story from the beginning. There is that great feature of .NET which they call ‘AnyCPU’ platform target. The idea is simple: one creates an application once and simply deploys it to any platform supporting .NET. Regardless, if the target computer runs on x86 or x64 or … ok, lets stop here for now ;)! As simple as the idea is, as successfull it turns out to work in the wild. Platform specific differences between 32 and 64 bit environments are transparently abstracted away by the .NET languages (see: IntPtr) and the CLR. It is all good and fine … until native dependencies come into play.

As the name suggests, native dlls are not managed. They are compiled from pure unmanaged code. They do no abstraction work nor support such attempts by the CLR. Most of the time (at least regarding the ILNumerics native dependencies) they incorporate all the nifty pointer calculations which bring the last little quant of performance and all its danger that eventually leads us to move away from C/C++, right? Nevertheless, we sometimes still need those native libs – even though the number of such places are decreasing. All visualizations in ILNumerics run purely managed. From version 3.0 we have presented our own pure managed matrix multiplication. It works similar to the famous GOTO BLAS and handles even largest matrices in a very cache friendly way – very efficiently. It uses almost all tricks the MKL utilizes as well. And it beats all other managed implementations known to us by factors. However, it does not utilize AVX extensions (yet). Hence, is still keeps behind the MKL …

That’s where the Hassle starts

So we sometimes need native dependencies. What is the best way to incorporate them into your project without also incorporating all their disadvantages? We certainly do not want the whole application to be tied to a specific target platform just because it utilizes some routines from a native dll! We want ILNumerics to target ‘AnyCPU’ and let the final application and the machine it eventually runs on decide the bitrate. The problem with this approach is, that we need different native binaries for every platform. And even worse, the dependencies must be visible to the application at runtime.

A common deployment scheme for such native DLLs is to simply place them next to your application assembly in the same folder. According to the way a module is loaded by .NET (and Windows in general), it first looks for matching modules in the same folder where the application itself lays in. This simple scheme is sufficient for most cases – if the target platform is known in advance! However, when it comes to AnyCPU targets, this is not the case. We simply do not know if the application is to be run as 64 bit or as 32 bit process eventually.

Placing all dependencies for both 32 bit and 64 bit into the execution folder does obviously not improve the situation either. The Intel^® MKL for example is compiled to arbitrary named DLLs. However, while the entry assembly can be given an individual name, differentiating between 32 and 64 bit, this is not true for dependencies of those dependencies. ‘libiomp5md.dll’ is needed by both. And it would require some serious dive into the MKL linking scheme to have individual mkl_custom.dll’s reference individually named dependencies. Hence, we cannot place all DLLs for all targets next to each other into the same folder. The former deploy scheme of ILNumerics (prior to version 3.2) used some naming scheme in order to solve those conflicts. But in the end, this did not really solve the issue but only helped preventing from accidentally mixing up files of different platforms – not without introducing new problems…

Introducing an old Pal: %PATH%

Several ‘official’ solutions are proposed for the problem:
1. Installations. The maintainer of the application (you) takes care of selecting correct binaries during installation time and installs the application for a specific target. Alternatively, on 64 bit systems, where applications have the option to run as both, 32 bit and 64 bit processes, native dependencies are placed (‘installed’) into corresponding system directories. All 64 bit dlls go into %SystemRoot%\system32 (not mistaken here) and all 32 bit DLLs go into %SystemRoot%\SysWoW64. Don’t blame me for the naming confusion. It is a good example for derived compatibility problems and actually makes sense – just not on the first sight.

If at runtime the assembly loader attempts to load a native dependency, it looks into these individual directories as well – into which one depends on the current bitrate of the process. Going that way, dependencies with similar names are nicely seperated and everything goes well. Obviously, administrative rights are necessary to store the DLLs into those system folders. And, unless one is really carefully, this may become the entry to the famous DLL hell…

EDIT: This is the deployment scheme used by all editions of ILNumerics Ultimate VS. It installs all CLR assemblies into the GAC and makes all native dlls immediately available for them at runtime. Use this scheme for your deployment if possible! We recommend to use Windows Installer Technology in order to realize a smooth deployment experience for your users.

2. AppDomain.AssemblyResolve. This is the .NET way of dealing with the issue. Unfortunately, it introduces a whole chain of other issues which are not obvious at first sight. But the biggest argument against it is the simplicity and beauty of the third option:

3. The environment path method. It has been internalized for a long time that modules are searched for in several directories, including those which are listed in the PATH environment variable. This offers an easy yet efficient way of dealing with native binaries for different platforms. All we have to do is to make sure that the native dependencies are seperated in individual (sub-)folders. At application or library startup the current bitrate of the process is examined and the PATH environment variable is altered accordingly to include the correct directory. Several variants exist to that approach. One of them is to preload the correct version of a dll from a subdirectory at startup and let the assembly loader cache handle repeated load attempts. However, due to its simplicity we stick to the PATH environment variable method. PATH can be altered even in a medium trust environment (which is important for our Web Code Component and ASP.NET in general). It needs some attention at startup once for the whole library but does not require special handling for individual dependencies afterwards.

Manually importing ILNumerics binaries into your project

Now let’s get our hands dirty! Here comes the new file scheme for deploying ILNumerics and its native dependencies. The whole setup is simplified dramatically by using the nuget packages, which is described at the end of this post. The following manual steps are only required, if you cannot or don’t want to use the nuget package manager for some reasons. One of those rare cases: if you want to use the source code distribution of the Community Edition (GPLv3) or want to setup ILNumerics without access to the official nuget repository.

The deploy package of the Professional Edition shows the following scheme:

/ - Root folder of the zip package
- bin
|- bin32 - 32 bit (Windows and Linux) native dependencies
|- bin64 - 64 bit (Windows) native dependencies
|- ILNumerics.dll - AnyCPU .NET merged assembly, used for all targets
|- ILNumerics.dll.config - Config file, needed for Linux plotting only
|- ILNumerics.dll.xml - Intellisense support file for Visual Studio
- doc - documentation folder, changelog and offline documentation
- ... (other files not considered here)


Other editions have a similar file structure. The important parts: There are two binary folders ‘bin32′ and ‘bin64′. Both include all native binaries necessary for each corresponding platform. These binaries contained for 32 bit and for 64 bit (may) have the same names but are strictly seperated into individual folders.

If you are targeting a single platform only (let’s say: x86) you might reuse the old deployment scheme: take the binaries from the bin32 folder and make sure they are found at runtime by ILNumerics.dll. So one might choose to simply place the binaries next to ILNumerics.dll. This old scheme will still work! However, in order to enable real multi-platform target support for both 32 and 64 bit, the following steps direct the solution:

Steps to incorporate ILNumerics as multi-platform target (AnyCPU) into your existing project:
1. Extract the whole distribution package into a directory on your harddisk.
2. Add a reference to ILNumerics.dll in the package as regular managed library reference for your project. Visual Studio copies the xml intellisense documentation and the corresponding .config file automatically.
3. Use Windows Explorer to copy both: bin32/ and bin64/ including all their content into the root folder of your new project. The root folder commonly is the one the *.csproj file lives in.
4. Back to Visual Studio, open the Solution Explorer and click on the ‘Show All Files’ icon. This will make all files from the directory visible – regardless if they are part of the VS project or not.
5. Find both folders (bin32 and bin64), right click and select: ‘Include In Project’.
6. Expand the content of both folders and select all DLLs contained. Press F4 to open the Property tool window.
7. Make sure to select ‘Copy to Output Directory’ -> ‘Copy If Newer’.

Your project is now setup to run with ILNumerics as AnyCPU target! You may try it out by simply switching the project target between x86, x64 (and AnyCPU if you like). In order to test if the native binaries are available at runtime, run the following snippet somewhere in your code:

ILNumerics.ILArray A = ILNumerics.ILMath.fft(ILNumerics.ILMath.rand(100,200));

FFT in ILNumerics (still) depends on the native MKL binaries. Hence, this code would fail if they would not be found at runtime. Make sure that all needed platform targets are working by switching the application targets in the Project Properties in Visual Studio and rerunning your application.

The Happy End – Recommended Way of Importing ILNumerics

Now it is certainly nice to have a setup with native binaries which runs on every platform target without ever having to exchange native dlls manually. However, the setup can be simplified further. NuGet comes in handy here. By utilizing the NuGet package manager the setup of ILNumerics for your project boils down to the follwing three simple steps:

1. Right Click on the project in the Visual Studio Solution Explorer. Select ‘Manage NuGet Packages’
2. Search for available packages by name: ‘ILNumerics’.
3. From the list of found packages, select ‘ILNumerics’ (not ‘ILNumerics32′ nor ‘ILNumerics64′ – these are deprecated now!) and install the ‘ILNumerics’ package.

If you are familiar with our older packages, you will notice that ILNumerics is now split into two individual packages: ILNumerics and ILNumerics.Native. Former basically consists out of the ILNumerics.dll (AnyCPU) only. It is a purely managed assembly, merged with several other .NET assemblies needed by the ILNumerics visualization part. This package does not include any native binaries. These come into play as dependency package ‘ILNumerics.Native’, referenced from the main ILNumerics package. It is automatically loaded when the main ILNumerics package is referenced.

NuGet does all the work described above in the manual setup for us: referencing the managed DLL, copying the bin32 and bin64 folder, including them into the project and making sure that the native binaries are deployed to your project output directory.

Note, the new AnyCPU target support is valid from version 3.2. It replaces the old (platform specific) deployment scheme immediately. This is a breaking change for all users, which have relied on the nuget packages ILNumerics.32Bit or ILNumerics.64Bit. Both old packages are deprecated now. We recommend switching to the new deployment scheme soon.

Download ILNumerics here! Please report back any problems you may find or any restrictions the new scheme may introduce for your setup. Thanks!