Category Archives: .NET

Fun with Visual Studio regexp search

I only recently realized that the Visual Studio regexp feature in Search & Replace is even able to handle regexp captures. Example: In order to locate and replace all line endings in your code which exist at the end of non-empty lines, exluding lines which end at ‘/’ or with other non-alphanumeric characters one can use:

Search Pattern: ([0-9a-zA-Z)])\r\n
Replace: $1;\r\n

searchandreplacevisualstudioMatches inside () parenthesis are captured. When it comes to replacing the match the new value is created by first inserting the captured value at the position marked by ‘$1′. This way it it possible to, let’s say insert a new char in the middle of a search pattern – if it fulfills certain conditions.

There appears to be an error in the MSDN documentation, unfortunately at the point describing how to reference captures in Visual Studio regexp replace patterns. Anyway, using the ‘$’ char in conjunction with the index is the common way and it works here just fine.

Multiple captures are possible as well. Just refer to the captured content by the index according to the count of ‘()’ captures in the search pattern.

The Visual Studio regexp support is really helpful when translating huge C header files to C# – to name only one situations. It saved me a huge amount of time already. Hope you find this useful too.

HDF5 and Matlab Files – Fun with ILNumerics

Why to use HDF5 and ILNumerics?

HDF5 is a file format (Hierarchical Data Format) especially desgined to handle huge amount of numerical data. Just to mention an example,  NASA chose it to be the standard file format for storing data from the Earth Observing System (EOS).

ILNumerics easily handles HDF5 files. They can be used to exchange data with other software tools, for example Matlab mat files. In this post I will show a step by step guide – how to interface ILNumerics with Matlab.

Continue reading HDF5 and Matlab Files – Fun with ILNumerics

ILNumerics for Scientists – Going 3D


Last time I started with one of the easiest problems in quantum mechanics: the particle in a box. This time I’ll add 1 dimension and we’ll see a particle in a 2D box. To visualize its wave function and density we need 3D surface plots.

2D Box

This time we have a particle that is confined in a 2D box. The potential within the box is zero and outside the box infinity. Again the solution is well-known and can be found on Wikipedia. This time the state of the wave function is determined by two numbers. These are typically called quantum numbers and refer to the X and the Y direction, respectively.

The absolute size of the box doesn’t really matter and we didn’t worry about it in the 1D case. However, the relative size of the length and the width make a difference. The solution to our problem reads

$\Psi_{n,k}(x,y) = \sqrt{\frac{4}{L_x L_y}} \cdot \sin(n \cdot \pi \cdot x / L_x) \cdot \sin(k \cdot \pi \cdot y / L_y)$

The Math

Very similar to the 1D case I quickly coded the wave function and the density for further plotting. I had to make sure that the arrays are fit for 3D plotting, so the code looks a little bit different compared to last post’s

     public static ILArray<double> CalcWF(int EVXID, int EVYID, double LX, double LY, int MeshSize)
        ILArray<double> X = linspace<double>(0, LX, MeshSize);
        ILArray<double> Y = linspace<double>(0, LY, MeshSize);

        ILArray<double> Y2d = 1;
        ILArray<double> X2d = meshgrid(X, Y, Y2d);

        ILArray<double> Z = sqrt(4.0 / LX / LY) * sin(EVXID * pi * X2d / LX) * sin(EVYID * pi * Y2d / LY);

        return Z.Concat(X2d,2).Concat(Y2d,2);

Again, this took me like 10 minutes and I was done.

The Visualization

This time the user can choose the quantum numbers for X and Y direction, the ratio between the length and the width of the box and also the number of mesh points along each axis for plotting. This makes the visualization panel a little bit more involved. Nevertheless, it’s still rather simple and easy to use. This time it took me only 45 minutes – I guess I learned a lot from last time.

The result

Here is the result of my little program. You can click and play with it. If you’re interested, you can download the Particle2DBox source code. Have fun!

Particle2DBoxThis is a screenshot of the application. I chose the second quantum number along the x axis and the fourth quantum number along the y axis. The box is twice as long in y direction as it is in x direction. The mesh size is 100 in each direction. On the left hand side you see the wave function and on the right hand side the probability density.

ILNumerics for Scientists – An easy start


I’ve been working as a scientist at universities for 10 years before deciding to go into industry. The one thing I hated most was coding. At the end of the day coding for scientists is like running for a football player. Obviously, you need it but it’s not what you’re here for.

I really dreaded the coding and the debugging. So much precious time for something that was so clear on paper and I just wanted the solution of my equations to see whether my idea made sense or not. More often than not scientists find that their idea was not so great and now they had spent so much time coding just to find out that the idea didn’t work. Continue reading ILNumerics for Scientists – An easy start

Fun with HDF5, ILNumerics and Excel

It is amazing how many complex business processes in major industries today are supported by a tool that shines by its simplicity: Microsoft Excel. ‘Recently’ (with Visual Studio 2010) Microsoft managed to polish the development tools for all Office applications significantly. The whole Office product line is now ready to serve as a convenient, flexible base framework for stunning custom business logic, custom computations and visualizations – with just a little help of tools like ILNumerics.

ExcelWorkbook1In this blog post I am going to show how easy it is to extend the common functionality of Excel. We will enable an Excel Workbook to load arbitrary HDF5 data files, inspect the content of such files and show the data as interactive 2D or 3D plots. Continue reading Fun with HDF5, ILNumerics and Excel

Performance on ILArray

Having a convenient data structure like ILArray<T> brings many advantages when handling numerical data in your algorithms. On the convenience side, there are flexible options for creating subarrays, altering existing data (i.e. lengthening or shortening individual dimensions on the run), keeping dimensionality information together with the data, and last but not least: being able to formulate an algorithm by concentrating on the math rather than on loops and the like.

Convenience and Speed

Another advantage is performance: by writing C = A + B, with A and B being large arrays, the inner implementation is able to choose the most efficient way of evaluating this expression. Here is, what ILNumerics internally does: Continue reading Performance on ILArray

Plotting Fun with ILNumerics and IronPython

Since the early days of IronPython, I keep shifting one bullet point down on my ToDo list:

* Evaluate options to use ILNumerics from IronPython

Several years ago there has been some attempts from ILNumerics users who successfully utilized ILNumerics from within IronPython. But despite our fascination for these attempts, we were not able to catch up and deeply evaluate all options for joining both projects. Years went by and Microsoft has dropped support for IronPython in the meantime. Nevertheless, a considerably large community seems to be active on IronPython. Finally, today is the day I am going to give this a first quick shot.

Continue reading Plotting Fun with ILNumerics and IronPython

Large Object Heap Compaction – on Demand ??

In the 4.5.1 side-by-side update of the .NET framework a new feature has been introduced, which will really remove one annoyance for us: Edit & Continue for 64 bit debugging targets. That is really a nice one! Thanks a million, dear fellows in “the corp”!

Another useful one: One can now investigate the return value of functions during a debug session.

Now, while both features will certainly help to create better applications by helping you to get through your debug session more quickly and conveniently, another feature was introduced, which deserves a more critical look: now, there exist an option to explicitly compact the large object heap (LOH) during garbage collections. MSDN says:

If you assign the property a value of GCLargeObjectHeapCompactionMode.CompactOnce, the LOH is compacted during the next full blocking garbage collection, and the property value is reset to GCLargeObjectHeapCompactionMode.Default.

Hm… They state further:

You can compact the LOH immediately by using code like the following:

GCSettings.LargeObjectHeapCompactionMode = GCLargeObjectHeapCompactionMode.CompactOnce;

Ok. Now, it looks like there has been quite some demand for ‘a’ solution for a serious problem: LOH fragmentation. This basically happens all the time when large objects are created within your applications and relased and created again and released… you get the point: disadvantageous allocation pattern with ‘large’ objects will almost certainly lead to holes in the heap due to reclaimed objects, which are no longer there, but other objects still resisting in the corresponding chunk, so the chunk is not given back to the memory manager and OutOfMemoryExceptions are thrown rather early …

If all this sounds new and confusing to you – no wonder! This is probably, because you are using ILNumerics :) Its memory management prevents you reliably from having to deal with these issues. How? Heap fragmentation is caused by garbage. And the best way to handle garbage is to prevent from it, right? This is especially true for large objects and the .NET framework. And how would one prevent from garbage? By reusing your plastic bags until they start disintegrating and your eggs get in danger of falling through (and switching to a solid basket afterwards, I guess).

In terms of computers this means: reuse your memory instead of throwing it away! Especially for large objects this puts way too much pressure on the garbage collector and at the end it doesn’t even help, because there is still fragmentation going on on the heap. For ‘reusing’ we must save the memory (i.e. large arrays in our case) somewhere. This directly leads to a pooling strategy: once an ILArray is not used anymore – its storage is kept safe in a pool and used for the next ILArray.

That way, no fragmentation occurs! And just as in real life – keeping the environment clean gives you even more advantages. It helps the caches by presenting recently used memory and it protects the application from having to waste half the execution time in the GC. Luckily, the whole pooling in ILNumerics works completely transparent in the back. There is nothing one needs to do in order to gain all advantages, except following the simple rules of writing ILNumerics functions. ILNumerics keeps track of the lifetime of the arrays, safes their underlying System.Arrays in the ILNumerics memory pool, and finds and returns any suitable array for the next computation from here.

The pool is smart enough to learn what ‘suitable’ means: if no array is available with the exact length as requested, a next larger array will do just as well:

public ILRetArray CreateSymm(int m, int n) {
    using (ILScope.Enter()) {
        ILArray A = rand(m,n); 
        // some very complicated stuff here...
        A = A * A + 2.3; 
        return multiply(A,A.T);

// use this function without worrying about your heap!
while (true) {
   dosomethingWithABigMatrix(CreateSymm(1000,2000)); // one can even vary the sizes here!
   // at this point, your heap is clean ! No fragmentation! No GC gen.2 collections ! 

Keep in mind, the next time you encounter an unexpected OutOfMemoryException, you can either go out and try to make use of that obscure GCSettings.LargeObjectHeapCompactionMode property, or … simply start using ILNumerics and forget about that problem at least.

Troubleshooting: Adding ILNumerics 3D Controls to the VS Toolbox

Adding ILNumerics visualizations to Visual Studio based projects has become a quite convenient task: It’s easy to use the ILNumerics math library for own projects in .NET. However, from time to time users have problems adding the ILNumerics controls to their Visual Studio Toolbox window.

Update: Since ILNumerics Ultimate VS version 4 this issue has been solved once for all. Simply install the MSI installer and find the ILNumerics ILPanel in the toolbox for all applicable situations.

That’s what a post on Stack Overflow from earlier this year was about: A developer who wanted to use our C# math library for 3d visualizations and simulations wasn’t able to access the ILNumerics controls. “How can I locate it?”, he was wondering. “Do I have to make some changes to my VS?”

Adding ILNumerics Controls to the Visual Studio Toolbox manually

If the ILNumerics Ultimate VS math library is installed on a system, normally the ILNumerics controls are automatically listed in the Visual Studio toolbox on all supported versions of Visual Studio. However, if that’s not the case there’s a way to a add them manually: After clicking right onto the toolbox, you can select “Choose Item”. The dialog allows you to select the assambly to load the controls from – that’s it! You will find the ILNumerics.dll in the installation folder on your system. By default this directory is located at:  “C:\Program Files (x86)\ILNumerics\ILNumerics Ultimate VS\bin\ILNumerics.dll”.

However, if that doesn’t work straightaway, it often helps to clear the toolbox from any copies of custom controls before – simply right-click it and choose “Reset Toolbox”.

Need help? ILNumerics Documentation and Support

You want to know more about our math library and its installation? Check out our documentation and the Quick Start Guide! If you have any technical questions, have a look at our Support Section.