Cell Arrays

Cells are arrays storing arrays as elements. They serve as a general container for all kinds of arrays. Elements of cells may be cells itself, which allows to construct flexible and arbitrary complex data container.

Cells in ILNumerics bring the following advantages:

• Value semantics: accessing elements of cells stores and returns (lazy, copy on-write) clones. The content of cells is not affected by modifications of the original arrays outside the cell.
• Deep indexing: extended single index access enables one to address indivdiual values stored at arbitrary positions in the cell and to get and set their value directly.
• The cells API was designed for simple, fast and typesafe access to cell content.
• Memory management: cells integrate smoothly and transparently into the ILNumerics memory management. Multi-level reference counting ensures to free and reuse memory as soon as possible.

This manual describes the following topics:

• Cell Creation
  • Initialization
• Writing to Cells
  • Expanding / Removal
• Reading from Cells
  • Ranges / slices
  • Single Element Access
• Value Semantics
• Deep indexing
  • Functions for Deep Indexing
• Notes on using Cells

Creating Cells

Most ILNumerics array types implicitly convert to scalar cells and can simply get assigned to cell indexers or be used as arguments in functions expecting cell parameters.

In order to create cells of specific size the ILMath.cell(...)  and ILMath.cellv(...) functions are used.

Cell Initialization

Cells are regular arrays to some extent and can be empty, scalar, vector shaped or have any number of dimensions. One can create cells of a certain size and fill the elements afterwards. Or, cells can be created from a set of existing arrays.

In order to simply put some arrays into a cell one can use cellv(...), which creates a cell vector:

Note that null is a valid value for cell elements. cellv(...).Reshape(...) corresponds to the recommended way of creating arrays in ILNumerics: create a vector with the new values first and reshape the vector to the desired size afterwards.

The ILMath.cell(size,arrays,order) function gives more control about the new cell. All arguments of cell(size,arrays,order) are optional. Called without any parameters an empty cell is created. The size parameter controls the shape of the new cell:

The arrays parameter of cell(size,arrays,order) allows to specify the values (arrays) to initialize the new cell elements with. Following types are allowed as cell elements: ILNumerics arrays of arbitrary type and cells. System scalar types (numeric constants, strings, etc.) are implicitly casted into scalar ILNumerics arrays before being stored into the cell. Scalar numeric literals and variables are converted to scalar ILNumerics arrays of <double>.

Note, that the cell function expects an arrays parameter of type IEnumerable<BaseArray>. Here are two examples on how to handle that:

In ILNumerics it is good practice to prevent from 'new'-ing managed objects in tight loops. The ILMath.vector<T>(...) function does exactly that and is recommended for providing enumerable collections of arrays.

cell(size,arrays,order) is a rare example where the abstract BaseArray type is directly used. In most other places throughout ILNumerics you will continue using concrete Array<T>, Cell, Logical & Co. 

Writing to Cells

Cells can be altered in any way known from regular arrays:

The size of the new value must match or be broadcastable to the addressed range.

Expanding / Removal

Depending on the current array style cells support removal and expansion, too:

Note, that array indexers in general expect the right side of an assignment to be an array containing the values to be stored to the range of elements on the left side. In the last assignment I is the value to be stored to the position [2,2]. It has to be wrapped into a cell to satisfy the indexing API.

When accessing a single value from a cell, consider using the SetValue(value, d0,d1,...) member, which expects the value to be set directly. The following statement has the same effect as the last example: 

While SetValue() supports expansion of cells, removal of cell parts is only supported by the indexers and by SetRange().

Reading from Cells

Similarities to regular arrays continue when reading from cells. All indexing options are supported, using indexers on the cell object or the Subarray(d0,...) function. Extended options exist for single element access.

Reading Ranges / Slices

In order to read a range of values from a cell indexers are convenient in C#. Alternatively, the Subarray() function can be used. The same options exist as for common subarrays. 'Subcells' are extracted from cells by specifying the range by strings, numerical constants, arrays of sequential or dimensional indices, range definitions by r(..) / slice(..) and cells or any combination thereof. The special keywords full, ellipsis, newaxis, and end are also fully supported. Read all about subarrays.

Retrieving single cell elements

Since indexers [] and Subarray() always return a cell which contains / wraps the element value(s) addressed: how do we get to the actual cell element  value? Cells provide specialized methods for the retrieval of cell element contents as its natural type. In ILNumerics a cell element can store objects of one of the three major array types: Array<T>, Cell, and Logical. Pick the function suitable for the type of array stored in the cell element addressed:

• C.GetArray<T>(...) for a cell C retrieves a clone of the ILNumerics array with element type T at the specified position from C. The type T of the element of the inner array must be known in advance:

  It is an error if no ILNumerics array of the matching element type is found at the location specified. This function supports deep indexing.

• C.GetCell(...) for a cell C retrieves a clone of the cell stored at the specified element.

  Again, the indices must point to an existing element of cell type. Otherwise an exception is raised. This function supports deep indexing.

• C.Getlogical(...) retrieves a clone of the logical array stored at the specified element.

  The indices provided to C.Getlogical(...) must point to an existing logical array stored at the specified element. This function supports deep indexing.

Value Semantics

Data stored in a cell is protected against unintended changes. When a value is stored into a cell a clone of the new value is done and stored instead. Afterwards, the original array can be modified without altering the clone inside the cell:

Data retrieval from a cell implements the same protection. Data arrays are fetched "by value". The array returned is a (lazy, copy on-write) clone and does not allow to change the source data in the cell.

Deep Indexing

Cells can assemble arbitrary complex hierarchies. In order to access an individual element value of a cell one may retrieve the cell element, alter it outside the cell and store it back. Nothing wrong with that! But retrieving the cell array can be cumbersome, especially for deeply nested cell structures. And storing it back to its place requires to update each cell element up to the root.

There is a more straight-forward way: deep indexing. This form of index access was specifically designed for ILNumerics cell arrays. It allows to walk deep into the cell storage hierarchy and to access a single value directly – all by a single call.

Let's recall the cell E, as created above! It stores cell arrays of size [2,2] at locations [2,2] and [3,2]. These cell elements originate from cell I, containing arrays of various types and another cell H, respectively:

The following example demonstrates how to change the value of 'pi' stored inside the cell element at position [3,2] of cell E – first without utilizing the deep indexing feature. Understanding this example will be useful for the correct application of deep indexing later. 

In order to get to 'pi' we must first address from E the cell 'I' at [3,2], from that we retrieve the array of type double at location 0 which is wrapping the actual value of pi at position 0:

Deep indexing allows to chain-up the indices for each cell element array on the path to the target value. At first we show this with the GetValue() method. It returns the value as BaseArray. Recall, that any data stored in a cell fits into the BaseArray type: Array<T>, Cell, and Logical – all derive from BaseArray.

What is going on here? All indices for each array on the path from the root cell E to the value pi are lined up. Note, that it is required to provide an index for each dimension stored with an array! This makes the path to the target element unambiguous. Here, the root cell E and the cell at position E[3,2] are both 2-dimensional arrays. Chaining up the 2 by 2 indices for both arrays gives (3,2,0,0).

At this point the set of indices aims at the array storing the value of pi. This array is a scalar array, having 0-dimensions. Hence, no more indices are required in order to get to the actual element storing pi. This value is automatically wrapped into another scalar array, so that it fits into the BaseArray type.

Handling BaseArray is not always easy and in fact is not recommended. ILNumerics cell expects you to know the actual type of its values. Specifying the value type in the generic overload of GetValue<T>() gives the value of pi directly:

For writing to arrays inside cells SetValue(value, d0, d1,...) serves as the deep indexing aware counterpart to GetValue<T>(). The type of value v must match the element value type to be changed. The indices d0, d1,... must point to the exact value, considering all dimensions of each array on the path to the value.

List of Functions supporting Deep indexing

Reading Functions

The following functions for reading from cells are aware of deep indices. Functions not listed here don't support deep indexing and relate to the single array / cell they are called on only.

Function defined on Cell C	Return Type	Purpose
C.GetValue([d0[,d1[,...]]])	BaseArray	Retrieves element values without knowing the value type. Caution, unsafe! Don't use the value more than once!! Use BaseArray conversion methods to work with the return value (see: ILMath.convert(), BaseArray.ToRetArray<T>()). This method is convenient in the Immediate Window and in interactive mode for prototyping / inspecting cells during debug. Its use is not recommended for production code.
C.GetValue<T>([d0[,d1[,...]]])	T	Safe way of retrieving individual values as their natural System type. Make sure T matches the actual value type at the element to read! All indices are optional.
C[d0,...]  (get access)	RetCell	Indexers defined on all cell arrays. A scalar cell is returned, wrapping the value found. Note, that multiple overloads exist for indexers. Only the overloads receiving all single integer indices (UInt32 or Int64) support deep indexing. Such overloads capable of working with ranges/ slices, or index arrays are not deep indexing aware!
C.GetArray<T>([d0,..])	RetArray<T>	Retrieves an ILNumerics array of element type T from the element addressed by the indices (none)/d0,... within the cell hierarchy. All indices are optional.
C.GetCell([d0,...])	RetCell	Retrieves an ILNumerics cell array from the element addressed by the indices (none)/d0,... within the cell hierarchy. All indices are optional.
C.GetLogical([d0,...])	RetLogical	Retrieves an ILNumerics logical array from the element addressed by the indices (none)/d0,... within the cell hierarchy. All indices are optional.

Writing Functions

Writing to array elements within cell elements via deep indexing is supported by the following functions:

Function defined on Cell C	Value Type	Purpose
C.SetValue(value,[d0[,d1[,...]]])	T	Replaces the value of type T of the element specified by given dimensional indices from C. Supports deep indexing and expansion of the last array addressed by the indices d0,... .
C[d0,...] = value   (set access)	N/A	Indexers defined on mutable cell arrays. As value a scalar matching the element type of the array specified by d0,... is expected. If this target array is a cell any array or scalar can be provided for 'value'. Note, that multiple overloads exist for indexers. Only the overloads receiving all single integer indices (UInt32 or Int64) support deep indexing. Such overloads capable of working with ranges/ slices, or index arrays do not support deep indexing.

 

random notes on Cells

• Deep indexing does only work with indexers, if and only if all provided indices are simple integer literals or integer variables. Any range specifiers or placeholders (end, ellipsis, full) don't support deep indexing.

• All deep indexing-aware methods do also accept empty set of index arguments (i.e.: no index). This is useful when working in numpy array style on a 0-dim scalar cell C: 

• In ILNumericsV4 array style regular arrays stored inside a cell are automatically expanded when attempting to replace non-existing elements. Note that this is not available for numpy array style, though. However, both styles support the expansion of cell arrays as described above.
• Expansion and removal of parts of an array are only possible for the last array in the set of arrays specified by the deep indices. Arrays at the beginning or in the middle of the path to the targeted value cannot be resized. 
• If the set of deep indices ends at an inner cell / array without providing indices for all dimensions of that last array then are missing indices assumed to be '0' (even in numpy mode: 0 – not 'full'!)
• The mutating functions / indexer set accessors are only available on mutable cells. You will not find them on InCell or RetCell. (This is exactly the same as for regular arrays.)
• null is a valid element value. Use isnull(GetValue(d0,...)) in order to determine whether an element is null. Also, GetValue<T>(d0,...) and SetValue(null, d0,...) work with null as expected.