Portable.NET on embedded systems

Rhys Weatherley, rweather@southern-storm.com.au.
Last Modified: $Date: 2002/06/12 01:52:31 $

Copyright © 2002 Southern Storm Software, Pty Ltd.
Permission to distribute unmodified copies of this work is hereby granted.

1. Introduction

This document describes how to port the Portable.NET runtime engine, ilrun, to embedded operating environments.

For the purposes of this document, an embedded operating environment is any system with a limited amount of RAM, little or no secondary storage, and a restricted set of I/O devices or user purpose.

Note: this document describes how to reduce the size of the engine so that it can operate in an embedded environment. It does not describe how to reduce the size of the runtime C# class library, pnetlib. Reducing the class library is a separate problem.

2. Basic requirements

The Portable.NET code assumes that the embedded operating environment has the following facilities:
  1. Flat 32-bit memory model.
  2. Memory blocks that are fixed in position.
  3. The ability to create large contiguous blocks of memory.
  4. Pre-emptive servicing of devices and other tasks.
If the operating environment does not have all of the above, it will be necessary to rewrite the Portable.NET engine from scratch. We have no plans to support such environments.

Primarily, if the engine allocates a block of memory, it is assumed that the block is fixed in place until it is freed.

Operating environments that attempt to reclaim memory by shifting previously allocated blocks will not work, unless the environment has some kind of virtual memory capability for remapping memory pages while retaining the original virtual addresses.

The Portable.NET engine does not have any hooks that would allow it to "return back" to a co-operative multi-tasking system at regular intervals. Any other tasks or devices in the system will need to be serviced via a pre-emptive kernel.

It is recommended that the system have at least 1 Mb of RAM. It may be possible to use the engine with smaller configurations, but the list of features will need to be heavily trimmed.

We assume that the compilation environment is GNU-compatible, with at least gcc, GNU make, autoconf, and automake available. Using other compile environments will require a lot of work.

We also assume that the operating environment's C library provides ANSI-like features such as stdio, string, and stdlib. So, you will need an embedded version of libc. The following are some suggestions:

  1. uClibc - http://www.uclibc.org/
  2. dietlibc - http://www.fefe.de/dietlibc/
  3. newlib - http://sources.redhat.com/newlib/
This is by no means a complete list of embedded C libraries, and we make no guarantees that they will work with Portable.NET. Your mileage may vary.

3. Platform support

The "support" directory contains the bulk of the platform specific code (with the exception of some math routines in the engine).

This directory is the first place to start when porting the engine to a new environment. You may need to replace memory management, locale handling, file operations, socket operations, threading, and floating-point routines, depending upon the operating environment.

Usually the "configure" script is smart enough to detect most platform-specific features, so you should only replace code in "support" if the environment is too strange for "configure" to detect automatically.

Some of the files in "support" are generic ANSI C (e.g. mempool.c, hashtab.c, utf8.c), and usually won't need to be replaced. However, your environment may already have similar routines (especially for UTF-8 handling), which you may want to use to reduce duplication.

4. Profiles

The primary means to reduce memory consumption is through profiles. These define which features are enabled or disabled when the system is built.

Each profile is defined by a file in the "profiles" directory. At compile time, the profile is converted into the header file "include/il_profile.h". Definitions in this header file are used to control which features are compiled into the final engine.

The profile is selected at build time using a "configure" option:

./configure --with-profile=kernel
The default profile is "full", which enables all features.

You may need to define a new profile that describes the capabilities of your embedded environment. Add a new file to the "profiles" directory and rebuild the system.

The following table summarises the available profile options:

IL_CONFIG_REDUCE_CODE
Take steps to reduce the size of the final code. This may disable certain debug modes that increase the size of the system to support debugging, rather than to support critical features.
IL_CONFIG_REDUCE_DATA
Take steps to reduce the amount of data that is used by the engine. This may adjust some data structures to use a more compact representation.
IL_CONFIG_DIRECT
If set, use the direct threaded version of the CVM interpreter if possible. The direct threaded interpreter uses much more memory than the alternative: token threading. Normally you will want token threading in an embedded environment.
IL_CONFIG_UNROLL
If set, use the unrolled version of the CVM interpreter if possible. This option is ignored if direct threading is disabled. Unrolled interpretation uses even more memory than direct threading, but is substantially faster.
IL_CONFIG_JAVA
Support Java class files if set. Not recommended for embedded environments (use an embedded JVM instead).
IL_CONFIG_LATIN1
Force the use of the Latin1 encoding if set. If this option is not set, the engine will attempt to use the underlying environment's notion of a locale, and a full set of Unicode character handling rules. Setting this option will dramatically reduce engine size because it can use smaller tables for Unicode character handling.
IL_CONFIG_CACHE_PAGE_SIZE
Size of a method cache page. The method cache contains the result of translating IL into CVM bytecode or native machine code. Memory is allocated from the system in blocks of this size, which must be big enough to hold the largest translated method that may be encountered in an application.
IL_CONFIG_STACK_SIZE
Size of a thread value stack, in stack words. This size is fixed once the thread has been created.
IL_CONFIG_FRAME_STACK_SIZE
Number of frames in the call stack. The call stack may grow in size if IL_CONFIG_GROW_FRAMES is set.
IL_CONFIG_GC_HEAP_SIZE
Maximum size for the garbage-collected heap. If this value is zero, then the heap is essentially unlimited (the garbage collector itself may have a hard-wired maximum limit).
IL_CONFIG_PINVOKE
Allow PInvoke methods if set. Note: if libffi is not available, then PInvoke will not be possible even if this option is set.
IL_CONFIG_REFLECTION
Provide support for "System.Reflection" classes if set.
IL_CONFIG_NETWORKING
Provide socket-based networking support for the "System.Net" classes if set.
IL_CONFIG_FP_SUPPORTED
If not set, disable floating-point instructions within the engine. Any attempt to use a floating-point instruction will throw "System.NotSupportedException".
IL_CONFIG_EXTENDED_NUMERICS
Provide support for the "System.Math", "System.Single", "System.Double", and "System.Decimal" classes if set. This option will be ignored if IL_CONFIG_FP_SUPPORTED is not set.
IL_CONFIG_NON_VECTOR_ARRAYS
Provide support for multi-dimensional arrays and arrays with non-zero lower bounds if set.
IL_CONFIG_APPDOMAINS
Allow the creation of new application domains if set.
IL_CONFIG_REMOTING
Provide support for remoting if set.
IL_CONFIG_VARARGS
Provide support for variable-argument methods if set. Variable arguments are not required for C# applications.
IL_CONFIG_GROW_FRAMES
Allow the call frame stack to grow beyond its initial size if set.
IL_CONFIG_FILTERED_EXCEPTIONS
Provide support for filtered exceptions if set. Filtered exceptions are not required for C# applications.
IL_CONFIG_DEBUG_LINES
Provide support debug line information within the engine. This is normally not much use unless reflection is also enabled.

5. Configuration options

In addition to the profiles, there are some configure options that can be supplied to alter the memory requirements and feature lists:
--enable-threads=none
Disable thread support, reducing the system to a single-threaded environment. Threading primitives (such as mutexes and monitors) are available to applications, but they behave as though there is only one thread in the system.
--without-libffi
Compile without libffi. This may actually the increase memory requirements because of the large number of marshalling stubs that are needed when libffi is unavailable. However, on systems where libffi cannot be used, this option may be unavoidable.
--without-libgc
Compile without libgc. The engine will use a default implementation, suitable for low-memory embedded systems. However, see the section entitled "Garbage collection" below for caveats.
These options do not have profile equivalents due to the configuration requirements for the third-party libffi and libgc libraries.

6. Garbage collection

Arguably, the hardest part of supporting embedded systems is the garbage collector. The libgc library is very powerful, but it is also very heavy-weight.

If you configure the system with the "--without-libgc" option, you will get a default garbage collector implementation (support/def_gc.c).

The default implementation allocates a fixed-sized block of memory and begins allocating from the lowest address. It keeps allocating until the block is full. At that time, out of memory is reported and the system stops. "Real" garbage collection is not performed.

This implementation may be suitable for use on low-memory devices where the application has been specially written to control its memory usage. It is unlikely to be suitable for executing arbitrary applications.

When porting the engine, you can replace "def_gc.c" with your own garbage collector, tuned to the particulars of your embedded environment.

The replacement algorithm must be a conservative collector, able to locate all stack-based and register-based roots on its own. Other roots are specified by the engine when it allocates "persistent" GC memory blocks. Objects are identified by pointer, not handle.

The GC should not move objects when it performs garbage collection, and it must be capable of detecting "interior" pointers, which point to the interior of an object rather than its beginning.

The collector also must be able to suspend all threads cleanly to perform mark and sweep collection. This may require some work in the thread support code to synchronise GC suspension with normal thread suspension (ILThreadSuspend).

There is no support in the Portable.NET engine for handle-based or compacting garbage collectors. A complete rewrite of the engine would be required, and we have no plans to support such collectors.

7. Embedding the engine

Normally the engine is launched using the "ilrun" program. This may not be possible in an embedded operating environment.

The "engine/ilrun.c" file demonstrates how to initialize and launch applications within the engine. Similar code will be required when embedding into other environments. The minimum steps required are:

  1. Initialize the engine with "ILExecInit".
  2. Create a "process" object with "ILExecProcessCreate".
  3. Load the application into the "process" with "ILExecProcessLoadFile" or something similar.
  4. Find the application's entry point with "ILExecProcessGetEntry".
  5. Find the "main" thread within the "process" with "ILExecProcessGetMain".
  6. Call the entry point within the context of the "main" thread with "ILExecThreadCall".
  7. Clean up the engine data structures with "ILExecProcessDestroy" (may not be necessary if the application is "infinite").

Copyright © 2002 Southern Storm Software, Pty Ltd.
Permission to distribute unmodified copies of this work is hereby granted.