2012 was an amazing year, here’s a summary!

Posted by Filip Ekberg on January 8 2013 1 Comment

Saying that a lot happened in 2012 is probably an understatement. At least both on this blog and in my personal life, a bunch of amazing things have happened. I really hope that your previous year was good and let’s hope for an even better 2013. To start this year off I want to summarize all the great posts that were shared on this blog in 2012.

Personally the two biggest achievements of last year was me getting engaged to my lovely Sofie and publishing my book C# Smorgasbord. As you might have seen already this year has already started very good as I have been awarded Microsoft MVP in Visual C#!

Let me know what you found most interesting on this blog from the collection of posts below! Here is the 2012 summary!

Architecture

Tips & Tricks

Screencasts

Software & Tool information

Windows 8, Windows RT, WinRT and Surface

C# Smorgasbord and Self-publishing

Other

I hope you found this collection of posts useful and that you’ve learnt a lot in 2012! Enjoy 2013 and let me know what you think of the posts! Before we take part for this time, I want to share with you an image that describes the feeling I got when I held the first printed copy of C# Smorgasbord:

Vote on HN

A summary of 2011 and a look at what is about to come

Posted by Filip Ekberg on January 5 2012 5 Comments

Happy 2012 folks! I hope you all had a wonderful new years eve!

The year 2011 has been very interesting, we’ve had the opportunity to see some very exciting things that is about to reach the market. I’d like to summarize what I’ve been writing about in 2011 and breifly tell you guys what’s about to expect from me in 2012.

Let’s get to it, here’s a link summary to all my posts in 2011

Dynamic programming

Software Architecture

Compiler as a Service

Screencasts

Other

As you can see a lot of exciting things has been discussed in 2011 and my vision is that 2012 gets at least as exciting! We’ve still got a lot of fun areas to cover and even more useful products to look at.

Worth mentioning is that I am currently in the work of assembling a book about C# Programming, almost all the above articles and some more will be featured in this book but with a lot of extra content that makes it even more fun to read. If you are interested in being a technical reviewer, editor or anything else that might be helpfull, just drop me an email or a dm.

Vote on HN

When can knowing about IL and the internals be useful?

Posted by Filip Ekberg on October 19 2011 Leave a Comment

Lately we’ve been looking a lot on how you can use IL to create types at runtime, the same IL that we have emitted at runtime is what the C# compiler compiles the C# code to. So the first thing that comes to mind when someone asks me when knowing about IL and the internals of C# can be useful is when you want to write a compiler. Let’s say that you for educational purposes want to create a very simple language and you want to be statically typed and usable by other .NET languages, then compiling the code to MSIL will allow you to do just that.

Back when I studied for my bachelor in Software Engineering I attended a course the last year that was called “Advance UNIX Programming” one of the laboratory practicals that we did in this course was to complete a compiler, and when I say to complete it I do not mean write it from scratch in this case. We had other courses that covered the depth of compiler technology, however in this particular programming course we were given an application that parsed a programming language and our assignment was to make it generate the correct assembler.

The assembler that we generated was in this case GAS and the custom programming language they called “Calc” looked like this:

a=732;
b=2684;
while(a != b) {
  if(a > b) {
    a=a-b;
  } else {
    b=b-a;
  }
}
print a;

print a gcd b;

As you can see on the bottom we also had to create some library functions such as “gcd“. Basically we only had one file that we had to edit in, or to be even more specific we did everything inside one big method that had a switch case that told us what kind of operation that we were currently working with. So we had a lot of parsing done already, there are a lot of tools out there if you want to parse a programming language, there’s something called lexical analyzing that you can look into.

In this laboratory practical the purpose was to spit out correct assembler and show that we had somewhat a knowledge of how the stack worked and how to use certain operations, here are two examples take from that laboratory practical, the actual programming language used in the generator is C:

case '+':   printf("\tpopl\t%%eax\n\tpopl\t%%ebx\n\tadd\t%%ebx, %%eax \n\tpushl\t%%eax\n"); break;
case '-':   printf("\tpopl\t%%ebx\n\tpopl\t%%eax\n\tsub\t%%ebx, %%eax \n\tpushl\t%%eax\n"); break;

So in this big switch when it has found a + and a – this is what is done. It might be a bit hard to see in that printf-statement, but here’s the generated assembler:

popl   %eax
popl   %ebx
add    %ebx, %eax
pushl  %eax

As you can see here, it’s pretty similar to MSIL. Now this might not normally be how generate the assembler, you might have it a little bit nicer in the actual compiler than just one-liners, but I just wanted to show you some really old stuff.

I won’t go through how you write a compiler here, it’s a very broad topic and take a very long time to master, if you like to read about deep level stuff like how the people behind the compilers think, read Eric Lipperts blog..

Essentially a compiler is just a program that translates your text into some other format that is either readable by another compiler or directly by a machine. In the laboratory practical that I shared with you above, we actually generated GAS and compiled it with GCC to make it executable.

So everything we’ve been looking at lately is directly applicable if you want to start off by writing a compiler for a new language. I really hope you’ve enjoyed the reflection series that has been focusing on IL generations and dynamic methods, there will most certainly be more of them around in the future.

I hope you found this interesting because I had a lot of fun writing it and if you have any thoughts please leave a comment below!

Vote on HN

Creating a Dynamic Method that uses a Switch

Posted by Filip Ekberg on October 18 2011 1 Comment

We’ve looked at some very interesting Dynamic Method creation in the last posts and let’s keep this up! We’ve looked at how we can compare values and then jump somewhere if the comparison is evaluates to true. Now let’s take a look at how we can implement a switch!

So I was thinking that I wanted to create the following method as a dynamic method:

int Calculate(int a, int b, int operation)
{
    switch(operation)
    {
        case 0:
            return a + b;
        case 1:
            return a * b;
        case 2:
            return a / b;
        case 3:
            return a - b;
        default:
            return 0;
    }
}

Basically it takes three integers, the first two integers are the ones that I do the mathematical operation on and the last argument tells the switch statement which of the operation to run. We already know how to do almost everything that we are about to look at, these are the following operation codes that we’ve seen before that we’re going to see again in this post:

There are just two operations that we haven’t looked at yet that we are going to use here and that is OpCodes.Switch and OpCodes.Br_S, we’ll see how these are used a bit further down.

But first off let’s take a look at how we get started, as we’ve done before we create our DynamicMethod instance:

Type[] methodArguments = {
    typeof(int),
    typeof(int),
    typeof(int)
};

var calculateMethod = new DynamicMethod(
        "Calculate",
        typeof(int),
        methodArguments,
        typeof(Program).Module);

The method will have three arguments and a return type of integer and we’re going to call it “Calculate”. Next up get the IL-generator:

var il = calculateMethod.GetILGenerator();

Let’s start emitting some IL!

However, before we do that, we need to define a couple of targets for our switch to jump to, if you’ve read the previous post you might remember that an if-statement evaluates something and if it’s true, it jumps into it’s context. It’s the same with a switch-statement.

You provide a value that the switch statement bases it’s evaluation on and then you give it entry points to where it can jump. By looking at the method we are trying to convert we have 4 different labels to where we want to jump and one default label.

So first of all we create our 5 labels like this:

var defaultCase = il.DefineLabel();
var endOfMethod = il.DefineLabel();
Label[] jumpTable = new [] {    il.DefineLabel(),
                                il.DefineLabel(),
                                il.DefineLabel(),
                                il.DefineLabel() };

Now we’ve prepared the jump table, so we can start look at the beginning of the method. And the first thing we want to do in the method is to perform a switch on our third argument, so what we need to do is to add the third argument to the evaluation stack and call the switch operation:

// Perform switch
il.Emit(OpCodes.Ldarg_2);
il.Emit(OpCodes.Switch, jumpTable);

The next thing we need to do is, if the switch didn’t jump anywhere, we need to define the default case and that is done like this:

il.Emit(OpCodes.Br_S, defaultCase);

We just emit the call because if it goes pass the Switch-operation, it didn’t find what it was looking for! Now next up is four almost identical operations. What we do is that we first of all mark a label saying where the current case is and then we load our two first arguments on to the evaluation stack and then we perform the mathematical operation and jump to the end of the method, it will look something like this:

// Case 0 - Perform Add on Ldarg_0 and Ldarg_1
il.MarkLabel(jumpTable[0]);
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Ldarg_1);
il.Emit(OpCodes.Add);
il.Emit(OpCodes.Br_S, endOfMethod);

Can you guess what the next case will look like?

Actually I am just going to give you the next three math operations!

// Case 1 - Perform Mul on Ldarg_0 and Ldarg_1
il.MarkLabel(jumpTable[1]);
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Ldarg_1);
il.Emit(OpCodes.Mul);
il.Emit(OpCodes.Br_S, endOfMethod);

// Case 2 - Perform Div on Ldarg_0 and Ldarg_1
il.MarkLabel(jumpTable[2]);
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Ldarg_1);
il.Emit(OpCodes.Div);
il.Emit(OpCodes.Br_S, endOfMethod);

// Case 3 - Perform Sub on Ldarg_0 and Ldarg_1
il.MarkLabel(jumpTable[3]);
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Ldarg_1);
il.Emit(OpCodes.Sub);
il.Emit(OpCodes.Br_S, endOfMethod);

As you can see, the only difference is the operation for the math operation and the label that we mark! Now we’ve almost reached the end of the method, so we need to define what happens if it was a default value or not and then return our value, since all the above operations have put the value on the evaluation stack, we are good to go!

il.MarkLabel(defaultCase);
il.Emit(OpCodes.Ldc_I4, 0);

il.MarkLabel(endOfMethod);

il.Emit(OpCodes.Ret);

If it’s the default case, all we do is add a 4 byte integer to the evaluation stack with the value 0 and then return from the method!

Let’s try it out!

var calculate =
    (Func<int, int, int, int>)calculateMethod.CreateDelegate(typeof(Func<int, int, int, int>));
var result = calculate(1, 2, 0); // 3
result     = calculate(1, 2, 1); // 2
result     = calculate(1, 2, 2); // 0
result     = calculate(1, 2, 3); // -1

Here’s the entire code that I used above:

Type[] methodArguments = {
    typeof(int),
    typeof(int),
    typeof(int)
};

var calculateMethod = new DynamicMethod(
        "Calculate",
        typeof(int),
        methodArguments,
        typeof(Program).Module);



var il = calculateMethod.GetILGenerator();

var defaultCase = il.DefineLabel();
var endOfMethod = il.DefineLabel();
Label[] jumpTable = new [] {    il.DefineLabel(),
                                il.DefineLabel(),
                                il.DefineLabel(),
                                il.DefineLabel() };

// Perform switch
il.Emit(OpCodes.Ldarg_2);
il.Emit(OpCodes.Switch, jumpTable);

// Default case
il.Emit(OpCodes.Br_S, defaultCase);

// Case 0 - Perform Add on Ldarg_0 and Ldarg_1
il.MarkLabel(jumpTable[0]);
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Ldarg_1);
il.Emit(OpCodes.Add);
il.Emit(OpCodes.Br_S, endOfMethod);

// Case 1 - Perform Mul on Ldarg_0 and Ldarg_1
il.MarkLabel(jumpTable[1]);
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Ldarg_1);
il.Emit(OpCodes.Mul);
il.Emit(OpCodes.Br_S, endOfMethod);

// Case 2 - Perform Div on Ldarg_0 and Ldarg_1
il.MarkLabel(jumpTable[2]);
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Ldarg_1);
il.Emit(OpCodes.Div);
il.Emit(OpCodes.Br_S, endOfMethod);

// Case 3 - Perform Sub on Ldarg_0 and Ldarg_1
il.MarkLabel(jumpTable[3]);
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Ldarg_1);
il.Emit(OpCodes.Sub);
il.Emit(OpCodes.Br_S, endOfMethod);

il.MarkLabel(defaultCase);
il.Emit(OpCodes.Ldc_I4, 0);
               
il.MarkLabel(endOfMethod);

il.Emit(OpCodes.Ret);

var calculate =
    (Func<int, int, int, int>)calculateMethod.CreateDelegate(typeof(Func<int, int, int, int>));

var result = calculate(1, 2, 3);

I hope you found this interesting because I had a lot of fun writing it and if you have any thoughts please leave a comment below!

Vote on HN

Creating a recursive dynamic method that calculates factorial

Posted by Filip Ekberg on October 17 2011 Leave a Comment

In the last post we looked at how we could call a dynamic method, now let’s have a look at how we can create a recursive method that calculates factorial for non-negative integers! First of all we should take a look at what factorial means, usually it’s written like this:

5!

Which means that it will perform this:

5*4*3*2*1

So by looking at the above sequence we can see that we want to perform the multiplication operation over and over again, reducing the integer by 1 until it’s equal to one. Here’s a good explanation if you don’t know what recursion is:

Recursion is the process of repeating items in a self-similar way. For instance, when the surfaces of two mirrors are exactly parallel with each other the nested images that occur are a form of infinite recursion.

If you are having troubles understanding what recursion is, start by reading this blog post from the beginning again. When talking about recursion, you look for one or more base cases, these are the times that the recursion ends. In the above, the base case is when the integer has been reduced to 1.

Let’s start off by looking at the method signature here, the method will return an integer and take an integer as its first argument:

int Factorial(int x)

Then we want to define our base base, which in this case will be when x is equal to 1, then we want to end the recursion and return x:

if (x == 1) return x;

The last part is a bit tricky to understand, what we want to do is that we want to multiply x with the value of Factorial(x-1) this means that if we call Factorial(3) we expect the following to happen the first time:

return 3*Factorial(3 - 1);

In this case, we will call the method with the value 2 and then the return-statement will look like this:

return 2*Factorial(2 - 1);

But now when we’re inside Factorial this time, x will be equal to 1, so we will return 1, which means that we will actually multiply 2 by 1 and then we return that value and we will return 3 by 2.

This is the entire recursive method that we want to produce as a dynamic method:

int Factorial(int x)
{
    if (x == 1) return x;
    return x*Factorial(x - 1);
}

Just as we’ve done in the previous posts, we instantiate our DynamicMethod like this:

Type[] methodArguments = {
    typeof(int)
};

var recursiveFactorial = new DynamicMethod(
        "Factorial",
        typeof(int),
        methodArguments,
        typeof(Program).Module);

var il = recursiveFactorial.GetILGenerator();

As you can see here we expect one parameter and that is an integer and we expect a return value which will be an integer as well. Now, there is one new operation code that we are going to look on today and there are two other methods on the il-generator that we will explore. First of all we need to look at how we can create an if-statement and jump to somewhere in the code.

Basically what an if-statement does is that it evaluates if two values conform to a certain rule, it might be equality or not-equality among others and then it tells you to go somewhere, either you enter the body or you continue after the body of the if-statement.

When we look at IL it’s not so different, but at a first glance it might look like that. So first of all, we need to be able to define somewhere where we will go if our statement is true, this is done by calling DefineLabel() on our il-generator instance like this:

var endOfMethod = il.DefineLabel();

This will give us an instance of the class Label and if you’ve not seen labels before in any programming languages, it might look like this:

someLabel:
    Console.WriteLine("Goto test");
goto someLabel;

A more common scenario might be if you have nested loops, which might not be a good idea in the long term anyways but if you do, using labels will allow you to break the outer loop. In our case on the other hand, we will use the label to jump somewhere when our check evaluates to true. Evaluating if two values are equal is done by using the operation code OpCodes.Beq. It will assume that two values are pushed onto the evaluation stack and if they are equal, it will jump to the label that you’ve emitted with the operation code. Like this:

il.Emit(OpCodes.Beq, endOfMethod);

This will jump to wherever endOfMethod is if the two values on the evaluation stack are equal. So now that we’ve covered almost all of the new things, let’s get rocking with some more IL!

The first thing that I want to do when my method enters, is reading the value passed to my method, this is because I want to use this method later on.

The second thing is to check if the parameter is equal to one, however the branch-is-equal operation will pop the two values from the evaluation stack, so in order for us not to lose our value passed to our method, we just push it onto the stack again, we also push the value 1 because that’s what we want to evaluate against and the stack will look something like this now:

Which means that after we’ve done:

il.Emit(OpCodes.Beq, endOfMethod);

It will look like this:

We haven’t defined where endOfMethod is yet, we just have the label for usage. So the next thing we want to emit is what happens if the branch did not work and this is the subtraction. This assumes that we have two values on the evaluation stack as well, so we just need to push the value 1, which is what we want to subtract with:

il.Emit(OpCodes.Ldc_I4, 1);

So now the stack looks like this:

The next thing we do is to actually call the subtraction:

il.Emit(OpCodes.Sub);

This will pop the two values off the stack and push the result so the stack will look like this:

When the subtraction is done, we want to call the method recursively and to do this, we just call our own dynamic method and since we have the result from the subtraction on the evaluation stack, this will be treated as the first argument to the method:

il.Emit(OpCodes.Call, recursiveFactorial);

The code after this is where we are when the method actually returns and here we want to multiply the return value by the argument that we first sent to our method, the return value from the method is already on the evaluation stack, so we just need to add the argument again and call the multiplication operation:

il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Mul);

Everything above up until where we did the branch is when the base case did not fall through now we need to define the entry point for the end of our method, which means that we have to mark where the label should be. This is how we do that:

il.MarkLabel(endOfMethod);

However, both the base case and every time this method is called have a similar ending and that is returning a value, the value will always be on the evaluation stack either if it’s from the multiplication above, or if it is from the subtraction even further up:

il.Emit(OpCodes.Ret);

This is the entire il that I emitted:

// Either to return or send as argument to recursive call
il.Emit(OpCodes.Ldarg_0);

// Compare the argument value to 1
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Ldc_I4, 1);
// Jump to endOfMethod if the argument value is equal to 1
il.Emit(OpCodes.Beq, endOfMethod);

// Subtract 1
il.Emit(OpCodes.Ldc_I4, 1);
il.Emit(OpCodes.Sub);

// Do recursive call
il.Emit(OpCodes.Call, recursiveFactorial);

// Multiply the return value by the argument value
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Mul);

il.MarkLabel(endOfMethod);
il.Emit(OpCodes.Ret);

To try it out, we can call it like this:

var toInvoke = (Func<int, int>)recursiveFactorial.CreateDelegate(typeof(Func<int, int>));
var fact = toInvoke(10);

And this is the result:

I hope you found this interesting because I had a lot of fun writing it and if you have any thoughts please leave a comment below!

Vote on HN

Calling a dynamic method from a dynamic method

Posted by Filip Ekberg on October 16 2011 Leave a Comment

Let’s take a look at how we can call a method from a dynamic method. In the last post we looked at how we called a static method in our current context, but let’s take a look at how we can call another dynamically created method that takes an integer parameter and then does some math operation on it and then returns it.

First off we need to create this method, we’re just going to use IL that we’ve seen before and I am going to add the input parameter with 2. The dynamic method will look like this with the emitted IL:

var addMethod = new DynamicMethod(
        "AddMethod",
        typeof(int),
        methodArguments,
        typeof(Program).Module
    );
var il = addMethod.GetILGenerator();

il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Ldc_I4, 2);
il.Emit(OpCodes.Add);
il.Emit(OpCodes.Ret);

So we load the first argument onto the evaluation stack, then we add a 4 byte integer with the value of 2 onto the evaluation stack and last we call OpCodes.Add.

I don’t know if you’ve noticed this before, but Dynamic Method derives from MethodInfo, which means that we can just call this method!

We’ve got the following code from the last blog post:

var mathOperation = new DynamicMethod(
        "AdvanceMathOperationMethod",
        typeof(void),
        methodArguments,
        typeof(Program).Module);

il = mathOperation.GetILGenerator();
var methods = typeof(Program).GetMethods();

In both these dynamic methods, we use the variable methodArguments that we’ve also seen before:

Type[] methodArguments = {
                typeof(int)
};

It just says that we expect an integer parameter sent to the method. Let’s take a look at the IL we’re going to emit, first of all we want to load the argument onto the evaluation stack then we want to add the value 10 and multiply these and pass the result as a parameter to the add method.

So this code is the same from the last blog post:

il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Ldc_I4, 10);
il.Emit(OpCodes.Mul);

So now we’re prepared to call the dynamic add method we’ve created and since it derives from MethodInfo we can just do this:

il.Emit(OpCodes.Call, addMethod);

Since the result from the multiplication is already on the evaluation stack it’s also the first argument that this method gets!

The Add method in the dynamic add method we’ve created will also have it’s value on the evaluation stack, so let’s print it with our static method from the last blog post:

il.Emit(OpCodes.Call, methods.First(x => x.Name == "PrintWithSpecificFormat"));
il.Emit(OpCodes.Ret);

And now if we invoke this:

var toInvoke = (Action<int>)mathOperation.CreateDelegate(typeof(Action<int>));
toInvoke(10);

we should see this printed:

The value is: 102

I hope you found this interesting and if you have any thoughts please leave a comment below!

Vote on HN

Exploring OpCodes with DynamicMethod and looking at the evaluation stack

Posted by Filip Ekberg on October 16 2011 1 Comment

In the previous post we looked at how our DynamicMethod could pass a value to another method, let’s take a look at something a little bit more interesting! Consider that I want to have a method that takes an integer and this integer is manipulated and then printed out by this method. In this case, the manipulation is a multiplication and the second method is just a method to print the result in a nicely formatted way.

This is what we have, from the previous post. First of all we got the method that prints it in a formatted way, I’ve renamed it for clarity in this post:

public static void PrintWithSpecificFormat(int a)
{
    Console.WriteLine("The value is: {0}", a);
}

The second thing we have is the DynamicMethod, however, I’ve made two changes here, instead of taking zero parameters, I am now taking an integer parameter, secondly I’ve renamed it to something more understandable:

Type[] methodArguments = {
                            typeof(int)
};

var mathOperation = new DynamicMethod(
        "AdvanceMathOperationMethod",
        typeof(void),
        methodArguments,
        typeof(Program).Module);

ILGenerator il = mathOperation .GetILGenerator();
var methods = typeof(Program).GetMethods();

Now let’s start off by looking at the operations that we are going to emit. First of all we need to load the argument that we pass onto the evaluation stack this is done by OpCodes.Ldarg_0. Before we look at the other operations, notice that I mentioned the evaluation stack here, from now on it’s important to understand a little bit about the evaluation stack. Values are pushed and popped onto the evaluation stack and certain operations except values to already be on the evaluation stack.

If you don’t know what a stack is, here’s a quick little intro with a bit of help from our friend Wikipedia. First of all, this is what a stack might look like when represented on a paper:

As you can see, elements that are pushed onto the stack, is “stacked” on top of the old values and when you want to pop something, you always take out the last item that was added. This is called LIFO which stands for Last In First Out.

So if we now apply this knowledge and start thinking about how the evaluation stack works and why it’s important you might see that operations such as OpCodes.Mul expects there to be two values that it can pop from the stack then when it’s popped them, the values are multiplied and the result is pushed back onto the evaluation stack for the caller to use.

This is what it looks like when we want to multiply 10 by 20

First the value 10 is pushed onto the stack

Then the value 20 is pushed onto the stack

Then we call OpCodes.Mul and it starts off by popping the first value

The it pops the second value

After the multiplication is done, the result is pushed onto the stack

This is the general pattern used, things are pushed onto the evaluation stack and then operations pops the values they want to use and then pushes a possible result back onto the evaluation stack. So how do we proceed now?

We already got the first value on the evaluation stack, let’s say that whatever we pass into our dynamic method will be multiplied by 10, since we did OpCodes.Ldarg_0 we only need to emit code that adds our next value!

This is done exactly as we did in the last post:

il.Emit(OpCodes.Ldc_I4, 10);

If the parameter sent to the dynamic method is 20 now, the stack will look like this:

Now let’s do the multiplication, this is done by emitting OpCodes.Mul:

il.Emit(OpCodes.Mul);

So what now? We want to pass the result from the multiplication operation to the PrintWithSpecificFormat method and how do we pass values again?

By pushing them to the evaluation stack!, but wait a minute, the result is already on the evaluation stack so we don’t need to do anything! We can just call the method as we like. The complete IL-emitting looks like this:

il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Ldc_I4, 10);
il.Emit(OpCodes.Mul);
il.Emit(OpCodes.Call, methods.First(x => x.Name == "PrintWithSpecificFormat"));
il.Emit(OpCodes.Ret);

Now the last thing we are going to do is creating the delegate and invoking it, we can use Action<T> as we did in the last post and then we just invoke the created delegate:

var toInvoke = (Action<int>)mathOperation.CreateDelegate(typeof(Action<int>));
toInvoke(10);

The result should look like this:

I hope you found this interesting and if you have any thoughts please leave a comment below!

Vote on HN

Calling a non-dynamic method with parameters from a dynamic method

Posted by Filip Ekberg on October 14 2011 Leave a Comment

I think it’s time that we explore some more OpCodes and in this post I will look at how we can call methods with parameters passed to them!

There are two OpCodes that we can use to execute a method these are:

The difference between these two are important, the first one OpCodes.Call, calls a method and expects that we are coming back after the OpCodes.Ret in the called method. Whereas OpCodes.Jmp exit the context.

Also the Jmp operation has a set of requirements ( from MSDN ):

  • There are no stack transition behaviors for this instruction.
  • The jmp (jump) instruction transfers control to the method specified by method, which is a metadata token for a method reference. The current arguments are transferred to the destination method.
  • The evaluation stack must be empty when this instruction is executed. The calling convention, number and type of arguments at the destination address must match that of the current method.
  • The OpCodes.Jmp instruction cannot be used to transferred control out of a try, filter, catch, or finally block.

So by the looks of it, we want to use OpCodes.Call! The method that I want to call is quite simple, it looks like this:

public static void Test(int a)
{
    Console.WriteLine("Test: {0}", a);
}

The method takes a parameter and prints “Test: {The Value of a goes here}”, so if we would call Test(10) we would get this printed in the console:

Test: 10

We’ve seen how we can read arguments with operation codes, but how do we actually emit and operation code with a certain value, the value that we want to pass to the method? There’s an operation code called Ldc_I4 and it basically means the following:

Pushes a supplied value of type int32 onto the evaluation stack as an int32.

This is exactly what we are looking for! So we got the operation that we want to emit in order to solve the argument issue, but how do we actually call our method? It turns out that the operation OpCodes.Call expects a parameter passed to the emit method and in this case it’s expecting information about the method that we want to call. And how do we pass information about the method?

By using MethodInfo of course!

Let’s first of all compose the DynamicMethodbefore we start emitting IL, it will look somewhat like this:

var someMethod = new DynamicMethod(
        "SomeMethod",
        typeof(void),
        null,
        typeof(Program).Module);

The third variable is null because I don’t want any parameters sent to my dynamic method at the time being! Okay so we need to do two more things before we actually start emitting the IL, this includes getting the actual IL generator and all methods in our current class:

ILGenerator il = someMethod.GetILGenerator();
var methodToCall = typeof(Program).GetMethods();

Now let’s start emitting IL!

The first thing that I want to emit is the argument that I will be passing to the method and this will look like this:

il.Emit(OpCodes.Ldc_I4, 123);

So I am saying that I want to push a 32bit integer and the value of the integer is 123. Next up we’ll be doing the actual call and for this I need to get the method information from the above methods collection and then I will just return from the method:

il.Emit(OpCodes.Call, methodToCall.First(x => x.Name == "Test"));
il.Emit(OpCodes.Ret);

Now everything is set, we have our dynamic method ready and all we need to do know is to create the delegate, but we don’t have a deleaget that we’ve defined? No worries! We can use Action:

var toInvoke = (Action)someMethod.CreateDelegate(typeof(Action));

And now we can call this just like we normally do:

toInvoke();

The result will look like this:

I hope you found this interesting and if you have any thoughts please leave a comment below!

Vote on HN

How Dynamic Methods affect resources like memory and disk space

Posted by Filip Ekberg on October 13 2011 Leave a Comment

We’ve seen how we can create static methods at runtime and one of the benefits that I’ve been talking about is that the methods will be disposed when they don’t have a reference anymore. This can be crucial if you work in an environment where memory resources are limited. So what I will do is I will allocate a couple of million methods and see what happens with my memory usage!

To do this I’ve changed a couple of things in MyMethodCreator so that I can get a list of math operations that I want to run, this is the entire class that I will be using:

public class MyMethodCreator
{
    public ICollection<MathInvoker> MathOperations { get; private set; }

    public MathInvoker CreateInvoker(OpCode operation)
    {
        Type[] methodArguments = {
            typeof(int),
            typeof(int)
        };

        DynamicMethod mathOperation = new DynamicMethod(
                "MathOperation",
                typeof(double),
                methodArguments,
                typeof(MyMethodCreator).Module);

        ILGenerator il = mathOperation.GetILGenerator();
        il.Emit(OpCodes.Ldarg_0);
        il.Emit(OpCodes.Ldarg_1);
        il.Emit(operation);
        il.Emit(OpCodes.Ret);

        var divideIt = (MathInvoker)mathOperation.CreateDelegate(typeof(MathInvoker));

        return divideIt;
    }
    public IEnumerable<MathInvoker> GenerateOperations(int count)
    {

        MathOperations = new List<MathInvoker>();
        MathOperations.Add(CreateInvoker(OpCodes.Div));

        for (var i = 0; i < count; i++)
            MathOperations.Add(CreateInvoker(OpCodes.Mul));

        return MathOperations;
    }

}

As you can see, all that I am doing is replacing the operation code for what type of math operation that I will be running, then I create this dynamic method and add it to a list inside my class. The idea here is that I want to see if I get almost all memory back when disposing the list of methods, this is interesting because we will most certainly notice a difference in initial / end size in memory if we compare it to a statically written application where all the millions of methods would be defined.

We could of course refactor the above a little bit in order to reduce the overhead when creating the dynamic methods, but let’s not bother with that at the moment. I’ve been doing some tests here to try these two scenarios:

1. I have an application that dynamically creates ~35 000 methods attached as an instance method. The only thing know at runtime is the method that creates this dynamically.

2. I have an application that has ~35 000 methods defined in a class, like any normal method would be. Everything is known at runtime.

Then the common part is that in both cases, I’ve added ~35 000 delegates to a list and then disposed that list, to see what happens with the data. Initially I expected that when I dynamically create ~35 000 methods I would get almost all the memory back once I dispose it and that this would be false for the second scenario above.

This is how I did the testing to see how much memory was used:

Console.WriteLine(GC.GetTotalMemory(true).ToString());

This will print the total amount of allocated bytes, the parameter just states that it’s OK for us to wait for the GC to do a correct calculation. My test for the first scenario was produced like this:

1. Create an instance of MyMethodCreator
2. Print the allocated bytes
3. Create the dynamic methods
4. Print the allocated bytes
5. Clear the list of delegates and Dispose MyMethodCreator
6. Call GC.Collect() and GC.WaitForPendingFinalizers()
7. Print the allocated bytes
8. Re-do step 1 to 7 once more

Before we go any further, I just need to say that you should’t ask the garbage collector to collect manually, it will solve this best by itself!

So the test for the second scenario was similar except that instead of doing step 3 above, I just added each method from my MyMethodCreator, since in the “static method” scenario ( scenario #2 above ) had all the method as instance variables pre-defined, they were not dynamically added to the class. So I just added these as delegate types to a list, just as the “create dynamic method”-step did above.

The code for the above could look somewhat like this, for scenario #1:

for (int i = 0; i < 2; i++)
{
    var methodCreator = new MyMethodCreator(OpCodes.Mul);
    Console.WriteLine(GC.GetTotalMemory(true).ToString());

    methodCreator.GenerateOperations(32768);

    Console.WriteLine(GC.GetTotalMemory(true).ToString());
   
    methodCreator.Operations.Clear();
    methodCreator = null;

    GC.Collect();
    GC.WaitForPendingFinalizers();

    Console.WriteLine(GC.GetTotalMemory(true).ToString());
}

The result from running this for me looked like this:

So I got the following values:

101120
1414064
365484
103140
1414064
365484

And when running scenario #2, I got the following values:

21344
1208416
159868
28728
1208416
159868

As you can see the two are pretty similar, the difference is that they have different overhead, this is most likely because the first scenario uses Dynamic Method that takes a little more resources. As I said in the beginning of this post, we could refactor MyMethodCreator a little bit, but would still be above scenario #2.

The important thing here isn’t really that they differ a little bit, but to see that there is a pattern, the methods are disposed as we imagine they would be.

But what about the resources I want to spare? By the looks of it, you won’t save any memory resources by using the dynamic method approach, at least not in the way I did it. If this scales to a lot of different and larger operations where the IL-commited is a lot more, the result might be a bit different.

However, there is one thing that differs a lot and that’s the manageability and size of the actual binary. First of all, the second scenario has ~35 000 methods in it, it’s impossible to maintain, who would ever write 35 000 methods? Probably no-one, but that’s beside the point. Having one method that helps us create these instead is Much easier on the eyes and more maintainable.

The second thing is the binary size, now, these binaries are already pretty small, but the difference between them is pretty huge in my opinion:

  • Dynamically created methods binary: 7KB
  • Staically created methods binary: 3MB

Seeing that both these are very small numbers might not raise an eyebrow but how about when I say that the binary size increased with 43886%? Might still not be a problem, but consider that you are programming a micro-controller where you have limited amount of flash-memory to install the software on, the system supports .NET and emitting code with reflection, you have more RAM than you have flash-memory. Then a variant like this could be useful to know about. I personally hoped to see that it would take up a lot less memory than having compiled before runtime.

I hope you found this interesting and if you have any thoughts please leave a comment below!

Vote on HN

Getting information about a method and its local variables

Posted by Filip Ekberg on October 12 2011 Leave a Comment

Sometimes it can be useful to get information about a method such as getting all the local variables. We’ve looked at how we can retrieve information about known types and setting / getting their values. Now let’s take a look at how we can get information about a method and it’s local variables in runtime!

I am going to use almost the same code as in the last example, except that I’ve changed it a little bit. I refactored everything out to a separate class to make it easier to follow me along the way, the class that creates my dynamic method now looks like this:

public delegate double DivideInvoker(int a, int b);
public class MyMethodCreator
{
   
    public DivideInvoker CreateInvoker()
    {
        Type[] methodArguments = {
                    typeof(int),
                    typeof(int)
                };

        DynamicMethod division = new DynamicMethod(
                "Division",
                typeof(double),
                methodArguments,
                typeof(MyMethodCreator));

        ILGenerator il = division.GetILGenerator();
        il.Emit(OpCodes.Ldarg_0);
        il.Emit(OpCodes.Ldarg_1);
        il.Emit(OpCodes.Div);
        il.Emit(OpCodes.Ret);

        var divideIt = (DivideInvoker)division.CreateDelegate(typeof(DivideInvoker));

        return divideIt;
    }
}

If you want to know more about what it does, be sure to check out my previous blog post! It’s a bit easier to create my dynamic method now and invoke it:

var methodCreator = new MyMethodCreator();
DivideInvoker divideIt = methodCreator.CreateInvoker();

double result = divideIt(4, 0);

Now we’ve come to the interesting part, I want to get information about the local variables inside my method that actually runs the above code. What I expect to get is information about my MyMethodCreator instance, my dynamic method and my result variable. All the information regarding this is stored inside something called a StackFrame, more commonly known as a call stack.

Let’s briefly take a look at what Wikipedia has to say about what a call stack is:

In computer science, a call stack is a stack data structure that stores information about the active subroutines of a computer program. This kind of stack is also known as an execution stack, control stack, run-time stack, or machine stack, and is often shortened to just “the stack”. Although maintenance of the call stack is important for the proper functioning of most software, the details are normally hidden and automatic in high-level programming languages.

This means that we can get information from the stack frame! Worth knowing is that there’s one StackFrame for each method, just as the above code says, each subroutine(method) has it’s own call stack! So we don’t just want to get the current call stack/stack frame, we want to get all of them!

In order to do this, we need to create an instance of a StackTrace object and then call it’s member function GetFrames(), this will give us all the frames that the stack trace can find.

So what I’ve done here is that I’ve created a method that I want to use to dump the all the stack frames and all information about the local variables.

void DumpStackFrames()
{
}

And the first thing that I am going to do is to create a new stack trace and fetch all the frames so I can iterate over them:

StackTrace trace = new StackTrace();
foreach (StackFrame frame in trace.GetFrames())
{
}

You can get some more information out of the frame-instance but I am just interested in getting the actual method, so in order to do that I call GetMethod() which will give me a MethodBase, which is a parent to MethodInfo:

MethodBase method = frame.GetMethod();

Now we’ve got information about the method itself, what we want now is the actual body of the method where all the goodies are, this is fairly straight forward, there’s a GetMethodBody() function we can use on the method-instance! Let’s print out the method name as well to the console so that we will see exactly what we’re dealing with. This is what we’ve come up with so far:

void DumpStackFrames()
{
    StackTrace trace = new StackTrace();
    foreach (StackFrame frame in trace.GetFrames())
    {
        MethodBase method = frame.GetMethod();
        MethodBody body = method.GetMethodBody();
        Console.WriteLine("Method: {0}", method.Name);
    }
}

Now when we’ve got the body for each stack frames method, we can ask for all the local variables, these are retrieved through the collection LocalVariables and since we want to iterate over them we can do it like this:

foreach (LocalVariableInfo variableInfo in body.LocalVariables)
{
}

LocalVariableInfo will give us all information that we need about the variable, we can among a lot of other things:

Both of the above is interesting to us, we want to print out the variable type and then go through all the properties on the variable and print the names of those as well.

We start off by printing the variable name like this:

Console.WriteLine("\tVariable: {0}", variableInfo.ToString());

In order for us to get all the properties, we need to access that through the LocalType variable which will give us a Type instance:

foreach (PropertyInfo property in variableInfo.LocalType.GetProperties())
{
    Console.WriteLine("\t\tProperty: {0}", property.Name);
}

As you can see we just use the property information as we’ve seen in earlier blog posts. So what will the result look like? The entire method looks like this:

void DumpStackFrames()
{
    StackTrace trace = new StackTrace();
    foreach (StackFrame frame in trace.GetFrames())
    {
        MethodBase method = frame.GetMethod();
        MethodBody body = method.GetMethodBody();
        Console.WriteLine("Method: {0}", method.Name);
        foreach (LocalVariableInfo variableInfo in body.LocalVariables)
        {
            Console.WriteLine("\tVariable: {0}", variableInfo.ToString());
            foreach (PropertyInfo property in variableInfo.LocalType.GetProperties())
            {
                Console.WriteLine("\t\tProperty: {0}", property.Name);
            }
        }
        Console.WriteLine();
    }
}

And when I run it I will get the following information ( more text after the picture! ):

As you can see the dynamic method has a pretty odd type as opposed to what it would look like if we used a lambda expression like this instead:

DivideInvoker divideIt = (a,b) => a / b;

This would output the following:

Variable: Reflection.DivideInvoker (1)
        Property: Method
        Property: Target

Compared to:

Variable: Reflection.MyMethodCreator+DivideInvoker (1)
        Property: Method
        Property: Target

As you can see it’s a bit different when you use a dynamic method, as you might expect as well! By getting the local variable information and fetching the type, you can explore the dynamic method even more.

I hope you found this interesting and if you have any thoughts please leave a comment below!

Vote on HN