Welcome to this week’s Friday with Filip!
There’s a saying that goes like this:
Use the right tool for the right job
Even if you find the right tool for your job, do you know how to use the tool correctly?
This is exactly what I want to talk about this Friday; you need to know your tools in order for you to know it’s the right tool for this job!
Do you know your tooling?
All too often I see developers using amazing new tools but in many cases not knowing the limits or how to use it correctly. It’s very important to understand how to use a tool and what the implications of using a tool in a certain way has. When I find a new tool or a new library that I want to use, I try to research it and understand it before I recommend it for production use.
To me a tool is not only a software that I run or a hammer that I use, but I also consider a third party library a tool. A tool for me is something that I use to solve a certain problem and smaller parts can build a larger tool.
Problems with not knowing your tools
If you don’t know what your tool does behind the scenes you can get very big problems in the end. If you try to use a hammer to screw something into the wall there might be an easy work around just to hammer the screw into the wall. But what if it was the other way around?
Imagine that you had a screwdriver but a nail that you had to put into a plank; getting that into the plank would be much harder, no?
It’s still possible of course to solve the problem by just using the other end of the screwdriver and hammer the nail slowly into the plank.
So where am I going with this?
If you use a library or software that is new and hype which solves a certain problem but makes it harder for you; maybe you’re doing it wrong! The tooling might be great for solving one of the problems, but the way that you use it just gives you too much of a headache. If you go back and do some research on the tools, you might find that you are using it wrong or that the tool simply is not for the specific use case.
A good example here is LINQ to EF. I’ve seen many developers knowing how to use LINQ but not knowing what happens behind the scenes. This can be very dangerous for the performance of your application.
One other problem that arises with this is that most of us test our applications with very little data in it, but in a real world application after some time there might be a lot more data to process which was not considered at the beginning.
Now to be a bit more concrete, see of the following LINQ to EF query:
var persons = from person in db.Persons
where person.Name.Contains("Filip")
select person;
This can of course be written like this as well:
var persons = db.Persons.Where(person => person.Name.Contains("Filip");
Both of these will be executed as soon as you request the result by doing persons.ToList(); for instance.
This all looks very good, it might be exactly what we want to do as well. But what happens if we have 1 billion persons in our database?
The SQL query that is generated from this will perform what is called a row search, which means it does not use any indexing which means it will have to go over 1 billion persons and do a search within each person’s name.
Consider that it generated the following SQL:
SELECT * FROM Persons WHERE Name LIKE '%Filip%';
Of course we can optimize this by using full text search! But that is not the point, the point is that we might not have considered that Contains() will actually be a slow operation to run. If we knew that this would skip all indexes we might had chosen to use StartsWith() instead.
Another problem with not knowing how LINQ works internally is that all too often I see developers doing ToList() just because they want to use the types in .NET instead of having it translated to SQL. If they had known of EntityFunctions or SqlFunctions they might had chosen to run the query on the database side instead of doing LINQ to Object!
Because if we perform the query below, we will actually fetch the 1 billion rows from the database and have them in memory and then perform the search in the application instead.
db.Persons.ToList().Where(person => person.Name.StartsWith("Filip").ToList();
Again, if we know our tooling and use the right tool for the right job, we are going to do a much better job.
Do you know your tools?
Vote on HN
After attending Scandinavian Developer Conference in Sweden 2010 and attending the talk from Roy Osherove ( http://osherove.com/ ) Test Driven Development ( TDD ) has been something that I have tried to focus a bit more on.
Roy talks about some really important aspects of programming that should be printed into the programmers brain. I for instance is one of these people that like to verify everything that I do; Did I Really lock the door? So when I’ve turned the keys in the lock, I verify that the locking mechanizm worked and that the door is no indeed locked.
So what does my weird habbit have to do with TDD?
Driving the development with tests is not the only part of TDD; as with anything else there are a lot of people that like to think a lot of different things about this method. I like to think of tests as a way to Verify that my code runs as expected.
This is not a case of miss-trust!
Don’t missunderstand the above as see me as someone that does’t trust anyone or anything created by others than myself, that’s not the case. But by verifying that things runs properly, the door is properly locked I know that everything is OK from my part in the scenario.
What if I didn’t run the tests on the code and handed it over to the next developer that Did not in fact know that the method he works in affects everything else? And what if I didn’t verify the lock; I was in a big hurry and someone just went into the appartment and stole my TV?
In my opinion, test driven development is way to:
- Verify that Code runs as it should
- Decrease mistakes in the future
- Help others gain knowledge of parts in the code faster
To take the locking mechanizm-checking even further and really turn it into something that is Driven you can follow these three very simple steps that Roy also talks about in his speeches:
- Make it fail
- Make it work
- Make it better
How do you make the lock fail to lock? Well you simply Don’t lock it!
So let’s head over to the development aspect of this, you want to make your code fail? No code means it will fail right? When I do this step 1 and 2 always go together, especially when I demo TDD for co-workers or others. I will use the same good example as Roy used in his demo on SDC 2010, let’s try to put our heads around a simple Calculator project.
Defining the requirements
For this “project” we only have fourrequirements:
- A method called Sum
- This method parses a String
- This method should return String.Empty when you pass Invalid or Empty data
- Adds two integers in the String together “1 1″ would be 2
So if we follow the above we would end up with a test that looks like the following:
[TestMethod]
public void Sum_EmtpyString_ReturnsZero()
{
var calculator = Calculator.GetInstance();
var actual = calculator.Sum("");
var expected = string.Empty();
Assert.AreEqual(expected, actual);
}
Since we haven’t really implemented Calculator, GetInstance or Sum yet, we made it fail, there is no Code. Visual Studio helps us out a bit and makes the process a bit faster, by simply pressing alt + enter when selecting Calculator or Sum we can select to create the Class and create the Method Stub.
So we actually finished both Step 1 and Step 2 at the same time, even though Step 2 isn’t complete yet, we are nearly there!
Now we need to make it work, what is the simpliest way possible to make it work for the above test-case?
public string Sum(string input)
{
return string.Emtpy();
}
This will actually work and it will indeed check this test as Passed. But it’s not really what we want. So how do we proceed?
Another Great point Roy pointed out is that, if there isn’t a test, there’s no code and if there’s code there’s gotta be a test! And if one test passes and you want to Change the code to behave different with other input; You need to prove it with a test!
So we can test if
will return 1, which we from looking at the code will see that it wont!
public void Sum_ValueContainsADigit_ReturnTheDigitThatIsPassed()
{
var calculator = Calculator.GetInstance();
var actual = calculator.Sum("1");
var expected = "1";
Assert.AreEqaul(expected, actual);
}
This test will most defently fail because of our implementation! So we need to go back to Sum and change this method.
Once all tests passes and the test case is approved, we head on to the next step, which is make it better! In other wors we need to refactor some code! Try to follow these steps to refactor ( i use ReSharper, which is a Great tool! ).
- Move the Library to an appropriet Class Library / Project
- Optimize the code
- Refactor your test to look better
- Comment the code
So you got a little peek at what TDD is and how it can be used, if everyone would use TDD, it would be so much easier to take on new projects that have already been started. You never know what your changes will impact on.
Vote on HN
Once again, there was a little mistake in the last post I posted here which clearly didn’t effect the result that much. But it is still worth mentioning again. The ^ was not meant to be XOR, I was clearly thinking of Math.Pow.
In the last post I didn’t spend to much time talking about the Parallel Extensions for LINQ which are Really great and helpful; If you just love to replace traditional loops with LINQ-expressions you will probably find this post somewhat amusing.
So let’s dig down to the coding shall we!
First of all I have set up a smaller list with numbers, in this scenario we will just assume that we have a list of something and depending on the contents of that list, the time it takes to perform an action on that result, will differ. So here’s my list
var latency
= new[] {1,
2,
4,
8};Side note: new[]{} is really helpful, especially when you are creating examples like this!
And then I have a method that I want to run for each element in my list, which I will call PerformLogic
static int PerformLogic(int latency)
{
var ms = 500 * latency;
Thread.Sleep(ms);
return ms;
}
In “traditional” programming, you would maybe do something like this:
for ( int i = 0; i < latency.Lenght; i++ )
{
PerformAction(i);
}
But I don’t find that so amusing anymore, using LINQ is so much neater so instead of that for-loop we can actually do this:
(from i in latency select PerformAction(i)).ToList();
We don’t really have to write the expession like this though, since LINQ + .NET 4.0 is so smart, we can Refactor this to look somewhat like this:
(latency.Select(PerformLogic)).ToList();
The previous one is a bit more elaborate but this is fine aswell.
Making it parallel
We’ve reached the place where we no longer can refactor our code to make it faster, we can’t replace anything in the logic to make everything faster; We need to parallelize it!
Let’s have a look at what LINQ provides us with, oh, there’s an method called
.
All I did now was changing the above code to this:
var result = (latency.Select(PerformLogic)).AsParallel().ToList();
And we have a parallelized query. For those with a fast mind can see that it will take about 7,5 seconds to run this since each of the “latency”-points will take itself times ½ second.
My final test-code looks like this
var latency
= new[] {1,
2,
4,
8};
var start
= DateTime
.Now;
var result
= (latency
.Select(PerformLogic
)).AsParallel().ToList();
var end
= DateTime
.Now;
Parallel
.ForEach(result, Console
.WriteLine);
Console
.WriteLine("Execution time: {0}",
(end
- start
));And this is the output
Vote on HN
As .NET 4.0 will be released in a couple of weeks and the RC has been out for a while. It’s about time that I write something about the new helpful features of .NET 4.0. One of these helpful things are the Parallel Extensions and Parallel helpers that allowes you to do parallel programming.
Parallel computing is a form of computation in which many calculations are carried out simultaneously
In this example I will be using a WebRequestPool which just helps me out a bit to carry on this example. You might think if it like this: You have different request types which takes different long to execute you might be doing some WebDAV uploading, Image Fetching and other Over-The-Web-Access which takes time. Instead of waiting for each request to stop, you might as well run them simultaneously.
internal delegate void WebRequest(int ms);
internal class WebRequestPool
{
public List Requests { get; set; }
}
So to make it easy I just have a simple Delegate which will allow us to Run/Invoke our WebMethods which all takes some input parameter that will, in some way, make the requests take longer / shorter.
Inside of the WebRequest class all we have is a Requests Pool, a simple list of delegates.
To make the whole a little bit interesting we have three different types of Requests: Standard Request, Long Request and ExtremeRequest
static void StandardRequest(int ms)
{
Thread.Sleep(ms);
}
static void LongRequest(int ms)
{
Thread.Sleep(ms^2);
}
static void ExtremeRequest(int ms)
{
Thread.Sleep(ms^10);
}
So now we have three different types of methods that all validate with our delegate! Let’s head on and fire up this pool
var pool
= new WebRequestPool
{
Requests
=
new List
{
StandardRequest,
LongRequest,
ExtremeRequest,
StandardRequest,
LongRequest,
ExtremeRequest
}
};This actually demonstrates something new too, the object initializers. So the list now contains six requests, All we want to do now is Processes these.
First of all we want to do it like we did in the old days, so I’ve created a method called ProcessPool which looks like this
private static void ProcessPool(WebRequestPool pool)
{
var start = DateTime.Now;
foreach (var item in pool.Requests)
item.Invoke(2000);
var end = DateTime.Now;
var span = end - start;
Console.WriteLine(
string.Format("Execution time: {0}", span));
}
Of we run this the output is
Now, that’s a bit too slow for me, so Let’s Parallelize that! All i’ve done now is to creat a new method called ProcessPoolAsParallel which takes the same input and expects to give the same result. There’s a little bit difference though, the foreach is now replaced with the
method.
private static void ProcessPoolAsParallel(WebRequestPool pool)
{
var start = DateTime.Now;
Parallel.ForEach(pool.Requests,
item => item.Invoke(2000));
var end = DateTime.Now;
var span = end - start;
Console.WriteLine(
string.Format("Execution time: {0}", span));
}
So if we run this now the result is:
So this increased significally!
This is just a small example of the power in using Parallel programming patterns. Look into the Task class to head on!
Edit
There’s a slightly miss-used method in the code above, the ^ in C# is XOR and I was thinking of the Math.Pow, however, this doesn’t really matter in this case since the result is still Parallel = Faster.
Vote on HN
Filip Ekberg is a Senior Software Engineer working in the country with all the polar bears, author of a self-published C# programming book and overall in love with programming.
