C# 3.0: a revolution in productivity

Miklos_Hollender

One of the most important new features in C# 3.0 is the concept of
lambda expressions, because they make functional programming
possible. This will mean a revolution of productivity in industrial/business programming!

A practical example: from interfaces to lambda expressions

Let's say we got a StringArrayWriter class that has a Write
method, that takes an array of strings as a parameter and writes
it on the screen. Each time it's called, it also writes a log file
like "678 lines printed at 2006.06.23 14:12:34", what we want to
analyze later. We will call all other objects that call this
method "client code", because they are "clients" of this class.
Soon we find out that in some cases, we don't want to print empty
lines. In some other cases, we don't want to print those lines
that contain only whitespaces, and in some other cases, we don't
want to print those lines that start with a comment: "//".

How can we implement that?

1. We can pass them as boolean parameters to the Write method,
f.e. StringArrayWriter.Write(strSomething,false,true,true) - but
such a solution can quickly grow to be big, hard to maintain and
hard to use.

2. The client code can iterate through the array of strings it
wants to pass and filter out those elements it does not want to
print, and pass only the filtered array. This is highly
inefficient. Inefficient from a performace viewpoint: if we got an
array of 100.000 elements, why should we loop trough that twice?
Besides, it's inefficient from programmer productivity viewpoint:
the one of the reasons of creating this StringArrayWriter class
was to abstract away looping through that array from the client
code, therefore to make client code shorter and more readable.

3. We can give up this approach and let the client code loop

through his array, print it and write the log. It's highly
inefficient from a programmer productivity viewpoint: we will
produce long, bloated code which is hard to maintain. The whole
point of structured, OO, or functional programming would be to
abstract away often used blocks of code, and now we just given up
on that.

4. Then another idea can occur: what if we let the client code
make the decision? Let's say we state that each client class must
implement and IWhatToPrintDecider interface, which means they must
define a DecideWhatToPrint method, which takes a string as a
method and returns true if it should be printed, false if not.
Then the client class would just pass a reference to itself to the
Write method: StringArrayWriter.Write(strSomething,this), and the
StringArrayWriter will loop through the array, call the
DecideWhatToPrint of the client object on each element of the
array, print those strings where it returns true, and when
finished, write the log accordingly. A great idea? It is. But...
using interfaces is quite a burder: if you would systematically
use this approach for each decision, each class of yours would
have to implement dozens of interfaces. Therefore if you have only
interfaces, and not lambda expressions, you will only use them
rarely, and in most other cases, you will use one of the
above-mentioned ideas. So, while it's a great idea to keep as much
decision-making in client code as possible and methods should
generally avoid making decisions on parameters they get, unless
it's absolutely necessary, interfaces are not the best way to
implement it.

But wait! Instead of defining a method and writing an interface
"contract" between the StringArrayWriter class and the client
class, couldn't we just pass that method - or rather a function -
as a parameter, just like any kind of data? It would be a lot
easier!

Now, exactly these are lambda expressions.
If the Write method expects a string and a lambds expression as a parameter (with the Func keyword), then the clien code can define
a lambda expression which returns false on empty strings and true
in every other cases, we could write it as:

(String s) => {return s.Length>0}

Note that the type definition and the return statement is
optional:

s => s.Length>0

and this kind of lambda expression works on any object that has a
Length property, not only strings, which allows an even higher
level of abstraction.

With such a simple syntax, we can keep most of the decisions in
the client code. Why is it so important? Because it's probably the
only way to keep large systems easy to maintain, change, extend and refactor

Theoretical background of functional programming

Functional programming is more than a nice trick: it's based on a
completely different paradigm of computation: lambda calculus.
Most programming languages we use are imperative in nature: do
this, do that, do another thing. This concept is based on the
Universal Turing Machine, which is the most popular concept of
computation. But it's not the most efficient, and not the most
elegant nor the easiest way to progra. The reason the imperative concepts of the Universal Turing Machine became widespread is that Alan Turing
purposefully defined his theory to be as close to a real machine
as possible, therefore his theory was easy to map to low-level CPU
instructions, and therefore such programs ran more efficiently on
early low-performace computers.

Lambda calculus, which is a completely different theory of
computation, was invented by Alonzo Church in the thirties, before
the age of computers. Lambda calculus is a formal mathematical way
of computation by defining functions that can take other functions
as parameters and can return other functions constructed "on the
fly". It's proven to be complete - can compute everything that is
computable - and it's smaller and more elegant than the Universal
Turing Machine, therefore makes programmers more productive. The
only reason it did not become popular in the early days was simply
because it was hard to map to CPU instructions, therefore programs
would run slow on the early computers.

A typical lambda function looks like (λ x. x + 2): it takes a
variable as a parameter, and returns that variable +2. In C# 3.0
it would be x => x +2 or (int x) => {return x +2} - the same idea, with a slightly different syntax.

Lambda calculus made it's way into real computer programming where
John McCarthy based his LISP programming language largely on it.
LISP was an amazingly productive programming language, used mainly
for AI research - the hardest programming task ever. It was also
LISP that helped Paul Graham and Robert Morris develop Viaweb,
which was sold to Yahoo! for $40M and became Yahoo! Store. Paul
wrote that the amazing flexibility of LISP made them so productive
that whenever a competitor announced a new feature, by the time
journalists asked them about that next day, they also developed
that feature.

And the famous MapReduce algorithm of Google comes from functional programming as well: to "map" generally means evaluating a lambda function for each member of a collection and "reduce" generally means "filter out those elements from an array/collection for which a lambda function returns false" - this is what we did in the above example.

These examples show that functional programming can mean
a huge performance boost not only in academic areas, but in
mainstream, industrial, business programming as well.

DenSter

That sounds a bit like the strong typed new query namespaces in LINQ. I am sure you've read about those. Check out Channel 9, there are videos that show that type of programming. Really cool, so you don't have to send in a SQL query as a text string and hope that it will return the right data. Instead you program the query with strong types, and because you're connected to the database while you are developing, it even knows the field types at design time. They are really making big improvements huh

kine

It remembers me my exam from Lambda calcul on university... uf, it was hard to understand the Lambda calcul...

Miklos_Hollender

Kine,

yeah, because in the 'varsities it is taught in a too mathematical way. But in reality it's just a powerful tool of code reuse. For example, let's say we want to integrate MSCRM with Navision and we write a C# method that can generate an INSERT-type XML for NIF. This method can take a lambda function as a parameter that on each call provides data to be inserted in a fieldname=>fieldvalue hashtable and then the method produces the XML.
Then we can write another method that generates a lambda function that yields, say, a Contact record within given filters on each call, and creates this hashtable.

Miklos_Hollender

DenSter,

I hope you only joked when you mentioned reading a database through SQL-strings from C# and reinventing every well-known wheel from connection pooling to security? ...

There is IdeaBlade DevForce, the Express version is actually for free (and it alone can do much), which gives everything from an Object-Relational Mapper to pooling, security or a sane way to map form fields to database tables. Check out the Concept Manual (ZIP)

DenSter

Check out this link and watch the movie. This will probably make a lot more sense than me trying to explain this stuff .

Miklos_Hollender

Formet Excel project manager Joel Spolsky describes in a nice article while lambda calculus is so important:

http://www.joelonsoftware.com/items/2006/08/01.html