1. Use the NullObject pattern, so that the null pointer instead points at a null-behaving object.
2. Gift all NullObjects with an identification number.
3. Place an assertion that stops the debugger at the point when a NullObject with the same Id as the one which originally caused the NullPointerException is constructed.

That first part will likely be the trickiest to pull off. In C++ there are several issues to consider. Is the null pointer of a user-created type (Thing*) or built-in (int*)? Is it easy to introduce a constructor which automatically changes Thing *t = null; into Thing *t = new NullThing()? Can this coercion also be done for subclasses (because a null subclass pointer can be type-converted to a base class pointer before the dereference)? Is it worth trying to change all Thing pointers to Thing references, just to make sure you catch everything?

One final issue: this technique will only work for deterministic code. Otherwise no guarantee can be made on the issuing of Id’s within the NullObject constructor (easiest done with a global incrementing counter).

#include <unistd.h>
#include <iostream>
#include <string>
 
void printProgBar(int percent)
{
    std::string bar;
 
    for (int i = 0; i < 50; i++) {
        if (i < (percent/2)) {
            bar.replace(i, 1, "=");
        } else if (i == (percent/2)) {
            bar.replace(i, 1, "D");
        } else {
            bar.replace(i, 1, " ");
        }
    }
 
    if (percent < 100) {
        std::cout << "\r" "8" << bar << "@ ";
        std::cout.width( 3 );
        std::cout << percent << "%     " << std::flush;
    } else {
        std::cout << "\r" "8" << bar;
        std::cout << "D~~~~ ~~~ ~" << std::flush;
    }
}
 
int main()
{
    for (int i=0; i<=95; i+=10) {
        sleep(1);
        printProgBar(i);
        std::cout << std::endl;
    }
    for (int i=95; i<=100; ++i) {
        sleep(1);
        printProgBar(i);
        std::cout << std::endl;
    }
    std::cout << std::endl;
}

8D                                                 @   0%
8=====D                                            @  10%
8==========D                                       @  20%
8===============D                                  @  30%
8====================D                             @  40%
8=========================D                        @  50%
8==============================D                   @  60%
8===================================D              @  70%
8========================================D         @  80%
8=============================================D    @  90%
8===============================================D  @  95%
8================================================D @  96%
8================================================D @  97%
8=================================================D@  98%
8=================================================D@  99%
8==================================================D~~~~ ~~~ ~

Prior to the midterm, in Project 2, we had done an exercise writing a in-order iterator over a binary search tree. This exercise had two important components: (1) The iterator allows pausing between elements. So, it must have a way to remember where it left off the last time an element was returned. (2) The binary search tree could be storing any type of data. So, the entire project had to satisify the typing constraints of Java’s generics system.

Then, in the following week (I told you the class moves quick), We used Haskell in Project 3 to cover the concept of Higher Order Functions (HOF). Now, most of my students have had their minds trained on Java, which doesn’t provide first class functions. So, the concept that a function might take other functions as arguments or return a function as a result is really foreign.

Since my students were clamoring for some extra credit, I tried to think up something that might tie together what has been done so far. My predecessor, Alex Thornton, had done a Higher Order Function implementation in C# as a fun and interesting demo for his last lecture. Now, I don’t know C#, but I’m pretty sure that I could do the same in Java. It turns out you can create for yourself a java Framework that provides this higher order functions, with a bit of work. Although, as we shall see, it’s verbose and clunky.

Let’s begin by defining what a function looks like. Since Java doesn’t support first-class functions, we start by defining a Function Object.

public interface IFunction<A, B> {
	B call(A arg);
}

Basically, any object that wishes to act as a function from A → B, implements the IFunction interface, so that it has a call method.

Next, I try and think of some task that would use some HOFs. So, let’s first create a list of Strings, and print them out.

public static void main(String[] args)
{
    ArrayList<String> words = new ArrayList<String>();
    words.add("File");
    words.add("Edit");
    words.add("Source");
    words.add("Refactor");
    words.add("Navigate");
    words.add("Search");
    words.add("Project");
    words.add("Run");
    words.add("Window");
    words.add("Help");
 
    System.out.println("Starting With: ");
    FunctionLib.map(new PrintFunction<String>(), words);
    System.out.println("\n");

Instead of using the normal for-each loop to I print out all the elements in a list, I map a PrintFunction (specialized on String) onto the list. Let’s look at the HOF map first. We know from using Haskell that map takes a function and a list, and applies that function to every element in the list. So, map takes two arguments: (1) a function from A → B, and (2) a list of A. Finally, after applying the function to every element, it returns a list of B. We collect all the HOFs we’ll implement into a FunctionLib class.

public class FunctionLib {
    // map :: (a -> b) -> [a] -> [b]
    public static <A,B> List<B> map(IFunction<A, B> fun, List<A> args)
    {
        List<B> results = new ArrayList<B>();
        for(A a : args)
            results.add(fun.call(a));
        return results;
    }
}

I’m mappping a <code>PrintFunction</code> that has the side-effect of printing each element it is called on. The primary difficulty with doing this is that according to my IFunction interface, I have to return something that fits in Java’s generics system. So, I specialize the return type parameter to Object and return null. Technically, the map method will remember the resulting nulls, and build a list of them, which then gets discarded. A bit wasteful, but it’s an eager language, so there’s not much we can do to avoid that.

public class PrintFunction<E> implements IFunction<E, Object> {
    @Override
    public Object call(E arg) {
        System.out.println(arg);
        return null;
    }
}

I don’t especially like creating a whole class file for each function I might want to pass into an HOF. Fortunately, Java supports anonymous classes. Let’s create one for getting the length of a string, call it lengthFunction, and pass it into map, so that we get back an array of Integers representing the lengths.

    System.out.println("The length of each String is: ");
    IFunction<String, Integer> lengthFunction = new IFunction<String, Integer>() {
        public Integer call(String arg) {
            return arg.length();
        }
    };
 
    List<Integer> lengths= FunctionLib.map(lengthFunction, words);
    FunctionLib.map(new PrintFunction<Integer>(), lengths);
    System.out.println("\n");

Although, not creating a class file for each new function is nice, it’s still hideously verbose to create a whole class. Oh, do I wish that Java provided syntax for lambda‘s.

Now that I have two lists, one of words and another of their lengths, I should be able to stitch them together. Haskell provides another HOF for precisely this task: zip is a function that takes two lists and returns a pair of their elements. The resulting list of pairs is the same length as the shortest input list.

public class FunctionLib {
    // zip :: [a] -> [b] -> [(a, b)]
    public static <A,B> List<Pair<A,B>> zip(List<A> listA, List<B&> listB)
    {
        List<Pair<A,B> results = new ArrayList<Pair<A,B>>();
        int len = Math.min(listA.size(), listB.size());
        for(int i=0; i<len; i++) {
            results.add( new Pair<A,B>(listA.get(i), listB.get(i)) );
        }
        return results;    
    }
}

Java is so hideously dysfunctional that it doesn’t come with a native Pair class. However, it’s easy to make one.

public class Pair<A,B> {
    public A first;
    public B second;
 
    Pair(A a, B b)
    {
        first = a;
        second = b;
    }
 
    @Override
    public String toString()
    {
        String firstStr = first == null ? first.toString() : "null";
        String secondStr = second == null ? second.toString() : "null";
        return "(" + firstStr + ", " + secondStr + ")";
    }
}

After all that monstrous cruft, we can finally get the list of pairs containing each word and its length.

    List<Pair<String, Integer>> associations = FunctionLib.zip(words, lengths);

But we’ve yet to do anything really interesting with that. So, let’s filter the list for words that are greater than 5 letters long. Haskell also provides an HOF filter which takes a function that is applied to every element in the list, and a list on which to apply it, returning a list of those elements for which the function evaluated true.

public class FunctionLib {
    // filter :: (a -> bool) -> [a] -> [a]
    public static <A> List<A> filter(IFunction<A, Boolean> pred, List<A> listA)
    {
        List<A> results = new ArrayList<A>();
        for(A a : listA) {
            if (pred.call(a))
                results.add(a);
        }
        return results;
    }
}

I use this function to pull out the pairs which have a string longer than 5 characters. Again, I again use the verbose anonymous class.

    System.out.println("The words with length bigger than 5 are:");
    IFunction<Pair<String, Integer>, Boolean> biggerThanFive = new IFunction<Pair<String, Integer>, Boolean>() {
        public Boolean call(Pair<String, Integer> arg)
        {
            return arg.second > 5;
        }
    };
    List<Pair<String, Integer>> longerThanFive = FunctionLib.filter(biggerThanFive, associations);

But, I’m really only interested in the strings, not the pairs. So, now I want a way to pull out only the first element of each pair.

    List<String> stringsLongerThanFive = FunctionLib.map(Pair.firstFunction(String.class, Integer.class), longerThanFive);
    FunctionLib.map(new PrintFunction<String>(), stringsLongerThanFive);
    System.out.println("\n");

Notice, that this time, I’ve given myself a convenience method, Pair.firstFunction. I’d really have preferred something like C++’s syntax here, but Java’s generic system has some pretty fundamental brokenness. First, as a result of type-erasure, the system doesn’t know what the type parameters into the function are at runtime. That makes an expression like Pair<String,Integer>.firstFunction invalid, because the type parameters are thrown away way before the method call takes place. Second, the type parameters to a generic method are determined via inspection on the types of the provided actual arguments. That makes an expression like Pair.firstFunction<String,Integer> invalid. You don’t pass type parameters at the function call site (as you would when you instantiate a generic object). Instead, as a workaround, you have to provide dummy actual arguments, that the typing system uses to specialize the call.

public class Pair<A,B> {
    ...
 
    public static <A,B> IFunction<Pair<A, B>, A> firstFunction(Class<A> a, Class<B> b) {
        return new IFunction<Pair<A, B>, A>() {
            public A call(Pair<A, B> arg) {
                return arg.first;
            }
        };
    }
 
    public static <A,B> IFunction<Pair<A,B>, B> secondFunction(Class<A> a, Class<B> b) {
        return new IFunction<Pair<A, B>, B>() {
            public B call(Pair<A, B> arg) {
                return arg.second;
            }
        };
    }
}

Things are a little bit different here than the PrintFunction we implemented earlier. We should like to keep the function that pulls out the first element of a pair in the Pair class. Doing this is not only trick because of Java’s generic type system deficiencies, but also because we can’t simply name a function as a reference (as you can in C). Instead, we create a firstFunction method, that when called with type parameters as actual arguments, returns an appropriately specialized IFunction which returns the first element when you pass it a pair in the call. Creation of the returned IFunction is also achieved via an anonymous class.

We now turn to the coup de grâce. We have a list of all the strings with length bigger than 5, so let’s use an HOF to concatenate them. Haskell again provides us with a function, foldl1 that will start at the beginning of a list and inductively apply a two argument function onto it, returning a single result. Essentially, if + were my string concatenation operator and my list were [a, b, c, d, ...] I’d be calculating (..((a + b) + c) + d) + ..). It’s a bit tricky in our implementation though. We have to pass into the HOF a function that takes two arguments, but my IFunction interface is only able to represent functions of a single argument. What shall we do?

Currying to the rescue! Haskell types the first argument to foldl1 as (a -> a -> a). Which you might naïvely think is an function of two arguments, but the arrow operator is right-associative. So, it’s actually parsed as (a -> (a -> a)), which is a function of one argument that returns a function of one argument. So, a function of two arguments is equivalent to a nested chain of functions, each accepting one argument. Which means, the IFunction interface is enough!

public class FunctionLib {
    // foldl1 :: (a -> a -> a) -> [a] -> a
    // In Haskell the function type operator -> is right associative,
    // So, the function description "a -> a -> a" is parsed as "a -> (a -> a)"
    // This order of application is represented in the nesting of IFunction's below
    public static <A> A foldl1(IFunction<A, IFunction<A,A>> fn, List<A> listA)
    {
        A accumulator = null;
        if (listA.size() == 0)
            return accumulator;
 
        accumulator = listA.get(0);
        if (listA.size() == 1)
            return accumulator;
 
        for(int i=1; i<listA.size(); i++) {
            // currying means two .call invocations
            accumulator = fn.call(accumulator).call(listA.get(i));
        }
        return accumulator;
    }
}

Finally, I can wrap up this exercise by applying a concatenation operator to a call to foldl1. This ends up being some of the most ugly code that can be written. Unlike the previous calls to map and filter, I did not want to save the temporary function into a variable. Rather, I’ve written it inline. Unfortunately, because of the currying, we have to nest anonymous functions!

    System.out.println("When we concatenate these we get:");
    String concat = FunctionLib.foldl1(
        new IFunction<String, IFunction>String, String>>() {
            public IFunction<String, String> call(final String arg1) {
                return new IFunction<String, String>() {
                    public String call(String arg2) {
                        return arg1 + arg2;
                    }
                };
            }
        },
        stringsLongerThanFive);
    System.out.println(concat);
    System.out.println("\n");
}

What’s going on here? Remember, we are representing a function of two arguments as nested functions of a single argument. So for example, I want to concatenate the list [a, b, c, d], I would pass in the concatenation operator, (+). Inside of foldl1, we take out the first element, a and invoke the call method with the first argument. This returns (a+), which is the concatenation operator with the first argument filled in. A second invocation is used to pass in the second argument, and that returns a string, which is saved in an accumulator.

This level of indirection means that I have to have some way of returning a function with the first argument filled in. I accomplish this by nesting anonymous classes and using a closure. The outer call method takes final String arg1 as an argument. final is necessary, because I have to tell Java that it is OK to ‘save’ or ‘cache’ this argument. I’m guaranteeing that I don’t have any code within this method that might change what arg1 referrs to. Once I’ve made this guarantee, I’m able to create and return a new IFunction with its own call(String arg2) that uses the outer arg1.

After going through all this awesome Computer Science (weird rules about the generics system, anonymous inner classes, currying, and closures), I decided that this is not something I could reasonably expect my students to do as extra credit. First, I would want extra credit to assist my student’s absorption of concepts, and yet still not be so difficult that it becomes a distraction to actual coursework. Second, it should be easy enough to help the students that need it to improve their grade. Although this example does hit the generics and higher-order functions topics, I think it has too many complex issues rolled together. For example, I’ve never actually had to use closures and anonymous inner classes before. Now, suddenly, I use them both in a single project!

UPDATE Mon Aug 8 19:56:37 PDT 2011: Jim Duey has a talk Functional Programming: A Pragmatic Introduction. At around 30mins he shows how Laziness can be used to implement a message queue for concurrent programs.

UPDATE Tue Aug 9 11:30:16 PDT 2011: You can download the project source here.

What’s interesting about this mapping is that, we can use it to engineer a language whose sole purpose would be to solve problems by authoring and combining DSL’s (what the Lisper’s are always claiming they do through the use of macro’s).
The focus here is to get a function call specification that can do a good job of representing all the control structures that are present within a grammar.
So let’s take a careful look at Wirth’s syntax notation to get a good idea of what needs to be modeled.

• Sequencing is conveyed by a space-delimited list of grammar elements: a b c d
• Repitition is denoted by curly brackets: {a} stands for ε | a | aa | aaa | …
• Optionality is expressed by square brackets: [a]b stands for ab | b
• Grouping is indicated by parenthesis: (a|b)c stands for ac | ab

Fowler gives some interesting suggestions, both in the above table and in the surrounding discussion within the book. I’d like to take some space here and explore each of the options, and how it relates to a Wirth-style grammar. Throughout this exposition, we’ll be using programming language constructs such as function calls and scoping to model the grammar rules typically seen in the above table. We’ll also assume that each grammar rule can be made to correspond to either a method or type (or both) in the implementors programming language.

• Mandatory list. Within a grammar, a mandatory list is indistinguishable from a sequence. Within a programming language, the mandatory list can also be expressed as a list of arguments that must be provided to a function. We can even invoke compile-time checking by having each grammar rule correspond to a type. Fowler doesn’t quite go that far, but he does give this illustrative example:
  

  Nested Function	Example declarations
  computer( processor( cores(2), speed(2500), i386 ), disk( size(150) ), disk( size(75), speed(7200), sata() ) )	ComputerT computer(ProcessorT, DiskT); ProcessorT processor(CoresT, SpeedT, ArchT); DiskT disk(SizeT);

• Optional List. As long as each list can be clearly expressed, we can achieve an optional list via function overloading. This way the compiler will match up directly to the instance of the correct rule. However, this amount of explicitness comes at a cost: either we keep the number of optional choices in the grammar to a minimum, or we commit ourselves to writing a large number of very similar functions (as the optionality can explode exponentially). If optionality is given in the grammar as a grouping, then it may make sense to create a new function representing only that grouping. This can reduce the amount of function overloading by reducing the number of function signatures that need to be supported. Fowler recommends two options:

  Method Chaining	Literal Map
  computer() .processor() .cores(2), .speed(2500), .i386() .disk() .size(150) .disk() .size(75) .speed(7200) .sata() .end()	computer( processor( :cores => 2, :speed => 2500, :type => :i386 ), disk( :size => 150 ), disk( :size => 75, :speed => 7200, :interface => :sata ) )

  Although the Method Chaining approach readily allows for the assumption of default parameters (for example the speed of the first disk), the hierarchical structure here is not fully captured by the chain of method calls. To support such a chain as that in the example, it would be necessary to hold a ContextVariable that each method automatically recognizes. (for example, so that size() can refer to the last disk inserted. Because this context is kept implicit, the compiler will not be able to capture an invalid sequence of method calls (instead this much be caught by validity checking code at runtime). Furthermore, it is important to keep straight the terminology of each different component. For example, it is not clear that speed(7200) should refer to the last mentioned disk or to the last mentioned processor. Reshuffling the order of method calls can be highly problematic, and can result in unwanted side-effects if terminology becomes mixed up.

  The Literal Map approach works pretty well for dynamic languages (python and ruby) which allow functions to accept a dictionary mapping terms to values. It would be nice to have named parameters in more of the static languages (Ada and C# have them, C/C++ and Java do not). Having a statically type-checked language implement this feature, including the ability to overload with default values, would be ideal. There is also nothing that preserves the order of entries (which isn’t all that important in this example).

• Homogeneous Bag. This is directly about modeling the repetition rule in Wirth’s syntax. In most programming languages, there do not exist easily used mechanisms for passing a variable length list of items.
  

  Literal List	Function Sequence
  martin.follows( "WardCunningham", "bigballofmud", "KentBeck", "neal4d");	computer(); processor(); cores(2); speed(2500); i386(); disk(); size(150); disk(); size(75); speed(7200); sata();

  The Literal List approach can be implemented, in languages such as C, with the varargs mechanism. But it forgoes type-checking, and definitely doesn’t feel ‘built-in’. Dynamic languages are more permissive in this regard. For example, Python has the *args mechanism, and JavaScript allows access to the arguments array. In either case, it turns out, aside from creating, explicitly, your own separate list to pass in, there just really isn’t a great way to implement this pattern. All is not quite lost, however, as some language allow for the in-line initialization of arrays. For example,

  • C++ allows passing of a streamed object: m(StringArray() << "blah" << "hey" << "yo"), which isn’t quite what we want.
  • Java does a bit better with m(new String[]{"blah", "hey", "yo"}); which sacrifices readability, but achieves the desired goal.
  • Java also allows declaration of a type-checked varargs m(String...); which is called like m("blah", "hey", "yo"), which is precisely what we want.

  The biggest drawback of using the varargs mechanism is that it lacks sequential composability. You cannot easily declare a function for the rule: A := B* C*. In this case, the more crufty, in-line instantiation mechanisms turn out to be better.

  The Function Sequence approach lacks pretty much everything I’ve considered thus far: it’s sequencing isn’t type-checked, terminology can be confused, it requires a State Variable to maintain context. The only way in which it isn’t worse than Method Chaining is that the compiler can tell you on what line you crashed, provided you responsibly littered your code with assert‘s.

• Heterogeneous Bag. Because this is more about repetition of optionality, it’s probably best handled by combining previous approaches. For example, you could choose to factor out the group as it’s own separate type, and then have a vararg listing of these types. Method Chaining can be used for list of configuration parameters, especially if order isn’t really the concern. You couldn’t implement this with a dictionary or map though, because of the repetition aspect.
• Set. Clearly the only approach to implementing an unordered collection of possibly heterogeneous types is a dictionary or associative array, best represented in the the Literal Map approach. Again, we’d like to have a clean syntax for this in our programming languages: type-checked keyword arguments.

For me, the most important aspect of this exposition has been the progressive realization that type-checked keyword arguments and inline-initialization are necessary elements to supporting a mini-DSL within your own programming language. Without these features, syntax too easily obscures what you are tying to express. It’s probably worth complicating both the semantics and parsing of the host language to provide these elements.

We also find that, using function calls as our implementation interface, the most difficult aspect of Wirth’s syntax is repetition. All the other rules fit very nicely into existing function interfaces. It would be very nice for more languages to support varargs in the way that Java has. However, I would like for this to be extended a bit further: it should be possible to have multiple vararg arguments, as long as they are type-distinguishable. For example, m(String..., Integer...), should be callable with m("a", "b", "c", 3, 1, 4, 1, 5, 9).

First, we’ll create a hierarchy of messages:

public abstract class Message extends Throwable
{
    public abstract String contents();
}
 
public class PrivateMessage extends Message
{
    String msg = "";	
    public PrivateMessage(String msg) {
        this.msg = msg;
    }
 
    public String contents() {
        return "(private)" + msg;
    }
}
 
public class PrivateMessage extends Message
{
    String msg = "";	
        public PrivateMessage(String msg) {
        this.msg = msg;
    }
 
    public String contents() {
        return "(private)" + msg;
    }
}

Then we’ll create a Sender and Receiver to test out a message pass.

public class Sender
{
    public static void main(String[] args) {
        Receiver r = new Receiver();
        r.message(new PrivateMessage("hello"));
        r.message(new PublicMessage("world"));
    }
}
 
public class Receiver
{
    public void message(Message msg) {
        try {
            throw msg;
        } catch (PrivateMessage pm) {
            System.out.println("got a private message: " + pm.contents());
        } catch (PublicMessage pm) {
            System.out.println("got a public message: " + pm.contents());
        } catch (Message m) {
            System.out.println("unknown message:" + m.contents());
        }
    }
}

Lo, and Behold! when the program is run, we get the expected output:

got a private message: (private)hello
got a public message: (public)world

I really have no idea if this feature is useful or just crazy verbiage for what could easily have been a daisy-chain of instanceof tests. Java makes it especially complicated because you must extend the Throwable class, in order to implement the messages as exceptions. Perhaps in languages without runtime type inspection this pattern might come in useful.