eric the fruitbat » Comp*

The Good IR: Other Control Flow Structures

erich — Thu, 02 Feb 2012 02:39:58 +0000

In my last post on The Good IR, I had arrived at a representation with two invariants:

Only BasicBlock’s carry information about the ControlFlowGraph; No other edges are allowed between BasicBlocks.
All BasicBlocks end with a control transfer Instruction.

I focused that post solely on the if-then-else control flow structure. I would now like to demonstrate how those invariants translate to the while-loop and if-without-else. Although other languages have do-while, for-loop, and switch-case, I won’t be addressing those now. Instead I’m limiting my focus on a much simpler language (that used by CS241 here at UCI). I want to get a clear picture of how the if-else and while-loop work, before complicating the picture with Return. So, for this post, I’m assuming that we have ‘the usual case’ where the ‘stuff’ inside the body of a loop or branch of an if-then doesn’t proceeds ‘normally’ and doesn’t exit the function.

While Loop
while (cond) { // stuff }

We should not two important aspect of the illustration:

The body of the loop is fully general. That is, I wanted to make clear that the ‘stuff’ in the loop body might have control structures which cause a separation of the first block of the body from the last block. This separation is indicated by a series of small orange blocks.
Not in the illustration is a restriction on the instructions in the loop header. During a parse, when the loop is encountered we must create a new block to store two items: The instructions for evaluating the loop condition and the ConditionalBranch. We do not include instructions for any statement prior to the loop condition, even if such statement appear in the source program. We do this because, we wish to evaluate only the conditional and no other instructions, when control is restored on the loop’s backward control flow edge.

Given these observations. The presence of UnconditionalJump and ConditionalBranch is straightforward, and needs no modification from the last post.

Consider the degenerate case, when the loop body is empty. We could choose to draw the control flow graph omitting the body node. We decide not to do so, because it creates a special case. Instead we prefer that the loop body contains at least one block, even if that block holds only an UncoditionalJump back to the loop header block.

Forcing all loops to have at minimum 3 nodes: the loop-header, the body, and the exit block; gives us a stable footing for control flow optimizations which are implemented later in the compiler. It also gives us another invariant: No block points back to itself, all blocks always point to other blocks.

If-NoElse
if (cond) { // stuff }

The if-noelse is handled similarly. We do not have to create any instructions other than ConditionalBranch and UnconditionalJump. Again, we prefer to keep that ‘empty’ else branch. Regularizing the if-then-else to a minimum of 4 blocks: the if-header, then-block, else-block, and join-block. Always creating these blocks definitely helps the control flow graph keep an organized and uniform appearance, especially when you consider the nesting of several different control flow structures. An example from the last post would be quite unmanageable without the presence of these ‘empty’ blocks. In the if-then-else control structure, keeping these blocks aids in the placement of SSA φ-instructions, as well as control flow graph traversals.

Design Rule of Thumb: Don’t have the parser optimize away empty blocks, they’re actually useful to keep around (both for SSA and CFG traversals).

Transforming Heuristics

erich — Wed, 01 Feb 2012 21:51:53 +0000

Many of the real problems in the world are NP. Things like Scheduling, Register Allocation, Routing packages, etc. In solving these really hard problems, we invent heuristics. Typically such heuristics are specific to the problem domain. For example, UPS might exploit certain characteristic about the geographical layout of the country; they face a certain subset of all possible graphs, and can exploit those features.

But we know that the NP problems are all reducible to each other. So, don’t the heuristics transform as well?

That is, given a heuristic that works well for the Knapsack problem, how well does that same heuristic (transformed) work on the Travelling Salesman Problem?

The Good IR

erich — Fri, 27 Jan 2012 09:13:19 +0000

After parsing, each function can be represented as a ControlFlowGraph of BasicBlocks. Each BasicBlock holds a list of Instructions. For illustrative purposes Instructions are colored Blue and BasicBlocks are colored Orange.

I’d now like to address the question: What should the ConditionalBranch instruction at the end of the If-Header point to?

Option	Illustration	Pro	Con
Point at the following BasicBlock		Convenient: the target BasicBlock is available during parse.	Redundant: the If-Header block has a ControlFlowGraph edge with the same destination. Non-Uniform: it’s unsettling to have some instructions (such as Add/Sub/Mul/Div) point at other Instructions, but then deal with ConditionalBranch as a special case.
Point at the first Instruction of the following BasicBlock.		Uniform: Instructions point only to other Instructions.	Redundant: The If-Header block has a ControlFlowGraph edge with the same destination. Problematic: Can you guarantee that target all blocks have at least one instruction to point at?

The common drawback with both of these proposals, is that of Redundancy. At some point, during a future optimization pass, we might like to transform the ControlFlowGraph. If such a transformation requires that we update both the BasicBlock→BasicBlock edges and the target of ConditionalBranch Instructions, then we are more likely to have a bug. We must put forth extra effort in keeping both kinds of link synchronized.

We can avoid this common drawback by adding a layer of indirection/abstraction. Instead of forcing the ConditionalBranch instruction to maintain a pointer to its target, we can access the target through a function. That function can query its own BasicBlock about the outgoing ControlFlowGraph edges, and return either the target BasicBlock or the target BasicBlock’s first Instruction. By making this abstraction, not only do we avoid the extra effort of keeping redundant links synchronized, but we also promote uniformity in the design through the introduction of a new invariant: Only BasicBlock’s carry information about the ControlFlowGraph; No other edges are allowed between BasicBlocks.

Illustration	Option	Pro	Con
ConditionalBranch queries its BasicBlock.		Uniform: The only links across/between BasicBlocks are control flow edges held by BasicBlocks. Convenient: Do not have to synchronize redundant information.	Performance: A function call is required when asking a ConditionalBranch for its targets.

Now that we have decided that ConditionalBranch will return information about the targets via a function call, we visit the question: Is the branch target the successor BasicBlock or is it the successor BasicBlock’s first Instruction?

To answer this question, I’m going to assume that the Internal Representation can also be interpreted. Such a design has the advantage that we can verify the structure of the IR by interpreting it after construction and between transformation/optimization passes.

Let’s assume a simple interpreter of the form:

Instruction i = function.firstInstruction()
while (!i.isEnd()) {
    i = i.evaluate();
}

This interpreter exploits polymorphic dispatch on the Instruction hierarchy. We can express the loop concisely because we follow the invariant: evaluate always returns the next instruction to be executed. This is analogous to setting the program counter (pc), but avoids passing the pc as either a function parameter (which most evaluate implementations will ignore) or as a global variable (yuck!). Importantly, this loop has no concept of BasicBlock’s; it evaluates only Instructions.

So, in the interests of keeping the interpreter loop immaculately clean, we have only one choice: ConditionalBranch must return the first Instruction of the target BasicBlock. But, this answer contains a potential pitfall: Can we guarantee that all target BasicBlocks have at least one Instruction?

Consider the following example code, and associated ControlFlowGraph.

Code	ControlFlowGraph
if (cond1) { // do something 1 } else { if (cond2) { // do something 2 } }

Notice that, those blocks which do not have instructions in them have been left blank. Suppose that cond1 is false and cond2 is false so that the interpreter ends up following the chain of empty BasicBlocks. We notice that there is immediately some difficulty in having the interpreter transfer from ConditionalBranch(cond2) to the the first instruction of the (empty) target BasicBlock. One possible solution comes to mind: Have ConditionalBranch(cond2) attempt to iterate the following blocks until it finds one with a first Instruction. Although this will work, it feels rather kludgy, and we should continue our search for a clean design.

Let’s analyze the situation in more detail. Specifically, let’s look at the last instruction of the ThenBlock. It contains code for do something 1, which has been left unspecified to emphasize the fact that it can be completely arbitrary. However, (because we are gifted with knowledge of assembly) we have advanced some foresight: when the instructions are finally emitted from the CodeGenerator in a linear stream, we must insert an UnconditionalJump which bypasses the code for the ElseBranch (assuming one exists), and lands the program counter at the first Instruction of the JoinBlock.

In our IR interpreter, the last Instruction of the code inside the ThenBlock can be arbitrary, so it will have some difficultly detecting that control should be transferred to the following JoinBlock. We can alleviate this difficulty by analogy to our foresight, and introduce an UnconditionalJump in the last BasicBlock of the ThenBranch. As with the ConditionalBranch, the UnconditionalJump will rely on the ControlFlowGraph edge (we are guaranteed only 1) of its BasicBlock to determine the following instruction during interpretation.

We now have two situations of BasicBlocks which end in a kind of control transfer Instruction:

If-Headers and Loop-Headers, which end in a ConditionalBranch Instruction.
The Last BasicBlock of a ThenBranch, which ends in an UnconditionalJump.

The explicit control transfer in these situations allows the interpreter to easily determine the next instruction, even though it lies in a different BasicBlock. That is, in these two cases, the interpreter does not need to know about the existence of BasicBlocks. Due to this advantage, we should try to arrange one of these two situations to our current concern: the empty BasicBlocks in the ElseBranch.

Only one of the two previous instructions applies: the UnconditionalJump. It certainly doesn’t hurt the semantics of the ControlFlowGraph to insert an UnconditionalJump at every edge (even fallthrough edges). So we can safely coin the invariant: All BasicBlocks end with a control transfer Instruction (ConditionalBranch or UnconditionalJump). This invariant unifies our design, and allows the interpreter to iterate only over Instructions.

Additionally, we now also have at least one Instruction (the UnconditionalJump) in every BasicBlock! So, we can positively guarantee that a ConditionalBranch is able to return the first Instruction of its target BasicBlock. Indeed, all control transfer Instructions are able to do likewise.

Project Course in Web Services

erich — Tue, 24 Jan 2012 04:10:29 +0000

I’ve just finished reading Phillip Greenspun’s experience report, Teaching Software Engineering, which details a project course in building Web Services. Even though I personally, hate the Web’s architecture (but that’s a rant for some other time), it still remains as THE most influentential and convenient place to showcase one’s work. It’s also convenient for shopping, learning, participating in niche communities, etc. The Web has real business value, and is therefore un-ignorable.

Based on his experience with the course, Greenspun had some nice quotes, which pertain to thought’s I’ve been building recently as I’ve been digging into the question: ‘What makes education valuable?’.

Building Real-World Skill

We’d like our students to be able to take vague and ambitious specifications and turn them into a system design that can be built and launched within a few months, with the most important-to-users and easy-to-develop features built first and the difficult bells and whistles deferred to a second version. We’d like our students to know how to test prototypes with end-users and refine their application design once or twice within even a three-month project.

For every project in 6.916 Classic, we insisted on having a client. This is a person who can describe desired capabilities for an information system but offers no hint as to how to build it. The best clients are people who are in fact passionate about some sort of Internet service and completely clueless about all matters technical. Good sources of clients are dotcom CEOs, MBA students, non-profit organization directors, and university administrators.

we invited alumni who were working as professional software engineers to return to campus on Tuesdays and Thursday evenings to coach students during the 6 hours of supervised laboratory time per week. There are perhaps 10 alumni out there for every current MIT undergraduate.

The best projects were ones with clients who had the wherewithal to extend and maintain the service after the course is over, possibly by hiring the students who built it.

A plethora of useful circumstances are brought into alignment: Students are challenged in the same underspecified way that they’d face in a real job. That challenge is met by performing fast, iterative development, ala XP or Agile. They get to interact with an actual customer: deriving project worth from satisifing the client, and building experience with client rejection and other realistic bumps. Finally, they get to forge connections in a business network, which can help them when entering the job market.

Student Learning

Software engineering is a craft and can only be learned by practice.

Our experience [with producing five complete internet service projects] contrasts with typical software engineering courses in which a student builds only one application (or a piece of one application) during the entire semester. Research on simple word association tasks has demonstrated that people who learned to perform quickly but not accurately would have remarkably good recall even months later and, with a bit of practice, could always be made to perform accurately. Whereas people who were slow but accurate forgot all of their skills within a month or two.

Just another example where Quantity trumps Quality. People learn by actually practicing and experimenting. They do not learn by listening to lectures. Learning can be reinforced by discussion and analysis, by questioning and tweaking.

Building a Portfolio

At the end of the semester, a student in 6.916 could look back upon four or five completed Internet services. The first ones that he or she built had been done for the problem sets. They won’t have been complex. They may not have been built to a very high standard of polish. But their existence enabled nearly all students to become fluent in the arts of designing a data model, specifying a page flow, and implementing the designed system in SQL and a procedural language.

At the end of the semester we drill into the students’ heads the cold hard facts of the world: nobody owes them attention. We have each student group prepare an overview page that is a single HTML document, with a few screen shots, that demonstrates the major functions of the Internet service that they’ve built. Visit http://philip.greenspun.com/seia/gallery/ to see these pages.

Finally, it has been fun to watch our students graduate and go onto the job market. During job interviews they are able to point their interviewer to the URL of the running Web service that they developed during 6.916. Oftentimes, the student-built service is more sophisticated and is running on a more reliable infrastructure than most of the Internet applications launched on the public Internet by the interviewer’s company!

This is where I’ll have to distinguish my school: The lessons are online, but the workshop let’s you build your Programmer’s Portfolio.

Reaching for the Sky

Universities have long taught theoretical methods for dealing with concurrency and transactions. The Internet raises new challenges in these areas. A dozen users may simultaneously ask for the same airline seat. Twenty responses to a discussion forum question may come in simultaneously. The radio or hardwired connection to a user may be interrupted halfway through an attempt to register at a site.

In the second problem set (“reservation system”), students built a collaborative conference room scheduling system. This raises the problem of concurrency in a natural manner. Every student can understand that you don’t want to book two people into a room at overlapping times.

Third, because all of the projects have a predictable shape we’ll be able to introduce distributed computing challenges merely by having students offer services to each other.

Students said that the “metadata” problem set was very valuable for speeding work on their projects. Students were asked to build a knowledge management system by writing a computer program to write all of the computer programs. I.e., we gave them a machine-readable language for representing the system capabilities and user experience and asked them to write a program to generate the SQL data model and then the scripts to support the user experience.

In the final exercise of the problem set, we ask the students to mark certain rooms as requiring fees. Users who wish to book those rooms must supply a credit card number. At MIT we hook up the servers to a live merchant account at CyberCash. Thus our better students will be able to open their credit card statements in the middle of the semester and discover a few dollars in charges made by their own Web server.

UPDATE: Greenspun also has a quick bullet list of all the lessons learned, and outline of the course’s structure.

Separating the Wheat from the Chaff

erich — Thu, 12 Jan 2012 00:54:23 +0000

From Coding Horror: Separating Programming Sheep from Non-Programming Goats I learned of a paper, The camel has two humps, which describes a test that allows teachers to differentiate students likely to do well studying computer science from those who will likely never ‘get it’.

This paper sounds awfully similar to the physics conceptual test mentioned on American RadioWorks program, Don’t Lecture Me, that uses questions with little to no arithmetic to probe students for their level of understanding. The physicsts don’t use their test to cull the herd, but rather to let the professor assess which concepts the students haven’t yet assimilated. Astoundingly, the computer science paper proclaims:

We point out that programming teaching is useless for those who are bound to fail and pointless for those who are certain to succeed.

Evidence for this statment comes from low retention rates among computer science departments (30–60% fail the first programming course). But that observation doesn’t necessarily mean the students are ‘bound to fail’. Rather, I see it as evidence that the current methodology, lecturing, has not only failed to transmit information to the students, but also leads to such criminally dismal expectations.

One of the more promising experiences related in the paper demonstrates that we have to match education techniques to our learning experience. We should only use those tools which leverage how the brain learns.

Programming teachers, being programmers and therefore formalists, are particularly prone to the ‘deductive fallacy’, the notion that there is a rational way in which knowledge can be laid out, through which students should be led step-by-step. One of us even wrote a book [8] which attempted to teach programming via formal reasoning. Expert programmers can justify their programs, he argued, so let’s teach novices to do the same! The novices protested that they didn’t know what counted as a justification, and Bornat was pushed further and further into formal reasoning. After seventeen years or so of futile effort, he was set free by a casual remark of Thomas Green’s, who observed “people don’t learn like that”, introducing him to the notion of inductive, exploratory learning.

Programmers, both professional and beginner, spend much time dubugging. Much language research is therefore devoted to finding and building systems that either (a) help to prevent the writing of bugs (ex: static type systems) or (b) help to discover them once written. As far as that goes, it is interesting to note from this observation:

Thomas Green put forward the notion of cognitive dimensions to characterise programming languages and programming problems [12]. … He is able to measure the difficulty levels of different languages (some are much worse than others) and even of particular constructs in particular languages. If-then-else is good, for example, if you want to answer the question “what happened next?” but bad if the question is “why did that happen?”, whereas Dijkstra’s guarded commands are precisely vice-versa.

That one form of language construct, if-then-else, would be easy for a beginner to write in, but the other form, guarded commands, aids debugging.

The difficulty in learning a skill such as programming, lies principally in forming a mental model of how the computer executes the program. Forming an accurate model implies both the ability to describe your problem as a program and the ability of expert programmers to justify their programs. This understanding of how comprehension works drives the construction of the conceptual test, both in physics and computer science.

The paper has a very important finding:

… in the first administration [students with no prior experience programming] they divided into three distinct groups with no overlap at all:

44% used the same model for all, or almost all, of the questions. We call this the consistent group.

39% used different models for different questions. We call this the the inconsistent group.

The remaining 8% refused to answer all or almost all of the questions. We call this the blank group.

…
Remarkably, it is the consistent group, and almost exclusively the consistent group, that is successful.
…
It has taken us some time to dare to believe in our own results. It now seems to us, although we are aware that at this point we do not have sufficient data, and so it must remain a speculation, that what distinguishes the three groups in the first test is their different attitudes to meaninglessness. Formal logical proofs, and therefore programs – formal logical proofs that particular computations possible, expressed in a formal system called a programming language – are utterly meaningless. To write a computer program you have to come to terms with this, to accept that whatever you might want the program to mean, the machine will blindly follow its meaningless rules and come to some meaningless conclusion. In the test the consistent group showed a pre-acceptance of this fact: they are capable of seeing mathematical calculation problems in terms of rules, and can follow those rules wheresoever they may lead. The inconsistent group, on the other hand, looks for meaning where it is not. The blank group knows that it is looking at meaninglessness, and refuses to deal with it.

The test, which teases out these speculations, involves the notion of assignment. It iterates through different statement orderings, and variable names allowed to carry misleading implications about the underlying values. Many possible answers are given for each question, so as to arrive at the mental model that a student might have used. They also come with a space for free response or side-work. For example,

1. Read the following statements and tick the correct answer in the front column.

int a = 10;
int b = 20;

a = b;

The new values of a and b are:

[ ] a = 30  b = 0
[ ] a = 30  b = 20
[ ] a = 20  b = 0
[ ] a = 20  b = 20
[ ] a = 10  b = 10
[ ] a = 10  b = 20
[ ] a = 20  b = 10
[ ] a = 0   b = 20
if none, give the correct values:
    a =        b =

So the open question is: Is it possible for a teacher to help the inconsistent students learn a consistent mental model? If so, then we shouldn’t give up, we should bring the learning techniques which make it possible into the classroom. I conjecture that those techniques will also help the students with consistent models correct the bugs in their mental model faster. So dispensing with lecture, clearly shown not to work, and replacing it with newer, more effective techniques derived from cog sci, will benefit us all.

Business as an Investment

erich — Thu, 29 Dec 2011 07:27:58 +0000

I finished my reading of Mike Maloney’s Guide to Investing in Gold and Silver, partially to get an idea of how he got started in the business of bullion. He’s actually had several businesses throughout his life, including one where he designed “stereo amplification electronics were selected as one of five permanent exhibits at the royal Victoria & Albert Museum in London”[WealthCycles]. The last chapter contained what I was looking for. Starting with a goal to accumulate high-cash-flow apartments, he decided (based on research) to invest in the gold and silver cycle as it was building momentum (~2001). He also realized that further leverage could be obtained in the gold and sliver mining company stocks, and in starting a business that would do well during the cycle. Promoting the book, and joining Robert Kiyosaki’s team are icing on the cake.

Clearly, he didn’t position himself without some self-education. He had good reasons (stock market was languishing) to uncover information about the next cycle. As a practiced entrepreneur he knew both how to form and promote the new business: it was really only a question of figuring out which business would be the most profitable. He’s now quite passionate about the data he’s collected, and in helping others to profit from the information.

But you don’t achieve that kind of success without some up-front costs and research, together with the tenacity to carry through on the plan.

Recently, I’ve been trying to figure out what is the best manner in which I can use my existing capital (education about programming, dedication to reading/learning more, and passion for clearly explaining it to others) to build myself a stable future. On the one hand I could get a regular job either as a programmer at a large tech company (producing more for them that I receive in salary) or as an instructor a college/university (collecting considerably less in salary). But neither of these options gives me the autonomy I desire. Besides which, I think that Kahn Academy, has shown us that a revolution in education is afoot.

So, my current plan is to find a way of effectively educating people about programming: to provide them with the skills that allow them to join the class of highly compensated professional programmers. If I can uncover a mechanism that scales, so that revenues are less a function of the time I spend talking and more a function of the skills instilled in others: then I think I can build a stable, reliable income. The mechanism that scales well seems to be short self-contained videos about language features and design patterns accompanied by an XP workshop to build the interpersonal skills and practice.

What I learned from Mike: Building a business on the boom cycle leverages your gains.

Learning the Abstractions

erich — Tue, 15 Nov 2011 07:47:00 +0000

Since much of programming is about creating and manipulating abstractions, it figures that a large amount of education is going to be about leaning those abstractions. Things like classic data structures, the useful sloppiness of O-notation for algorithm analysis, and Design Patterns. But how should we introduce these things to our students?

Should you teach the abstraction first, because that’s what you wind up working with as a programmer? This risks disengagement and confusion. Without a clear concept of what is being abstracted, it’s hard to understand and pay attention. It’s difficult to see the benefit.

Should you teach the old methodology first? This risks building bad habits into the students. They typically feel resentful when you force them through muck and then reveal the pristine diamond-studded gold-brick road they could have used instead.

It’s important to know not only what is being abstracted, but also how and why.

Joel Spolsky, in his post, The Law of Leaky Abstractions notes (emphasis added):

The law of leaky abstractions means that whenever somebody comes up with a wizzy new … tool that is supposed to make us all ever-so-efficient, you hear a lot of people saying “learn how to do it manually first, then use the wizzy tool to save time.” [...] tools which pretend to abstract out something, like all abstractions, leak, and the only way to deal with the leaks competently is to learn about how the abstractions work and what they are abstracting. So the abstractions save us time working, but they don’t save us time learning.
And all this means that paradoxically, even as we have higher and higher level programming tools with better and better abstractions, becoming a proficient programmer is getting harder and harder.

Clearly, this indicates that the old methodology should be worked through first. And then students should, in a discussion section of some kind, discuss that methodology as a ‘code smell’ working out (with some guidance) a better solution for themselves. By following this path you motivate the understand through a previously worked example (which can be referred to during the discussion) and can clearly identify the leaks as you work toward the abstraction. Any bad coding habits picked up during the initial slog can be eradicated by revisiting the assignment and applying the new abstraction. There shouldn’t be resentment, because it’s framed as a way of applying some cool new pattern just learned.

I wonder… I haven’t seen anybody teach virtual dispatch and dynamic polymorphism from first principles in this manner. Usually that abstraction is granted as a built-in of the language being learned. Would it be useful to hand-hold the students through designing a virtual method table? could you find a motivating enough example? First doing it some crufty way, then identifying the VMT as a pattern, and finish by refactoring the initial assignment.

Object Oriented Compiler Architecture

erich — Sun, 13 Nov 2011 00:48:52 +0000

Out of some curiosity, I quickly read this paper today:

The Object-Oriented Architecture of High-Performance Compilers, Lutz Hamel, Diane Meirowitz and Spiro Michaylov, Technical Report, Thinking Machines Corporation, 1996.

Even though they assume that the compilers internal representation is some kind of AST, the underlying lessons are worth communicating, and are applicable even if your compiler has a block or sea-of-nodes representation.

First, we should separate the data structure of the IR from the algorithms (optimization passes and code transforms) which manipulate the IR. The Visitor pattern allows us to do this in the most generic way possible. This allows the algorithms to differ in their order of traversal. The only constraint is that they must adhere to the IR’s API, which should be kept very small. Since learning the Visitor pattern, I’ve been thinking that the phases of a compiler make it a Meta-Visitor or a Visitor of Visitors. It’s nice to see that viewpoint is shared by others.

Second, attributes (such as use-def chains) can be added to the IR separately, so that they are not visible outside of the transformations which require them. They propose attaching attributes to IR nodes by a templatized AttributeManager class, backed by an associative array. I would prefer if the Node class itself was extended (via subclass or decorator) so that it had a (possibly templatized) setAttr(...) method. However, we should still keep it possible to add and remove attributes “on-the-fly” so that they do not stick around past their usefulness, and so that they may be created on demand. I’m not sure how best to handle updating the attributes when the IR is transformed (i.e. lifting code from a loop might change the nesting count attribute). Perhaps a call-back mechanism to update or just invalidate that attribute’s manager.

Strong Typing for Security

erich — Fri, 11 Nov 2011 08:13:11 +0000

I got into a mild argument about static vs. dynamic typing. I recognize that static typing can be verbose to the point of being repetitious. Take Java generics for example:

List<String> astr = new ArrayList<String>();

There really isn’t a great reason why the compiler can’t infer the type of the variable on the right hand side of the assignment. C# already implements type inference for this case, and C++ is adding it. ML and Haskell are strongly typed and have practiced type-inference since their inception. So we should actually dismiss the verbosity objection to static typing right now, because it’s an artifact of implementation that the more popular languages, C++ and Java, represent really poor examples of what could otherwise be a really good thing.

In my opinion, a static typing system is actually a proof system over your code. We shouldn’t complain about having compiler errors, rather we should rejoice that the compiler is able to automatically detect cases where we were ambiguous or tried to do something ill-defined. We should try to write our code so that the compiler can tell us when we make a mistake. Really, we want to express as many constraints as possible, so that the machine can do more checking and we end up with less buggy code. Statically typing all our variables and expressing systematic constraints is an effort that pays off in spades for large code bases.

But couldn’t we all just use the more flexible dynamic typing languages, and catch the bugs with testing? In my opinion, no. Testing should be done anyway, but it isn’t enough to prove the absence of a bug. Only a proof checker, such as a static typing system, can come close to doing that. I think I can really drive this point home by examining web applications.

The Problem

Web applications are really glorified string processors. HTML requests come in as strings, and web pages are emitted as strings. JavaScript processes more strings in the page layout, potentially requesting even more information from the server in response to user-generated events. Forums, Social Networking, and other participatory applications allow for user generated content. This widespread and popular practice actually leaves our glorified string parser (web app) at risk: for, if we are not careful, a malicious user can supply a string which, if it appears in the ‘wrong’ context, might be interpreted as legitimate JavaScript code by the application. That is, malicious users can execute arbitrary code, with the full rights and privileges as the application itself. This vulnerability is known as Cross-Site Scripting (XSS).

So, we find ourselves writing a string processor which must deal with strings of various encodings, special characters, and escape conventions. Namely, HTML, JavaScript, XML, CSS, URL. If one of these strings (even from our own database) manages to arrive in a context without first going through a filter to sanitize it, then our application has a security vulnerability. Do you think that it’s possible to write test cases (or even auto-generate them) given all the code paths, all the different sources (user, cookie, url, database, etc) and all the contexts in which a string might appear. In my opinion, the exponential complexity makes testing an infeasible approach. What we really need, then, is a proof system to verify that no strings end up in the wrong context.

Static Typing to the Rescue

If we are willing to go back to our application and examine it in detail, we find that we should really be treating each of the above strings as different types. HtmlString should be a different type from JSString, which are again both different from UrlString. Simply expressing each context as a different type enables our static typing system to verify that we never use the wrong kind of string in the wrong context. We can also provide explicit conversion functions, which provide the proper escaping and sanitization when moving from one context to another.

void addToDocument(HtmlString hStr);
HtmlString fromURL(UrlString uStr);
UnsafeString HttpRequest(UrlString uStr);

Language Support

What’s most unfortunate about this approach is that neither C++ nor Java provide us with an easy way to distinguish two strings. We certainly don’t want to use C’s typedef, because that enables automatic coercion between the different kinds of string, which defeats the point. So, we’re forced into creating a separate class for each of these strings, including implementing all the operators that make for convenient string manipulation. I’d really love a language that would allow me to extend my existing string type without fully re-implementing everything, yet still be able to treat the extension as a completely different type.

Conclusion

Essentially we’re using the static typing as a proof system to constrain our programming practices. The static type verification provides a proof that we never use a string in the wrong context. In my opinion, this coding technique is of enormous benefit, and represents a use-case that dynamic typing + unit testing simply cannot approach.

The real trick is recognizing that two strings aren’t necessarily the same type.

Just for reference, I did not come up with this example myself.
Joel Spolsky advocates using Hungarian notation, which I think is too weak for solving security vulnerabilities.
Tom Moertel provides an inplementation of this approach in Haskell.

Unit Testing in Education

erich — Thu, 29 Sep 2011 08:11:57 +0000

In my readings of Extreme Programming, one thing I’m struggling to pick up on is Unit Testing. Mostly, it’s the pain of writing tests for each thing I can imagine goes wrong. Partly it’s an imaginary horror: I think about the full extent of testing all at once. I think about exhaustively testing my program’s behavior. But in reality, writing the tests is not that bad. You don’t sit down and write a hundred unit tests for every little thing you can think of. Rather, you test (and program) incrementally. So, my overactive paranoia about what I have to test doesn’t represent the reality of testing. Instead of imagining everything at one, I’ll have to train myself to focus on the small, achievable, easily testable things.

Once I get over that, and actually start to write my tests, I’ve noticed some other effects. Writing the tests before the code, actually gives you a good idea of how I’ll want to interact with the code. The very act of hypothetically using it gives me a more concrete picture of what I want to accomplish. The burden of writing the test helps to keep the complexity down. You end up with a better mental model, cleaner architecture, and fewer dependencies.

Ok. Great. So unit testing my code has nice benefits. But can it be applied elsewhere? Say, in education?

Well certainly you could teach unit testing philosophy. You could drill into your students the virtues of unit testing their code. But that’s not what I mean by applying unit testing to education. I mean, that we should look at the education process itself, and see where unit testing fits.

Most classes consist of some mix of the following:

Lecture and Discussion.
Exams. Quizzes, Midterms, and Final.
Assignments. These can be Projects (individual and group), Research Papers, Homework drills, etc. Work that the students are responsible for doing, and which they hand in for a grade.

Looking at these components, we should ask ourselves: How effective is each piece in contributing to the total outcome? How does each add to a students memory and experience? How well does each help the student learn? When we follow these questions an interesting picture begins to emerge.

The purpose of lecture is typically used to demonstrate some of the concepts through pretty pictures, compelling narrative, and highlighting of important things to remember. Discussion sections are typically used to get the students somewhat involved with the material. That is, we expect active participation in discussion vs passive participation in lecture. Assignments are typically used to give the students practice with the material. They are contrived to highlight certain aspects of the field, or certain skill sets that have been found useful in actual research. They are a way to artificially build experience.

Exams and quizzes stand apart in that they don’t necessarily contribute to student learning. Rather, they are used as a metric for assessing how much knowledge the student has absorbed, or how well they are able to assemble that knowledge for solving a problem. These devices are pedagogical unit tests. But, unlike unit tests I write for my code, they are exercised last, not first. They aren’t exercised often, and they don’t really follow iterative development.

So, now that we’ve identified a component of our educational system that’s analogous to the extreme programming methodology, how do we improve our classroom techniques to actually use the same methodology we want to teach?

Maybe the tests should come first. Instead of having final exams as an exit criteria, we should have entrance exams as a placement/enrollment criteria. I’m not quite sure how well this would work, but it should improve the relationship between teacher and student. No longer are the teacher and student in adversarial positions about the grading outcome.

Quizzes are a great way to measure incremental progress, and many instructors already use them as an appropriate feedback mechanism about the material. In fact, a first-day quiz about ‘what do you expect to get from this course’ is often a great way for an instructor to customize the material to student learning objectives. Just as programmers use unit tests to pinpoint software errors, we can use quizzes to pinpoint conceptual errors where they occur.

The XP methodology seems to suggest that a comprehensive test is not the way to go. Rather, it’s better to have a greater number of smaller quizzes which test specific items. It also suggest running tests repeatedly. For humans, too much repetition can become monotonous, but just enough definitely improves retention. We also want to make sure that the most recent material learned doesn’t supplant material covered at the beginning of the course.

Finally, and most radically, XP suggests writing your own tests. I think it would be a great world if students became responsible for their own learning outcome. And what better way to motivate them, than by feeding back their own ‘what if’s on the tests? Coming up with good questions is actually very difficult, and provides an altogether new way of viewing the material. It forces a re-conceptualization of what was learned, providing yet another avenue of engagement. I think it will also viscerally demonstrate that the students aren’t helpless sheep waiting to be watered with a font of knowledge from the professor. IF we can find a way to incorporate this radical step, I think it would help to form a cycle of learning (ask questions, analyze answers, repeat!) and give our students practice with the self-reliance they will need in the real world.