Yesterday, I had an interesting thought. My advisor once made the cultural observation that many people in Computer Science invent their own language and then immediately write a self-hosting compiler. I agree that a compiler is quite a feat of engineering and serves as a nice test case to demonstrate that the language you’ve invented is powerful enough that it can handle real-world complexity. Unfortunately, this test fails in a few important ways.

First, It doesn’t actually show as much as you think it might. There is a very strong filter on failed languages. By using this test the author runs the risk of re-designing the language, specifically to insert constructs that help them build the compiler. Now, this isn’t necessarily a bad thing, except that compiler writing is now a fairly mature field. There are standard abstractions (esp. in the lexing and parsing) that a new language will probably not experiment with. So, the author will usually just build these existing and well-understood abstractions into the new language. Rather than encouraging language experimentation we get more of the same, but with different syntax.

Second, Not all useful languages even have their own compiler. I’m specifically thinking of the domain specific languages (DSL). Nobody would write an awk interpreter in awk; or a mail engine using sendmail (even if it is Turing Complete). These are languages designed to do a specific task, many of them are quite essential to their respective fields, but none of them are self-hosting. Nor should we expect them to be.

My argument here is that the cultural practice of writing a self-hosting compiler is a big distraction. New languages should be for experimenting with new linguistic constructs. We should be looking toward the DSLs, and incorporating their innovations into our more main-stream languages. Right now, we seem to be optimizing our languages for compiler construction.

I’d rather see our languages evolve in a different direction. I’m really eager to witness the birth of an AI. For this to happen though, we need languages for expressing patterns of thought, not patterns of bits. We need the ability to cohesively and flexibly assemble the stuff of thought. I’m thinking Society of Mind stuff here. We need languages that allow for statistical fuzziness, sloppy associativity, and the ability to construct metaphor.

The linguistic tools that we find useful for building compilers are not necessarily the same tools that will help us build a mind.

So far, in my reading of Minsky’s Society of Mind, his hypothesis, that the mind is an agglomeration of specialized agents working in conjunction with each other completely meshes with observations of my own behavior. In particular, I’ve noticed that when I get stuck thinking about a problem, I’ll endlessly repeat, in my head, the knowledge and reasons surrounding the problem until the solution/new path/new thought occurs to me. It’s not me that’s doing the thinking here, rather It’s me that the thought occurs to, which explains the phrase, “It occurred to me that…”). So our Ego has the mistaken opinion that it’s the originator of all the thought in the mind, while all the time it’s more the receiver of the thoughts which occur in the brain.

But if this is true, then what is it that makes one person smarter than another? It must be that more thoughts (maybe of different character) occur to the smarter person. But then how does one make themselves smarter? One probable method would be to do daily exercises in logic puzzles and brain teasers, on the presumption that it will exercise and stimulate some parts of the brain from latent dormancy into activity, and that this sort of change in brain activity will be of general use in life’s daily problems. I’m not a psychologist, and have no data on the efficacy of this approach, but it seems plausible. More helpful, would be a correlation between specific types of problems and wether experience in solving particular instances of that type will extend to an increased ability to solve all problems in that class.

Yet the revered smarts of Einstein and Leibniz isn’t that they were particularly good at solving instances of know problems, computers can do that better than any human, it’s that they saw connections and aspects of unsolved problems that then allowed those problems to be solved. What brain calisthenics would help you to answer the currently unanswered questions? Here I draw a blank and even have a difficult time speculating. History is replete with anecdotes about flashes of insight that answer the prepared mind (penicillin was found in dirty dishes, structure of benzene revealed in a dream of snakes, gravity with the fall of an apple, etc). But beyond extended concentration on a given problem to prepare the mind, none of these tales suggest a general approach for encouraging the frothy bubbling of thoughts that the brain must present to the consciousness trying to solve the problem. Intelligence then will vary as a result of the computational structure in underlying medium (neural brain) that supports thought.

I’m at a loss when I try to conjure up a method by which we can transform my brain’s architecture so that it can better do my thinking for me. If Minsky is right (and I really think he is) then the solution must lie in the study of multi-agent systems and emergent behavior. Unfortunately, we are still developing the non-linear methods and mathematical tools that will help an understanding of such systems. But the research will be useful for much more than the study of thought, it applies to a very wide range of things found in nature (Economic behavior, Environment/Ecosystems, Evolution + Game Theory, etc.) and cuts across so many fields that it’s likely everyone has a roughly equal chance of contributing, wether they realize it or not. This really is the age of the multi-disciplined researcher.

Update: a Reasonable Deviations post about the Creativity Machine, which incorporates an apropos feedback mechanism, that readily models the difference between the consciousness which experiences thought and the separate generative mechanism of thought.

I was briefly reading Minsky’s Society of Mind and found these passages memorable.

The secret of what anything means to us depends on how we’ve connected it to all the other things we know. That’s why it’s almost always wrong to seek “the real meaning” of anything. A thing with just one meaning has scarcely any meaning at all.

The smaller two languages are the harder it will be to translate between them. This is not because there are too many meanings, but because there are too few. The fewer things an agent does, the less likely that what another agent does will correspond to any of those things. And if two agents have nothing in common, no translation is conceivable.

Dictionaries are really great tools, but they can only go so far. The really good ones (The OED) will give the user a really good ‘feel’ for the word, a sense of the connotations that go beyond the straightforward definition. A thesaurus can also be really useful, especially when you’re searching for a specific word, but can only remember associated words. But both of these tools lack visualization. Here is where technology can help out and create a more immersive, exploratory environment for our words. The thesaurus, through it’s simple listing of words related to other words, has some really interesting features, specifically that of directionality. Sometimes you find wordA in the listing for wordB but not vice-versa. Using these listings, we could build one giant directed graph of the English language. But what would we expect to find?

1. The graph is probably not acyclic.
2. High probability that it’s not planar, and therefore will be hard to draw.
3. There will be clusterings of words (probably short words) around certain concepts
4. These clusterings will probably center around descriptive features of our world (such as the mythical Eskimo words for snow)
5. It will reveal interesting conceptual connections between words, (sounds, cheese and knifes can all be ’sharp’)
6. Those connections probably relate to our internal models of the world (synethesia).
7. Emphasis of these connections probably varies by culture (but anyone familiar with idioms from more than one languages already knows this)
8. There will be gaps and holes in the language, that will show up as empty areas between conceptual clusterings.

I’m primarily interested in seeing how this type of visualization can help us to understand the tangled relationship between language and cognition. It’s been a combination of laziness and that 2^nd item that’s prevented me from writing this kind of software (which would be a great exercise to try out graph visualization techniques). But then again, others have forseen my vision (as usual).