The Four Stages of Exploration

Posted on 2007-11-20 09:03 JST by Curt Sampson :: comments enabled

As we improve our programming skills, we learn in different ways, and from different sources. Drawing my my experience and current state, my theory de jour is that your level of learning and the technologies and methods you use fall into four different stages.

The first stage might be called, “Learn Foo in 21 Days.” Here you’re finding out, from either books like this or people around you, about things that have already achieved mass acceptance, and catching up with what anybody who’s been working for a while in the industry, no matter how good or bad they are, already knows. Some of what you learn here, such as “structured programming is good,” few will disagree with. Other things, such as “plan very carefully before you start coding,” a few more advanced developers may strongly disagree with (at least in some situations), but the majority will accept it as received wisdom. And in many cases these approaches, while perhaps far from the most efficient way to develop software, will at least get the job done.

Many people, especially those without formal training, spend years in this stage. A fair number spend all of their lives in it. My time here extended over a decade and a half, from 1982, when I first started as a kid hacking in BASIC and assembler on an Apple II, until the late ’90s, somewhere around the time I switched from being a full-time sysadmin to a full-time programmer.

In the second stage you’re still learning from books, but learning from the cutting edge in what’s published, and it’s much more rare to have others around that can teach you. While you’re not doing anything that other programmers haven’t done before (and probably done a fair number of times), you’re a bit ahead of the majority and often going it alone, at least within your workplace or community. It’s your own drive that leads you to figure out how you can take the lessons from the books and articles and apply them to the software you’re building. At the beginning of this stage, you’re using fairly advanced tools, such as unit test frameworks, that others aren’t yet; by the end, if you take it far enough, you’re casually writing these tools where they don’t already exist.

This was where I was in the early 2000s, shortly after Kent Beck’s book Extreme Programming Explained came out. Nobody else I was working with was trying anything like this and, particularly in the area of test-driven development, I had to work out a lot of things myself when it came to applying what I was reading. Eventually I reached the point where I could and did easily use these methods in any situation, building the tools I needed where they didn’t already exist. (Last year, for example, within hours of using the Flash IDE and ActionScript for the first time, I’d written from scratch a basic unit test framework for it.)

You can usually tell when a technology or methodology has reached stage two because the ideas are easily available to the mainstream, but not yet widely accepted, and often prompting vehement objection. Extreme programming is in the later part of this stage; while it’s now popular enough to spawn conferences left and right, and everybody I talk to knows about it, many people still don’t believe it it can actually work except in very limited situations, and only with exceptional programmers.

If you reach the third stage, you’re in the same territory as those who are writing the books: there are other people doing similar things but, though the ideas are out there in blogs, mailing list postings, and the like, they’re still somewhat experimental and you’ve got to do a little more original work. (Students writing a thesis are expected to be working at this level, at least for their particular topic.) There are no complete, concise books out there written by people who’ve been doing this on many projects (though they may be in progress), and easy-to-use tools are not yet built. You dig around, find an article here or a posting there, but you need to experiment to integrate these ideas into what you’re doing, and do a reasonable amount of original work as well.

Domain specific languages (the topic of my next post) appear to be in this stage. Here, the mainstream is more bewildered by the ideas than dismissive of them; there’s not enough understanding of the ideas to hate them.

The last stage is where you’re blazing new ground: you’re the one writing those articles and blog entries, but what you’re doing is still enough in the experimental stage that nobody’s even ready to start a book on the subject. While you’ve got specific systems using these techniques up and running, there’s just not enough experience between you and anybody else using those techniques to generalize what you do. Starling’s QAM appears to be in this situation right now. Very few others are paying the kind of attention that we are to the build and release process, and the system requires people outside of our very specific circumstances to understand intimately what we’re doing so that they can modify it for their circumstances.

Your technologies and skills in different areas can be (and almost certainly will be) at different stages at the same time. Which stage you’re at also depends on your environment and your point of view. For example, while I’d say I’m personally at stage two in terms of advanced functional programming, many of the ideas I’m working with are decades old and have seen extensive use. They just haven’t made it out of academia or research labs in any significant way. This was the case with object oriented programming in the early 90s, before the advent of Java, and this is where Haskell is right now (and has been for a decade).

Broad understanding of an idea (or lack thereof) makes a big difference. It’s a lot harder to learn and use truly new concepts when you have to learn them alone, rather than while working with experienced people. It’s even worse when those around you push hard against what you’re doing because they misunderstand it.

The Dreyfus model of levels of expertise, which is particularly well described in Patricia Benner’s From Novice To Expert: Excellence and Power in Clinical Nursing Practice, offers a slightly different take on this whole matter, but comes to essentially the same conclusions as far as the differences in learning styles and the sharing of understanding between the different levels. (This, incidently, is another idea that has not been noticed by the mainstream, despite being decades old. Wikipedia has yet even to start an entry on it.) I may have more to say about this one day in another post; in the meantime, I highly recommend the book.

In my next post, I’m going to talk about Domain Specific Languages, or DSLs, an idea that is just ready to be noticed by the mainstream.

Add a comment »
comments are moderated