Writing software is hard, particularly when the tools you use force you to think at too low a level; it's time to start thinking about changing the way you write code… by making it easier to write code. Back in the days of our fathers, programming meant focusing on learning one language, one platform, and one environment, and mastering it over a span of years. Those years are long behind us, along with half-decade project development times and bell-bottomed pants. It's time to take a hard look at the “state of the union”, per se, and see how you can think about how to work better, rather than just putting in more time.
Greetings. As frequent readers of this magazine will already know, this space used to be where Ken Getz would prognosticate and discuss all manner of things technical, offering his insights and experience. Ken decided to hang up his spikes, as they say, and Rod and the crew at CoDe Magazine asked me to help fill in for his absence.
Let's be honest, nobody could ever replace Ken's… let's say… unique… perspective, but it's flattering to be asked to try. In fact, it's a touch intimidating, because Ken had a perspective that stretched across decades. Fortunately, I have a solid crew of people with me at ThoughtWorks, the consulting company I work for, who are willing to step in and help me try to fill in Ken's footsteps.
Hence the name of this column.
In the UNIX world, developers have known for years that sometimes the easiest way to attack a problem is to create “a program that writes a program.” Sometimes this is done in Perl, often combined with code-generation. Sometimes this is done with regular expressions, either fed to a library that returns any hits from a string, or compiled into an actual code library that can be included with your code, which is another example of code generation but in binary form. A third trick was to create an entirely new tool, sometimes known as creating a “little language.” Today, in a grinning endorsement of the truism that “everything old is new again”, this notion of creating the “little language” rises again under the new moniker “domain-specific language” (or DSL), and currently enjoys tremendous media discussion.
The subject of programming languages-or, more specifically, building programming languages-is one that most “line-of-business” developers find hard to relate to. This isn't surprising given that most programming language research has been so incredibly restricted to the academic and research communities. These restrictions weren't because those communities don't want their research widely read or consumed, but because for the “line-of-business” developer, it didn't seem really relevant… or useful.
Unfortunately, that “relevance gap” meant that if a developer were out hunting around on the Internet looking for cool new tools or libraries to help build a highly-scalable (meaning highly-concurrent and thread-safe) Web application, she might run across some papers or discussions that reference this concept of “functional languages.” Further research on the subject might lead to some papers on why functional languages are great for concurrency, sparking her interest. She does a bit more reading, likes what she sees, and sees a reference to a concrete implementation of a functional language called OCaml.
OCaml, for those who've not spent a stint in the research or academic community, meshes the functional and object-oriented programming approaches in some interesting ways. For that reason, among others, it enjoys a pretty good following in academic circles, including some interesting forays into the concurrency space. So our intrepid developer begins working with the language, learning the new syntax and playing with the new libraries, and decides that it's interesting enough to want to see about building something real with it, prototyping it for use on her current project.
Like any good horror film, at this point the audience can see it coming, but our developer can't, for whatever reason.
She runs into problems. Not tiny little problems like finding a version that runs on Windows, but larger problems. Obstacles that cannot be overcome in any reasonable timeframe. Building a UI with OCaml is awkward and difficult at best, since it tries to be platform-agnostic and thus relies on other adaptation layers to reach the underlying widget set. Accessing Windows-only features (such as printing) will be a pain, requiring her to build a “trampoline” layer to access native C-based Windows APIs to do anything. Trying to connect with any of the modern relational database implementations will require… patience.
The problem isn't really our developer's fault, nor is it OCaml's, to be honest. It's not that OCaml isn't interesting-it is-but as it comes, out of the box (so to speak), OCaml isn't something that the line-of-business developer can make use of in any sort of pragmatic sense. Which means, unfortunately, that not only did our developer waste her time researching the subject and discovering its applicability, but she also learned that academics and researchers are not nearly as interested in creating real-world, production-ready solutions as they are in exploring the various ideas and research, at least to the point where they can prove/disprove the idea. How likely is our frustrated developer to trust something out of the academic or research community in the future?
Therein lies the heart of the problem: academics and researchers are focused on trying to solve the problems that line-of-business developers will face over the next five or ten years… often at the expense of building solutions for today.
Stay with me, friends, there is light at the end of this rather bleak tunnel.
In the past couple of years, an interesting trend-starting with the Java Virtual Machine's original release-has begun to take root in the .NET world as well: Some researchers started building their languages to run on top of the CLR. They do this not just to make their languages and tools available for .NET developers to use (although that's a nice side benefit), but because when they target the CLR, a large part of the problem they don't want to solve (creating a garbage collector, creating a module-loading facility, accessing the underlying platform through a native-API layer, and so on) is already solved for them. They simply build their language to emit IL instead of native code, and suddenly, they can take advantage of everything the CLR offers, from ADO.NET to WPF to Castle to NHibernate to… you name it.
All of which leaves the academic and the researcher in a fine place… but so far nothing's been said again about the line-of-business developer. My story started with a developer looking for a way to build high-scale concurrent applications, and so far, I've established that the world is a better place for the academics and researchers; the benefit back to the “mainstream” development community seems a bit tenuous so far, since it generally takes a while for new concepts and ideas to filter their way from “research” to “mainstream”. After all, it took object-orientation something like 25 years to go from the academic/research world into the “mainstream” world.
(Somewhere in the back of the room, a hand is waving madly and its owner is shouting “Smalltalk! Smalltalk!” Sorry, dude. Yes, Smalltalk was useful, yes it was powerful, but you'll have a hard time convincing me it was ever "mainstream.")
The fortunate thing about this new renaissance around programming languages is that they're being created on top of the platforms found in commercial enterprise settings-the JVM and the CLR. So when an academic decides to take the OCaml programming language and move it to the CLR, line-of-business developers can use this work almost immediately. For example, a Microsoft Research project called “AbsIL” made it easier for developers to take compiled assemblies, untwist them back into a tree structure (called an Abstract Syntax Tree) and modify the code before re-emitting them back out as compiled assemblies again. Doing this is intrinsically a functional kind of operation-given the same input, the code will always generate the exact same output-and so as the Microsoft Researcher responsible for AbsIL built that library, he also sat down to produce a CLR version of OCaml.
But to make that version of OCaml-on-CLR work correctly, he first had to implement a form of parametric polymorphism for the CLR…. So he did, as a series of source patches on top of the Microsoft open-source .NET implementation, known as the Shared Source CLI, or “Rotor.” That set of source patches is still available to this day, named “Gyro.”
Still not convinced that all of this has relevance to the line-of-business developer? You might change your mind when you realize that this series of source patches to “Rotor” became the seed for the generics model that Microsoft folded into the CLR in the .NET 2.0 release, and that the CLR-based version of the OCaml language came to be known as "F#", and is being included in Visual Studio 2010.
In fact, the next generation of the .NET platform is about to explode into new languages. In addition to F#, Microsoft will release two more languages via the open-source community, ports of Python (IronPython) and Ruby (IronRuby). And the rest of the community is also stepping up to the languages plate, with new languages like Boo and Nemerle and P# (Prolog#) to add even more variety to the mix.
And because of this increased focus on languages, it's become even easier for developers to build their own languages-whether a “DSL” or something that makes it easier for other line-of-business developers to stay focused on their chosen tasks. Given some good examples to work from, such as Joel Pobar's “Good For Nothing” compiler from TechEd a few years ago, a developer can produce their first simple custom language in about a day, even with no prior experience in building languages.
This isn't to suggest that every single project anybody works on from here on out is going to require a custom development language or a DSL; like all new tools, there is a time and a place for building a custom language or a DSL. But it is fair to say that the next five or ten years is going to see an explosion of new language interest and implementation, and �tis the wise developer indeed who looks to get in front of the wave to ride it, instead of paddling like mad from the back.
