In Show #449 we spoke to Niklas Gustafsson and Josh Phillips about Axum, a new language developed specifically for parallelism.
Carl Franklin: Well, this is a subject that’s near and dear to my heart, parallelism. We’ve done quite a few shows ranging from what’s in the [.NET] Framework to best practices to the Parallel Extensions, and of course this is an old problem. As long as there have been computers and computer programming, there have been multiple threads and asynchronous programs, and it has never been fun.
Niklas Gustafsson: Right.
Josh Phillips: Agreed.
Carl Franklin: So, tell us about Axum.
Niklas Gustafsson: Okay. Thank you for having us today. We think that Axum is an experiment, first of all. It’s not something that Microsoft has committed to shipping at this point, but it is a very exciting experiment I think, and one that has potential of solving a lot of problems with concurrency and parallelism. The primary premise is that most of the current models that are offered even in .NET 4.0 have a lot of safety issues around them and that as long as you follow the rules for accessing shared state and accessing your variables and heap-allocated objects, [if you] follow certain quite restrictive patterns you’re going to be safe. But, you have to understand those patterns, you have to understand how to apply them to your algorithm, and you have to understand when to deviate from them if at all.
Carl Franklin: Give us just a one-sentence description of what this is. Is this a set of guidelines? Is it a language? What is it?
Niklas Gustafsson: It’s a language and the reason that we believe that having a language encoding of these patterns is important is that when ’you’re trying to restrict what somebody is doing, restrict somebody that’s used to living in a free-for-all that the object-oriented paradigm really is, into something safe. One of the best ways to do that is to define language semantics, to do that: to actually define a language that embodies those restrictive patterns, that’s what Axum is.
Richard Campbell: So the first thing that sort of jumps out of my mind is that this is like what Visual Basic 1.0 was to Windows programming, now for parallel programming. Just give us a safe place to play where we won’t mess up and destroy things.
Niklas Gustafsson: Yeah, that is a very flattering parallel to draw, I think. If we can live up to that, that’s fantastic. Of course, Visual Basic 1.0 came with an integrated library and an integrated tool and that was very instrumental in making it successful as a GUI programming platform. But, I do think that you’re right that we’re addressing some of the same productivity issues. Parallel programming is hard. And GUI programming in the late 1980s and early 1990s was hard. You had to do it in C or C++.
Richard Campbell: Right.
Niklas Gustafsson: It was difficult; it was challenging and not very productive. So we’re facing the same situation now with parallel programming and we need some language, great tools, a great framework, but we also need some language support to help it.
Carl Franklin: Is this a DSL, or is it an M language, or is it something that plugs into Visual Studio? What’s the nature of it?
Josh Phillips: We don’t like to call it a DSL but it’s a special purpose language that you can use to write your whole application in, but you probably won’t get much done. If you’re going to do anything significant, you’re going to have to interoperate with other languages like C# and Visual Basic. It’s a coordination language really.
Carl Franklin: I just have to clarify. Did you say you do use this with Visual Studio, this is a Visual Studio .NET language?
Niklas Gustafsson: Yes, yes.
Carl Franklin: Okay.
Niklas Gustafsson: But the idea is that you want to establish safety at the very top level of your application.
Carl Franklin: Right.
Niklas Gustafsson: And then you’ll have these safe bubbles in which you will write most of your application code in a language like C#, which Axum looks a lot like syntactically. But you can also use any .NET language, obviously Visual Basic or IronPython or F# or whatever you want because they’re all .NET languages and the whole .NET infrastructure is there to support all languages. But the idea is that Axum would be used for certain aspects of your application. We call it special purpose. We don’t like the domain specific language label because those are typically associated with very specific technology domains and Axum is not about that. Parallelism is not a domain in that sense in my opinion. It’s a particular aspect of general purpose programming.
Richard Campbell: Right.
Josh Phillips: What’s also important is, parallelism is very in your face in languages like C# and VB. It’s something that you sort of have to think about explicitly. We want to sort of move away from that model where you just define components and you let the runtime worry about the parallelism for you.
Carl Franklin: So it sounds like when you use Axum, you’re designing up front for parallelism and that is maybe why you’re suggesting you do the bulk of your application programming in C# or VB.NET and use Axum for the parallel pieces. I imagine since it’s a .NET language, I imagine it can co-exist as a separate component or as a separate project in a solution.
Niklas Gustafsson: Yes, correct.
Richard Campbell: So we’re really just getting at the idea that certain pieces of your app are going to be parallel. I mean, I still jump back to the parallel task library and there you are quite explicit in saying, okay, here is a piece that I want to do in parallel.
Josh Phillips: So when you look at your application, you want to look at it in certain layers and the way that we look at it is there’s an upper level which we talk about as these components that are really just asynchronous agents that message to each other and talk to each other and you get concurrency at that level. That’s more coarse-grained. Then there’s also the parallelism that you get with things like task parallel library, and we’re not saying that those are necessarily mutually exclusive. We want to use them together. And Axum is really about that first piece there, about the component-level concurrency.
Carl Franklin: And you use the word agent there, which is a new word for people who are going to use Axum, it’s the first thing in the programmer’s guide that you come up against. They are sort of entities, you might want to think of them as objects but you call them agents. What’s the difference really?
Josh Phillips: That’s a great segue from the abstract into the concrete tier. We’ve been talking about this establishing safety but we really haven’t talked about what Axum does to establish safety.
Carl Franklin: Okay.
Niklas Gustafsson: The interaction with the task parallel library and the contrast to it is also very interesting in that particular concept. So let’s start with TPL and while it’s not appropriate at the safe programming models, TPL is extremely useful for-and it’s available in .NET 4.0-it’s extremely useful when you have either an embarrassingly parallel algorithm, in other words something that trivially it’s parallelizable and trivially it’s safe when parallelized. The first example you look at typically when you look at parallel algorithms is matrix multiplication where you really have three nested for loops that go over two matrixes and multiply them together and you wind up with something that is very, very easily parallelized. You take the outermost loop and parallelize it. Why is this so easy to parallelize? Because in each iteration of the loop, there’s no dependency on previous or following iterations of the loop, so everything runs perfectly well in parallel. The data is very separable too. Each column or row of the matrixes are independent of each other so you already have that partitioning there. Now that’s an extreme example. Most algorithms are not that easily parallelized and not that free of data dependency between the pieces that you would parallelize.
Carl Franklin: Right.
Niklas Gustafsson: Another extreme is I/O-driven concurrency where you’re really not looking at a parallel algorithm per se, but you’re looking at multiple independent computations all driven by the availability of data coming in from a network or data coming off of disk where your parallelism has nothing to do with the computation that you’re trying to do, but it’s really about multiple computations going on at once and over shared data. Shared data is the basis of almost...
Carl Franklin: All evil.
Niklas Gustafsson: All evil, exactly. And most simple parallelism problems are based on the fact that you don’t know what you’re doing when you’re accessing shared state from concurrent or parallel pieces.
Carl Franklin: Yeah. So you have these agents that are these little autonomous guys that pass messages back and forth to each other and that’s really the only way that they can communicate, isn’t it?
Niklas Gustafsson: Yes. They cannot communicate with each other through shared space, through shared data.
Carl Franklin: Yeah.
Niklas Gustafsson: So what they do is they send messages to each other and the messages are safe. Now we eliminate one class of problems, which are the data that races here. I want to be careful to say that we’re not eliminating deadlocks. You can still deadlock with the message passing system. We can eliminate one class of very, very common; I would say the most common parallelism problems, which are data races.
Carl Franklin: Race conditions, yeah. So you set up a little mini SOA on your desktop almost.
Josh Phillips: Exactly.
Niklas Gustafsson: That’s an extremely good analogy. And in designing this, The principles that we’ve started with were the principles of how the Web is programmed because if you think about the Web and what it really is, it’s a massively, massively parallel application.
Carl Franklin: Yes, it is.
Niklas Gustafsson: And yet it is programmed not by rocket scientists but by everyday programmers who are, you know, many of them are self-taught, many of them have never taken a class on computer science or parallelism or whatever it may be. And it’s remarkably robust. So a lot of the principles behind Axum directly borrow from that solution space if you will.
Josh Phillips: The major one being isolation of course.
Carl Franklin: Right. In your design of parallelizing tasks that are in your application, would these agents have access to components that already exist in your application typically, or would these agents be subordinate to your components in your application, or both?
Niklas Gustafsson: We define two main isolation concepts. We have one we call a domain which is really the, if you will, the component concept. That’s the thing that draws the boundary around state.
Carl Franklin: Yeah.
Niklas Gustafsson: Which is typically what components in our model do. And the other pieces are the agents, and the agents are really the actors within the domain where the agent is acting on behalf of other components, if you will, accepting messages and manipulating domain state whether it’s actually writing to it or just reading it or doing something with it, sharing it with other agents. The agents are subordinate to the domain, and of course, there are other component models and runtime models that are farther on the outside of domains like, for example, operating system processes, we’re definitely subordinates to them, all code and windows are except maybe drivers; and we have also, of course, .NET application domains, which are in some way in between operating system processes and Axum domain in both cost and complexity.
The conversation continues online at http://shrinkster.com/17pz.
Carl Franklin
