Saturday 20 November 2010

Clojure's Time/Concurrency Model - A Gentle Critique

While at QCon I sat through Stuart Halloway's talk on the Clojure time/concurrency model, it was very interesting. I watched a copy of Rich Hickey's talk on the same subject some time ago. I'm not going to rehash the entire model here, if you are unfamiliar with it, best to head straight to the source. However, I am going to offer a few (small) criticisms. In fact they are not criticisms of the model but more of the way it is presented.

First off I would like to say that I think the approach the Clojure guys are taking is excellent. I am currently playing with a small prototype application that is based on similar principals. Admittedly I'm using Scala rather than Clojure, but it just shows that their model can be generalised to other languages easily.

Focus On The Model

One of the enabling features of Clojure's concurrency model is the Hash Array Mapped Trie, which allows for a path copy based structure to be used for persistent vector and dictionary type structures. What was not presented during the talk - maybe all Clojure developers know this already - is how the path copy metaphor can (and should) be extended to your entire object model.

Consider an account management service that provides a function for updating an individual account's post code. An object graph for such as service could look something like this:



After an update to the post code for a specific account - using immutable objects to represent the model - the resulting object graph would like the following:



This closely follows the pattern displayed when updating one of the hash tries (q.v.) and retains the property that readers will always see a consistent view of the model no matter which part of the model the reader holds a reference to. If the reader needs a more up to date view of the object graph, it will have to re-enter the model through the accountRef atom. This brings me to my next point.

Identity vs. Entry Point

One of the questions that I asked was around whether there was any real applications built using this model. The response mentioned 2, one being a web framework. However, in both cases those systems only had a single reference, i.e. a single identity. When considering the concurrency model, this makes perfect sense, but from a data modelling or a domain modelling perspective the concept of identity is closely tied to the notion of entities. This use of terminology suggests that you implement a system that puts every entity behind a reference and while this may sound appealing initially, it has 2 negative effects. Firstly is clutters your domain model with an artificial construct, mixing an infrastructure concern (concurrency) with your domain logic. It is generally accepted that separation of concerns is a good thing, so heavily mixing concerns can be considered as bad. The second issue is that an operation that spans multiple entities is difficult to make consistent if all of the entities have individual references. For example, reading threads will be able to see the result of partially applied operations, unless you apply some extensive and complex bookkeeping to ensure that references are made visible in the right order. There is also a performance cost, but I talk about that later.

Using 'Identity' feels wrong as it adds confusion due existing definitions and/or usages of the term. I think a better term is 'Entry Point' or from Domain Driven Design 'Aggregate Root', as this is closer to what actually happening when the code interacts with the model.  Another option would be to break the strong linkage between the concept of Identity and the use of Refs to represent them.  Using the account service example above, the account repository provides a point with the domain model that code can enter and then reach other entities with that model. Maintaining the reference at the level of the repository allows operations that modify an number of entities that exist below that aggregation point can be made visible as a single atomic action, providing simple, clean transaction semantics.

It's Not Free

One of the statements that irked me the most was around that using Atoms, STM or Agents from a read perspective is free. It's fast, cheap, non-blocking, runs in user-space, but it is NOT free. Using Atoms as an example, the swap! function on the Atom uses an AtomicReference to compare and swap the values after a change has occurred. On the metal this is using a machine level compare and exchange operation (on Intel this is a LOCK CMPXCHG). In order to ensure visibility of the changes the CPU has take out a memory bus lock (or cache lock on newer x86 CPUs) and flush the pipeline. Therefore if your reading thread happened to try and dereference the atom (or potentially any other operation) it won't be able to have its load instruction pipelined along with the write. The slow down is small (and getting smaller on newer CPUs, e.g. Nehalem's CMPXCHG instruction is 40% faster than Core 2) but can't be considered cheaper than a normal non-volatile object reference. A reference within an AtomicReference is declared volatile.  Volatile variables are accessed differently to standard variables in that the JVM generates instructions that enforce ordering, which restricts both the compiler's (Hotspot) and the CPU's ability to optimise said instructions.  I have anecdotal evidence of code littered with volatile references slowing down significantly.

The other area around performance is the use of completely immutable structures to represent your domain model. Before I get flamed into oblivion, I'm not going to make blanket statement that mutable structures are faster than immutable ones. Before making a judgement it is worth ensuring you understand the behaviour of your own program, specifically the read/write bias. If you have a very high write bias (like in a financial exchange) there is cost to using pure immutable structures. There is a significant memory allocation and copying hit on a write, plus the system will create a lot of garbage (which may cost you in GC pauses). As operations within your application shift toward a read bias, then immutable structures make a lot more sense as the data can be shared.

5 comments:

Unknown said...

Hi Mike,

Your point about reads not being free is fair. I should say that reads are as free as perception can be. That's still a pretty good deal, since the generic ability to read information is nonexistent in mutable OO languages, until you layer in some kind of concurrency protection. But not free. :-)

I think you have the separation of concerns argument completely backwards, however. Reference types quite literally separate a concern: identity. I also think that treating concurrency as an infrastructure concern is a mistake. Concurrency is a basic fact of many domains, and we should not be confused by our infrastructure tools, which are perpetually lagging our needs in this regard.

You say that an read operation spanning multiple entities is difficult to make consistent if each identity is a reference. I am not sure why this seems difficult--transactions do the bookkeeping, and scoping reads in a transaction is trivial to do. So maybe the better questions is: difficult compared to what? How would you solve this problem *without* references and transactions?

Stuart Sierra said...

"The second issue is that an operation that spans multiple entities is difficult to make consistent if all of the entities have individual references. For example, reading threads will be able to see the result of partially applied operations...."

Not with Refs, which can only be modified within a transaction. Transactions are atomic. Other threads will never see partially applied operations.

http://clojure.org/refs

Michael Barker said...

I think my where my greatest confusion lies is the the notion that all and only entities have identity and identity is always and only represented using a reference (I am happy to be wrong about this). This was the understanding that I came away with after your presentation (and Rich Hickey's earlier one). As I model my current business domain (financial exchanges) using the Clojure approach, I find myself moving references about based on my concerns around concurrency, consistency & visibility not on my understanding of the business.

As I do this, based on my understanding, I am changing the definition of what things are entities. This worries my as I am letting the implementation dictate my model. Like the Smalltalk guys said - "Model First". I found that when (in my head) I broke apart the strong relationship between identity and reference, everything became much simpler. Using a reference to represent an entry point to the model, meant that I could separate out those concurrency concerns from my domain model concerns. At implementation time, simply pick an appropriate aggregate root below which, I want to enforce consistency and apply the reference there. Not worrying about the conflict that my notion of an Entity (defined by DDD) was different to Clojure's notion of an Entity helped. This is why I said that I found the Clojure definitions of Identity and Entities confusing. I think they will also prove confusing to those who already think using Domain Driven Design or come from a heavy data modelling background.

How would you solve this problem *without* references and transactions?

I would still use references (specifically agents with a single writing thread) and like I mention above chose an aggregate root within my model to serve as an entry point.

I will do a another post soon focusing on the specifics on the app I'm toying with...

Like I mentioned in the blog post, I really like the concurrency model, I find terms screw me up.

Couple of other small points:

the generic ability to read information is nonexistent in mutable OO languages

I definitely agree with the mutability part, but OO is neither here not there. I can screw up just as badly using C.

we should not be confused by our infrastructure tools, which are perpetually lagging our needs in this regard

I'm not so sure that we are. I think we are more limited by our own tendency to over-complicate, not clearly model our problems and not really understand how our infrastructure works. E.g. the majority of developers mess up with multi-threaded code because they don't understand how memory works.

jozef.wagner said...

"This use of terminology suggests that you implement a system that puts every entity behind a reference and while this may sound appealing initially, it has 2 negative effects. Firstly is clutters your domain model with an artificial construct, mixing an infrastructure concern (concurrency) with your domain logic. it is generally accepted that separation of concerns is a good thing, so heavily mixing concerns can be considered as bad."

Try not to think of a reference as a concurrency construct, but as a identity ("mutable") construct. That way IMO you can in Clojure explicitly separate immutable and mutable world and you achieve a better separation of concerns as in the traditional approach (separate mutable variables and concurrency artifacts)

Michael Barker said...

Try not to think of a reference as a concurrency construct, but as a identity ("mutable") construct.

This is where I think the terms cause confusion. Identity (and entities) are domain modelling concepts, where as immutability/mutability are implementation details.