Object orientation is more than a language primitive

Distinguishing object oriented languages from non-object oriented languages is like talking about free trade and regulated trade. All trade is regulated in both extreme and subtle ways; it's never free (think about slave trade, child labour, pollution control, safety, truth in advertising, and so forth). So called object orient languages provide primitives generally for just one object oriented abstraction; so for any given object oriented language there are many object oriented abstractions and features that it doesn't implement. All object oriented languages are just some person's or group's preferred abstraction (or merely easily implemented abstraction), not at all a universal truth.

I have a distrust of coupling my domain abstraction to language features. A lot of what a like about using functional style in imperative languages is that it helps me to recognise and explicitly represent some things that I might overlook if I just stayed in the mindset of the imperative language I'm working with. Object orientation abstraction is one thing regularly overlooked and left as a non choice. We generally fail to ask the question: what is the right way to use object orientation in this specific application?

It's important to me to recognise what different object oriented abstractions provide: different ways of expressing type relationships, different ways of sharing common code, different ways of protecting access to code and data, different ways to communicate between objects, different ways to get new objects, and so forth. With that understanding I want to build the most appropriate object oriented abstraction to get the benefits most relevant to my particular problem.

One tool to help reach this end is to sneer a little bit at the object oriented primitives in whatever language I'm using. By not letting myself lazily treat that set of primitives as automatically the right way to do object orientation I make myself ask valuable questions: what is the right way to partition code in this application, what is the right way of expressing type in application, what states or life cycle will my object pass through, and much more.

This has recently been brought home to me in discussions about programming Lisp. Programmers that I respect say they have no idea where to start writing a program in Lisp. I think they expect to have the language tell them to organise their code. But really in Lisp, beyond the beginners stage, you're job is to decide how to organise your code, how to represent object orientation if that's part of your application, or how to model data flow, or logical structure, how to represent what ever is important in your application.

In the JavaScript world there are a number of systems that offer support for building object hierarchies, providing abstractions that are generally more like Java's object oriented abstraction. This is a great thing if that object oriented model is the right abstraction for your client side programming. Sadly I think it's been done because people find it difficult to see object orientation as a choice of application specific abstraction rather than a constant of the tools they are using, and aim to make all their tools homogenous with regard to object oriented abstraction.

Oddly, the best object oriented code I've ever dealt with has been in C and Lisp, probably because the developers made choices about organising their code, and didn't treat the primitives of the language as the end of the story. Higher level abstractions of data flow, and of aggregating data and function, and object life cycle, were clearly chosen and expressed in code. Avoiding accidental design driven by language primitives.

Saying or acting as if there can be only one object oriented abstraction and one set of perfect primitives implementing it is frankly and obviously silly. There are many object orient abstractions out there being used productively every day. Good programmers should be able to choose the abstraction that's right for the application and implement it in the language that they are using for the application.

Dynamic dispatch is not enough to replace the Visitor Pattern

I've been bugged a little by how poorly understood the Visitor pattern is by people claiming it's not needed in their particular language because of some funky feature of the language. In the Clojure world dynamic dispatch or multi-methods are the supposed giant killer, (and I'm an Lisp fan of old, and Clojure is fast growing on me, this will be a rant about the fans who rail against an enemy and get it wrong).

The Visitor pattern has two parts, an Iterator and Dispatcher.

The mistake is that because Clojure has dynamic dispatch then it must be the dispatcher that gets replaced. In fact the dynamic dispatch really can help with the iterator, allowing different things in the structure to be treated differently. But as the example below shows it's not sufficient for some even more arbitrary dispatch types, not without adding extra information to parameters to facilitate the dispatch, which is an unnecessary added complexity; unnecessary added complexity being my basic definition of bad design.

My favourite example is a tree traversal engine, for something like an AST, with data like this: (def tree '(:a (:b :c :d) :e))

Now I want to do arbitrary things to this tree, for example, render it as JSON data:

{"fork": ":a",
 "left": {
   "fork": ":b",
   "left": ":c",
   "right": ":d"},
 "right": ":e"}

And also as an fully bracketed infix language: ((:c :b :d) :a :e)

There is absolutely nothing, but nothing, neither in the data nor the traversal that can be used to distinguish between rendering as JSON data or rendering for an infix language. This not a candidate for any sort of algorithmic dynamic dispatch based on any part of the data: the distinction is arbitrary based on functions chosen without regard to the data.

It's quite fair to note that there are two different types of nodes, branching and terminal, that need to be traversed in different ways, which can be usefully used for dynamic dispatch, but in that's dynamic dispatch in the traversal engine not the final functional/Visitor dispatch.

First define a generic traversal:

(declare ^:dynamic prefix 
         ^:dynamic infix 
         ^:dynamic postfix 
         ^:dynamic terminal)

(defmulti traverse #(if (and (seq? %) (= 3 (count %))) :branching :leaf))

(defmethod traverse :branching [[fork left right]] 
  (do
    (prefix fork)
    (traverse left)
    (infix fork)
    (traverse right)
    (postfix fork)))

(defmethod traverse :leaf [leaf]
  (terminal leaf))

Nice application of multi-methods and dynamic dispatch. Note that in the most recent versions of Clojure (1.3.0) dispatch defaults to a slightly faster common case of non-dynamic dispatch, so you have to declare the dynamic binding type. If you know you need dynamic dispatch you need to tinker with meta-data, and you can do that to other people's code from outside: (alter-meta! #'their-symbol assoc :dynamic true)

For ease sake I'm also going to provide a default set of behaviour, doing nothing. Notice the matching dynamic binding type specified as meta-data:

(defn ^:dynamic prefix   [fork] nil)
(defn ^:dynamic infix    [fork] nil)
(defn ^:dynamic postfix  [fork] nil)
(defn ^:dynamic terminal [leaf] nil)

Now to get it to do something real, rendering in a fully bracketed in infix style. Notice that binding doesn't need explicit dynamic meta-data, because dynamic is the heart of binding:

(binding [prefix   (fn [_] (print "("))
          infix    (fn [fork] (print " " fork " "))
          postfix  (fn [_] (print ")"))
          terminal (fn [leaf] (print leaf))]
  (traverse '(:a (:b :c :d) :e)))

How about printing the symbols space separated in reverse polish notation, as if for a stack language like PostScript or Forth. Notice how I only need to provide two of the four possible bindings thanks to the default handlers.

(binding [postfix  #(print % " ")
          terminal #(print % " ")]
  (traverse '(:a (:b :c :d) :e)))

The fully bracketed rendering illustrates one other very naive mistake with many peoples understanding of the Visitor pattern. Some nodes are the data to more than one function call during the traversal, the distinction being dependent on the traversal pattern and the nodes location in the structure. The final dispatch to the is not determined by the data of the node in isolation, the individual node data being passed to the function is insufficient to choose the function.

There are some useful applications of multi-methods in the final dispatch, for instance to distinguish between different types of terminals as in this example:

(defmulti funky-terminal #(if (number? %) :number (if (string? %) :string :other)))
(defmethod funky-terminal :other [terminal] (println "other: " terminal))
(defmethod funky-terminal :number [terminal] (println "number: " terminal))
(defmethod funky-terminal :string [terminal] (println "string: " terminal))
(binding [terminal funky-terminal]
  (traverse '(:a (:b 42 :x) "one")))

But that dispatch isn't the whole of the Visitor pattern, and frankly to hold it up as the Visitor pattern only demonstrates that you haven't grokked the complexity of dispatch from a complex data structure.

Consider another typical use of the Visitor pattern: traversal of graphs. Let's propose a traversal of a graph that is is guaranteed to reach all nodes and to traverse all edges while following edges through the graph. There are many graphs where this would require visiting some nodes and some edges multiple times. But there are many applications that would only want to give a special meaning to the first traversal of an edge or the first reaching of a node. There may even need to indicate the traversal has restarted discontinuously in a new part graph in those cases were there is not edge leading back from a node or where the graph is made of disconnected sub-graphs. Nothing in the node itself can tell you that this is the first or a subsequent reaching or traversal, much less that the traversal has had to become discontinuous with regard to following edges, (maybe you could infer that from reaching a node without passing an edge, maybe you could track nodes and edges, but why do all that work when a good visitor will tell you directly by calling a suitably named functions; if we're going to enjoy functional programming, lets not be ashamed to use functions in simple direct ways). All these different function calls represent different targets of dynamic binding in the model I've presented her. Of course you could use multi-methods to achieve dynamic dispatch if there are different types of nodes for example. But it would still be the Visitor pattern if there was only one type of node and no reason for dynamic dispatch.

Dynamic dispatch is a powerful feature in Clojure, but trying to justify it as a giant killer to the Visitor pattern shows a lack of understanding of Visitor pattern, and weakens the argument. Present the languages strengths, not your own weaknesses.

A few of my favourite programming heuristics

There are many aphorisms and heuristics that can helpfully guide programming. Here are four that mean quite concrete things to me, and that have been characteristic of easy to understand but practical and performant implementations during my career.

1. Clear object life cycle
2. Analysis and action are separate concerns
3. Single direction of flow
4. Hide your technologies

(1) and (2) are particular takes on the single responsibility principle, focusing on the system, at any point in time, be doing one thing. (3) has that in it as well, do one thing and then move on to the next; design so that you're not coming back with feedback and then branching in response to intermediate states. (4) is addresses the laziness that accompanies excitement about the shiny new stuff; it's new, it's great; what could possible be wrong with letting it touch everything, (languages and frameworks being the most dangerous shiny stuff because they are hard to contain).

Know the life cycles of your objects and make them clear. Consider an object system that involves building up a web of objects and then sending it request. It's really useful to draw a sharp line between the phase of building up the web of objects and then subsequently triggering operations. Keep them separate. Where possible try not mutate the web of objects so the system is less likely to wander into invalid arrangements. Keep mutation separate from queries about it's present state or asking it to perform it's work. Explicitly representing object life cycle stages and the transitions and causes of transitions between them is one of the most powerful types of self documenting coding style I've every encountered.

Getters and setters are the enemy of the well managed object life cycles. The classic (anti-)pattern of creating an object and then using setters to tweak it into a valid state means you created an object in an invalid state. You really don't want your objects to have stages in their life cycle when they are invalid; don't allow object tweaking: ban setters. Getters mean other objects can react to what ought to be this object's internal representation, making it hard to change that representation. In messaging terms freely accessible getters are one step away from public attributes, broadcast messages, with each one exponentially increasing the combination of state information available in your program; don't feed runaway complexity: hide state, ban getters.

Many objectives require analysing a situation and acting, but analysis and action are very different beasts and it's best to keep them separate; keep two clear phases that carry you to your goal. Separation of analysis and action is my reaction to the nasty anti-pattern of getting a little information, doing a little work, getting a little more information, branching off somewhere, doing a bit more work, getting some more information. That way of programming means you need to know the history of the processing that got you to the interesting point you're trying to understand. My preference is to take time to figure out the current state, then act on that knowledge. Planning complex mutations or the construction of complex new objects and object collections this way is excellent; like the builder pattern, gather all the information, then act on it; similarly the interpreter pattern, build up a set of actions, then execute those actions.

Single direction of flow, or pipelining, is more of the same really. Avoid going up and down the stack repeatedly, and avoid too much back and forth in complex dialogues. Tell the down stream process or function everything it needs, how to handle errors, where to send results; fire your message; and turn your back on it. It requires you to more explicitly represent different states and possible reactions to those states. Frankly it's easier to trace and debug, less of the nonsense of being in very different states from one line to the next, less needing to know the accumulated side effects of the lines of code in front of you. Further, if you start with a model like this it's much easier to scale via message passing systems, a real win if you're actually planning to be successful.

You use many complex technologies, keep each one well contained. Hiding your technologies is about keeping separate different kinds of complexity. Don't get trapped in a framework, don't let someone else's technology dictate how you organise and present your code. You have complex business logic, represent it clearly. It is almost never the case that your choice of technology is the same as your business logic, so don't let them blend together. Don't design objects that expose they way they are being persisted, (the fact that they are being persisted is clearly a requirement and should be represented, but not specific how).

We all have different experiences and abilities and they shaped our personal measures of quality and our personal rules for achieving quality. It's good to know the values you hold and what in your history led to you holding those values. These four principles deliver their primary value by making large bodies of code easier to understand and particularly easier to debug and extend. I've spent too many hours puzzling over code, particularly stressful hours debugging code or trying to feel safe putting in place what ought to be a simple fix. They work by making temporal interactions simpler, making it clear what has been done, what is known, and what remains to be done. They're shaped by both success and failure at working on some very large bodies of code, and working on the same bodies of code for long periods as I tend to keep my jobs for several years.

Low Level Leadership

By dint of having more years of experience than many of my colleagues I get to be a leader, not as a formal position, but as an inevitable fact of humans in groups. I've been thinking about this for a while and I wonder about what a leader from within should be aware of in order to support those around about. These questions capture some of the things that I try to notice about my co-workers.

• What do they fear most, or what makes them insecure in their work?
  
• What would this person like to do better?
  
• Of the things that need to be done what do they most enjoy?
  
• What do they find most disappointing?
  
• Should I or could I do what I've recommended that they do?
  
• What grabs the attention of this person and what do they miss or ignore?
  
• What factors or environment surround their best work, (collaborators, location, tools, time)?
  
• What do they think is their best work?
  
• What things most regularly slow this person down?
  

There's another angle to good leadership and that's to model good practices and behaviour. So I think asking myself these questions would be a good way to spend some time this weekend.

Thinking about some things my personal trainer in the gym has been asking I probably need to help him answer similar questions about me as my leader in that context.

The Right of Veto

I'm trying to get my head around the question, "Why do good developers write bad code, and what can we do to fix it?" It's provoking dozens of ideas. This one came to me yesterday:

All developers should have the right of veto on any code in development. Developers should be taking the time to look at the rest of the team's work, to ask the questions, "Do I understand this?", "Could I maintain this?", and "Do I know a better way to do this?". If there's a problem in understanding, or a better way to do it, then exercise the veto.

Exercising the veto means that code doesn't get committed, or should be reverted, or must be improved.

Exercising the veto means you must now get involved in making it better.

Pairs of developers can't pass their own code, they need to get someone else to consider their code and possibly veto it.

No one is so senior as to be above the veto.

If there is disagreement between the code author(s) and the developer exercising the veto then don't argue, bring in more of the team. Perhaps the resolution will be to share knowledge better rather than the veto. But ideally you want to write code that any other team member will readily accept.

---

As I was thinking of this and talking about with my pair I realised we were writing code that I felt I would want to veto, so the next idea is that every developer or pair announces the state of their code. I plan to stick a big green or red button on top of my monitor; the red button means I veto my own work, and green means I think this is okay.

Self veto, (the red button), is a visible an invitation to help, work with me, pair with me, take me aside and talk about design or the domain or our existing code base. Open to veto, (the green button), is a request, please look at my code, judge it, share in the ownership of it, take responsibility for the whole teams output.

One of the threads that led to this idea came from circumstances that have led to us pairing less. My response was to ask what else we could do to improve code quality: how do we conduct worthwhile code reviews.

I'm heading toward the idea that even pair programmed code could benefit for code review, but I want to make it as light weight as possible. And the real point of code review, the thing that makes it meaningful, is the possibility of rejecting the work. Otherwise it's no more than a poor tool for sharing knowledge.

That realisation that code review is most valuable if it can be acted on tied in with a quote I read, "If you can’t say “No” at work, then your “Yes” is meaningless." The simplest "no" in code review is the veto on commit.

---

When working in sprints or iterations there's often a step where the developers commit to doing a set of tasks. But I've never really been in a position where I thought I could say no. The best has normally been to say that it will take to long or to negotiate details, but at the end of the day we assume that the right things are being asked for and we try to do them.

It's not an ideal commitment if I can't say no. Can my commitment be real consent if I can't say no? Certainly there have been times when I thought the requirements were unwise, and I've been demoralised doing what I thought to be the wrong thing. Could we create a team, or a work flow, or a process, that really empowered developers to say, "No, we cannot commit to doing that", and be honoured for it?

XP is more than Pair Programming and TDD

There are 12 practices in XP and 5 values. The values are open to interpretation but the practices are pretty clear. Many people magpie practices and while the practices are often useful in isolation there is a synergy between them which can be lost.

Coding Standards is one of the practices. To me it encompasses more than the simple things captured by findbugs, checkstyle, and source code analysis tools. It goes beyond naming standards, line lengths, function and class size, and even sophisticated measures like fan out, cylcometric complexity, and whatever other panaceas.

Coding Standards should bring us toward principles like: any developer should be able to understand the code others have written; our implementations should correspond to the way we talk about our system; a developer should be able to figure out requirements from the written code; it should be easy to represent or articulate what we have implemented so that we can meaningfully reason about it and reliably build on it.

It's frequently not so. Frequently TDD and Pair Programming produce code that only makes sense to the developers writing it while they are actively working on it, sometimes developers don't understand what they're doing even as they satisfy tests. Tests frequently capture behaviour that is incidental to requirements. Developers hunker down and satisfy requirements expressed in tests but don't look up beyond that narrow focus.

Failing to attend to Coding Standards is failing to do XP, or at least failing at the whole cloth, and undoubted exposing development to greater risk of producing poor quality results.

There's nothing wrong with building up a personal or local process by picking elements from different development systems, but sometimes it feels like those personal processes are about making life easier by rejecting or neglecting things that require active practice to get substantial quality.

Coding Standards, don't leave home without them.

Implicit and Explicit

Each step of language design gives us tools that make more behaviour implicit, except some things that languages make implicit are really requirements better made explicit. So another thread in language design is to make it possible to be explicit about the things that matter.

The three basics of structured programming are sequence, selection, and repetition. I'm going to use them to discuss how languages make things implicit or explicit.

Selection

The basic tool of selection is the if statement which connects different states to different code paths:

if (condition) then

    fnA()

else

    fnB()

The condition inspects available state and in some states selections fnA and in others fnB. This is an implicit relationship. It is a coincidence of placing these statements in this way. If we had to make the choice again in some other piece of code we could easily get it wrong, we could construct the condition badly, or forget we had to make a choice at all and just call one of the functions, or perhaps not even have access to the state required to make the decision.

The use of polymorphism in typical OO languages makes the connection between state and function explicit.

obj.fn()

The relevant state is represented in obj and how it was instantiated. There are in principle different types of obj, but for each one there is a correct version of fn and that is the one that will be called. It is explicit in the fact of obj being of a particular type that a particular fn will be called.

Repetition

The basic tool of repetition is the loop:

while (condition) do

     fn()

done

But we do many different things with these loops, we might for example iterate over a collection of numbers and sum them. Or we might make a copy of that collection while translating each value in some way. Many languages provide constructs that make the meanings of such loops explicit while taking away all the accidental machinery of loop conditions and so forth.

collection.reduce(0, lambda a x: a + x)

or

collection.map(lambda x: fn(x))

Given constructs like these there's no possibility of missing elements because of the condition being wrong, and by drawing on the well established vocabulary of list processing the meaning is made clear.

Sequence

This is the tricky one, it's an implicit behaviour to which we are so accustomed that we rarely notice we've relied on it. Many bugs happen through accidental dependency on implicit sequence. Consider:

 def fn()

    stat_a

    stat_b

end



stat_1

fn()

stat_2

We all know that stat_2 is the statement that will be executed after stat_b. First, what else could it be?

In a parallel environment if could be that statements at the same level in a block are executed in parallel, perhaps interleaved, but with no guaranteed order. Simple as that, and very important given the rise of multicore devices.

In such an environment we would need to make sequence explicit. Consider a construct that forces consecutive execution in a parallel environment:

stat_a ::: stat_b

So stat_a will be followed by stat_b. You might also find things like this in functional programmings in the form of monads.

Another form of explicit sequence is the continuation, a mechanism for representing where the next line to be execute is to be found, and a matching mechanism to actual go there. These mechanisms often look like a type of function call, but one for which there is no notion of return, which results in no need for a stack. I admit continuations are one of the weirder twists in computing languages, it can be hard to know when to use them, but I've found they become useful in patterns where you manage the calls, perhaps for deferred execution, or to trap exceptions.

There are other interesting constructs that can help express important, required, sequential operation. With resource management that execute a block of code and then guarantee that after the block something particular will be called. For example the automatic try with construct in Java 7, or perhaps yielding to a block in Ruby.

Don't look at me like that...

It might seem that all the simple forms that I'm calling implicit are spelling out the steps more clearly and must surely be explicit. The trick is to think in terms of required behaviours: if you have a required behaviour how easy would it be get it wrong with the simple forms? A statement shifted to the wrong place, a sign wrong in a condition, and generally nothing in the simple form to tell you what the right form would have been, what was the intention of the construct. The meaning is implicit not explicit.

Conversely, if you use the explicit forms when sequence, repetition, or selection are truly requirements and you will be announcing your intentions very clearly. You'll be making it easier for libraries and languages to do clever optimisations to make use of parallel processing or for safe and efficient use of memory and other resources.