Variance expression in tell don’t ask programs

LMAX Exchange

This time we’re going to explore how tell don’t ask (TDA) programs
express variance, and hopefully come across an interesting corollary
of our TDA definition.

We’ll consider two ubiquitous operations: getting and storing a value
into a map. In both cases we’ll discuss the standard Java
implementation before creating a speculative TDA version.

Example one: value retrieval

We want to lookup the value associated with a particular key.

This operation has entropy two. Either the map contains the value or
it does not.

java.util.Map

V get(Object key);

Here the returned value is either:

• null to signify absence
• the present value

Amusingly nothing prevents us from storing a null value other than
some very hopeful javadoc.

Clearly this interface violates our TDA definition.

• Should we be perturbed by this?

If such a core library concept is against us, perhaps we’ve chosen a
poor language as for TDA implementation; Java, after all, is
frequently mocked for not being a tremendous OO language.

What about a lauded OO language, like Smalltalk? Well, Smalltalk’s
dictionary class still provides value access via at:, which generates an
error if a value isn’t present.

Even the more robust accessor

at: key ifAbsent: action

still returns the present value, rather than forwarding a message to it.

Let’s continue to persist with our TDA approach and see what happens.

TDA like we mean it

void forward(final K key, final Consumer<V> onPresent, final Runnable onAbsent);

Now we obey our TDA definition and end up passing in two more objects.

There’s that number two again! It sounds like we’ve expressed the
variance in function parameters – we’ve given the map two functions
that it could call depending on whether a value is present or not, and
we trust it to call only one of them.

Perhaps we could express the TDA variance as follows:

variance ⊂ downstream call graph

Example two: Storage

Now we want to put a new value into the map.

Once again, the entropy of this function sounds like it might be two:

• There could be a value already present for that key; we overwrite it
• There is no existing value and we just add the new key value pair.

In both cases the state of the map changes.

Java standard

V put(K key, V value)

Once again, not quite tell don’t ask compliant.

Side note: If one looks at the javadoc on java.util.Map, one notices
that with the addition of exceptions and the possibilities of null the
entropy is now far higher than two.

TDA like we mean it

void put(K key, V value, Consumer<V> oldValueExisted, Runnable noExistingValue);

Here again we have two extra parameters.

• Does this new definition express the complete variance?

N.B We could argue that oldValueExisted and
thereWasNoExistingValue don’t have access to the new key and
value. Nothing in the TDA definition prevents us from binding this
information into the implementations at construction time, though.

What’s really missing is the variance in the map’s state. This is
left implicit – the caller still has a reference to the (now mutated)
map, and the old version is gone, forever!

What if we wanted to use an immutable map in this style? A naive first
attempt might look something like this:

Map<K,V> put(K key, V value, Consumer<V> onOldValue, Runnable noExistingValue);

The map decides which function to call next, but must also provide a
new map to pass back to the original caller. This violates our TDA
definition, however, and if either of the two callbacks want to play
this game, we’re suddenly in trouble (they could be longer lived
objects than a simple closure – how do we update their callers with
their new values?).

It’s hard to see how we can scale this approach.

Conjecture: TDA mutation expression

What a pretentious subheading.

void put(K key, V value, Consumer<V> onOldValue, 
         Runnable noExistingValue, Consumer<Map<K,V>> onNewMap);

Now we’re fully tell don’t ask compliant, but what are we going to do
with that new map?

Well, temporarily ignoring TDA, a caller could pass themselves as
onNewMap. It could then tell its callers that it has changed, and so
on, up the stack.

• Can we do that and keep to the TDA definition?

We need the traditional extra layer of indirection.

Messages outgoing from our object can no longer be delivered directly,
but will need to be captured/dispatched by some piece of architecture
that knows where messages should be routed to and delivers them
appropriately.

What’s the tapping noise I hear? Ye gods, we’ve just dug up the
foundations of the actor model.

So, we have two choices:

• Keep direct dispatch, but leave state change implicit
• Abandon direct dispatch and build towards an actor like model.

Very important scoping note

In order to achieve TDA certification with the actor model, we’ll
actually need to abandon synchronous dispatch (and qualify the TDA
definition slightly).

Precisely why this is necessary is left (for now) as an exercise for
the reader. We’ll spend the rest of this post discussing simple,
direct dispatch TDA, and leave discussion of the actor model for
another day.

The Consequences Of Implicit Mutation

This really ought to be the name of the next Blotted Science album.

We have roughly agreed that in TDA world:

variance ⊂ downstream call graph + (implicitly) mutable object state

This is perhaps more elegantly expressed as:

variance ⊂ downstream call graph for current method invocation +
           downstream call graph of future method invocations

That sounds quite worrying, no? We could see the variance of a call at
the point of any future invocation. That’s a helluva lot of variance!

That is true, but our insistence on TDA at least allows each object to
know precisely what messages it might get passed (and thus the possible
states it can get into).

This fits with my experience in this area; as soon as we disregard
TDA, understanding (and changing) code becomes extremely difficult.

On the up side

We do gain some immutability elsewhere. A large chunk of our object
graph often is fixed at start time (spot the final fields of a given
object).

For example, anyone using our TDA map could hold on to it for the
entire program runtime, even though its contents (and thus behaviour)
will evolve over time. The TDA map feels more like a subscription
registry.

Conclusion

Caveat implementor!

TDA as we have defined it leads inexorably to mutable objects.

The only way to be sane (citation needed) in a mutable world is to
encapsulate decisions made about state to the places where the state
is. In a way, TDA is both the cause and solution of all of our
problems.

Coming up

We are yet to discover the limits of TDA, and we haven’t really
explored the effects of TDA at anything other than the unit level.

If TDA is such a good idea:

• Why don’t library classes (like Map) obey our definition?
• What effect does their lawlessness have on code that wants to be good?

We’ll try to make some headway on this soon.

In addition, we also need to cover:

1. Variance expression and encapsulation in functional programs (FP).

   This should lead to a nice lemma on nesting FP in TDA (and vice versa).
   We’ll probably go here next, as a brief diversion (not least because
   I just extracted most of it from this post).

2. The actor model