Chaotic Programming

Introduction

It has been known for many years that developing large software systems is far more
difficult than developing small ones, but why is this the case ?

Over the years we have developed many improvements to the process and many new
techniques designed to allow the construction of more complex systems, but all that
seems to happen is that we plan even bigger systems and end up with budget over-
runs and failed systems, sometimes on a truly monumental scale. We have moved
from unstructured assembly language, to structures languages (Cobol, C) to object
oriented languages ( Java, C++) and now to SOA techniques, but at each stage we
have projects that seem to be too large to succeed.

The Model

We can describe the work done by a software development team as a function of the
requirements for the project.

or, the rate of development (dw/dt) is a function of the difference between the
requirements and the amount of work done so far. This is just a statement that work
begins quickly and tapers off as it approaches the final state.

If we characterise a large system as one that has several development teams working
in unison then we can generalise the above as applying to each team, but with an
additional factor that depends on the state of all the other projects.

where the second term expresses that there is some coupling between this component
project and each other component project, and we only know the state of the other
project as it was at some previous time.

The above expression is a system of "Delay Differential Equations" or DDEs.

In the terms of a software developer, what appears to happen is that as each other
component project publishes its state our project has to modify its code to resatisfy the
new interfaces. The larger the interval between the other projects publishing the more
out of date is our view of their interfaces, and the more work we need to do correct
the situation.

Chaos

A DDE will often have a solution governed by Chaos Theory. As the delay increases,
the solution will initially settle to a single state, then switch between an increasing
number of states, and finally become seemingly random.

It seems reasonable to assume that similar behaviour can be expected from an entire
system of DDE's, as we have in a large software development project.

For small projects where the delay is near zero (i.e. the developers have good
communications) the solution will quickly settle to the desired state. On huge projects
with large delays in communications the solution may never stabilise to a stable state.

Mitigation

The above formula provides clues to several ways to handle large projects.

Reduce the work required: By using efficient development techniques the amount
of work required to implement the project can be reduced. This lessens the impact of
the other component projects, and can reduce the number of other component projects
required. However, what often happens is that more efficient development techniques
just lead to more ambitious projects.

Reduce the delay: If every component project publishes its state at frequent intervals
then the delay is reduced, and hence the chances of a chaotic solution are reduced.
Some large projects do nightly builds in an attempt to minimise the delay. However,
the delay is not just the time to publish, but also the time for the other developers to
absorb the changes, and understand the implications.

Decouple the projects: If the coupling between the component projects ('a' in the
above expression) can be reduced the chances of falling into a chaotic solution will be
reduced. This can be achieved by using well defined, and stable, interfaces between
the component projects.


This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.