Real old style engineering, for building things like a steel bridge over a river, is different from building a software application because you can see and touch a bridge whereas software is invisible. If somebody tells you they are making good progress, you pretty much have to believe them. You only find out when the delivery date floats by, the code is buggy and won’t run, or the users refuse to use it because it doesn’t do what they need it to do that you’ve been a victim of what I call the ‘tyranny of optimism’ (which can be understood as ‘don’t worry, somebody else knows what is going on’).
Software engineering tries to excuse itself by saying that it’s a young discipline so it can’t be expected to get it right yet. But software engineering has been going for 50 years or more, and in the same time, if bridge engineering hadn’t managed to get its act together people would still be using ferries to get across rivers; they would have concluded it couldn’t be done.
There have obviously been software successes. It’s said the computing power in the Nasa rocket to the moon was much less than the machine I’m using to write this. Some teams are better at delivering software than others. How would it be if some bridge builders built bridges that stayed up while others built bridges that fell down. If you needed a bridge built you’d know where to go – to the successful team.
It’s easy to know if a bridge is successful (i.e. it doesn’t fall down), it’s easy to see what progress is being made over time and it’s easy to know if the bridge is going to come in on budget . If you were on the team whose bridges kept falling down, you would probably go and see what the successful teams were doing that was so different from what you were doing. Instead of working from a drawing scribbled on the back of a cigarette pack, you might find they had a good cohesive set of engineering diagrams and conclude that these were prerequisites of project success.
In the days of old, it wasn’t the bridge architect that drew the diagrams, it was the draughtsmen. Draughtsmen were the power behind the creative vision. Draughtsmen studied long and hard to learn their craft. Draftsmen would sanity check a design. They would ensure the maths were right and the bridge wouldn’t collapse. They figured out how much steel and how many rivets to buy, they drew diagrams that explained the order in which things needed to be built. They drew the bridge from different angles, they drew it from the top and from the side to provide perspectives.
Software engineering needs good drawings. Up until now good diagrams have defeated the software engineering community. The state of the art can describe approximately what they should look like, but the state of the practice is to forget them all together. In the old days we used to use Data Flow Diagrams (DFDs) and other arcane representations. Programmers found they were of no use. There was no way to go from the drawings to the code – so nobody could see the point. Instead, sections of the IT community decided it couldn’t be done, it was a waste of effort so they dreamed up the Agile way of working and somehow this got mis-interpreted as meaning the drawings weren’t important. They never really said that, but it was easy for programmers who always hated doing drawings anyway to interpret it that way. So they didn’t do any drawings, they didn’t believe in capturing the requirements upfront, they just believed in programmers getting their heads down and working as a band of heroes to somehow build software systems in a warm fuzzy collaborative environment where the requirements would simply present themselves as a consequence of the programmers building something and showing it to the users. Think about it – what I’m describing is a process where the making of the diagrams is too hard, so nobody bothered. And if that had worked we’d see the crisis in IT project delivery solved, but that hasn’t happened.
By the model described in this paper, Agile can be broken down into Agile Delivery (writing code and sprints etc.) and Agile Requirements. Agile Requirements is an approach that requires the breadth of the requirements to be articulated. It does not require the depth of the requirements to be articulated in advance; depth (or detail) can be deferred on a ‘need to know basis’. Another way of thinking about this is as a ‘just in time’ approach to requirements representation.
It is possible to draw good diagrams, but it can’t be done without training aimed at doing exactly that. It’s not something you learn on the job. Good drawings are mostly done in UML and are drawn according to internationally recognised standards. The drawings show the system from different perspectives including both data and behaviour. All the drawings are related, they are all cross-referenced and they all tell a different part of the story, but it’s all about the same story and it’s clear how they are related, how one starts where another ends, or how one provides an input to another.
Engineering drawings are divided into static and dynamic models. The important views are:
- Data view
- Use case model view (the ‘epic’ view)
- Activity view (or the process models)
- The wireframe view
These views are used for a host of purposes. The epic view is akin to the ‘table of contents and is especially useful in:
- defining the user stories or iteration stories programmers use to write code against
- informing release planning
- informing project progress reporting against the project plan
- informing user acceptance testing
- defining ‘done’ i.e. when the system is finished
It takes education to produce software engineering diagrams, and it takes training and motivation to read them. To the uninformed a UML engineering diagram might be right or it might be wrong. The uninformed can’t tell the difference between good and bad. Diagrams that are not validated as correct and which do not go on to be used effectively are useless. I’ve coined the term ‘Analyst Draughtsmen’ to describe the people who produce the diagrams. Ideally, two analyst draughtsmen should work together to verify the diagrams one of them produces. The diagrams will go on to be used by the product owner, the project owner, the project manager, the software designer, the architect the programmers and the coders.
A sufficiently accurate representation of the requirements can be reliably constructed through the application of requirements patterns. All database driven applications are similar at an abstracted level and an understanding of the patterns allows the professional analyst draftsman (or woman) to create a requirements framework very early in a project, typically taking no more than six weeks. The approach taken is both top down and bottom up (meet in the middle) where a requirements framework is created using requirements patterns that are verified with the business through interviews/workshops. All that is required to apply the requisite requirements patterns is a natural language description of the system, ideally a scope document or project mandate. This is sufficient in most cases.
A typical IT team is a group of people educated as programmers and computer scientists brought together and led by business people. The business people receive no training in how they are supposed to manage the IT team. It’s a formula that shouldn’t work – and normally it doesn’t. Too often the project is reduced to mutual recrimination where the business blames IT for failing to deliver and the IT team blame the business for failing to engage with the project. But nobody’s trained the business to manage the IT project – that kind of training is not currently available. This is a big obstacle and a real opportunity for forward thinking educationalists.
In future I envisage a Masters level degree course with two components, the first exists to train business staff to manage IT projects. The second component is for business or technical analysts who want to become analyst draughtsmen. It doesn’t sound particularly high-powered, the role of draughtsman has never attracted great status, but it is where a real difference can be made in representing requirements properly and allowing managers to make informed decisions based on a common language of understanding. The analyst draughtsman is first a technically minded analyst who understands business process and is second an artiste with a CASE tool.
