Careers: Interviews
Software Development Expert: Andy Carmichael
This week, Stephen Ibaraki, I.S.P.,
has an exclusive interview with, Dr. Andy
Carmichael, an internationally renowned expert in
software development methods and processes with more
than 20 years experience in the industry. His most
recent book, co-authored with noted J2EE expert Dan
Haywood, “Better Software Faster,” is most
definitely a ‘must’ read for all software
architects, project managers and developers and the
only book available that discusses their
“single-source model” approach.
Discussion:
Q: Your list of accomplishments are impressive
indeed! We thank you for sharing your years of
experience and knowledge with us.
A: No need to stress the years. The grey hairs in
the mirror are enough of a reminder!
Q: What do your partner and three daughters think
about the attention you are receiving as a noted
author, speaker, consultant, and internationally
known guru?
A: Sharing my life with four wonderful young women
is a privilege, though I don’t think it’s ever
crossed their mind that I’m a guru! They are all
involved in music and the arts and are only really
interested in software when the computer they are
using for school work, communication or recreation
misbehaves. At this point software consultants are
best advised to keep a low profile rather than boast
of their skills!
Of course it should be a wake-up call to everyone
involved in software when they come across users –
especially ones they love – who are less than
satisfied with the general quality of software! At
least it encourages me to keep up the search for
better ways to produce it.
Q: Can you describe what led you into the computing
field and lessons you learned along the way?
A: Like many of my peers I didn’t start out studying
software or computer science, but in using software
to do something useful – in my case trying to
calculate stress levels in aircraft components with
holes in. This was a PhD research project in
Mechanical Engineering at Southampton University in
the UK. But soon afterwards I found myself working
in a larger commercial team, where the problems of
how programmers worked together, shared changes to a
common code base and used tools to measure and
improve quality, often outweighed the problems of
how to program a particular algorithm or subroutine.
This is when I first got involved in the evaluation
and specification of software methods and then
software development tools. Probably the most
important lessons I learned in those years were the
importance of the foundational principles of
software engineering: abstraction, modularity,
information hiding and dependency management (loose
coupling and high cohesion). When object-oriented
design and programming became a commercially viable
prospect towards the end of the ’80s, it was clear
to me that they added important mechanisms to
improve the way we apply these principles, which was
why I wanted to be involved in the exploitation of
object-oriented methods.
Q: You founded Object UK Ltd. Can you tell us more
about this company and its purchase by TogetherSoft?
A: I founded Object UK in 1992 in order to provide
software tools and services to support
object-oriented development methods. By then I had
been working with CASE tools (Computer-Aided
Software Engineering) for a number of years and I
felt the emerging object-oriented methods would
provide much needed impetus to this aspect of the
industry. This proved the case, though the most
significant development for Object UK was when the
main product we were distributing, Together�, was
rewritten in the new Java language and took the Java
world by storm with its “single-source” approach.
Basically it bypassed what had been the major
Achilles heel of CASE products – the need to
synchronize design and code through what is termed
“forward and reverse engineering”. Not only is this
difficult for tool vendors to implement, it’s
incredibly difficult for developers to use
effectively. Together overcame this weakness at a
stroke by using the same source file – actual Java
or C++ code – to store the models and diagrams that
designers and developers drew and updated
graphically. It was a master-stroke and one that
even now the competition has not matched.
The main difference it made to us, as more and more
development teams wanted to buy into the new
approach Together had introduced, was that we found
our whole business was now focused around this
product. Since TogetherSoft wanted to have its own
UK presence it was natural to begin talks to bring
Object UK into the company that owned Together. We
signed this deal in November 2000 and I then took on
responsibility for growing the European team of
mentors who would train and consult with the teams
developing software with Together in Java, C++ and
more recently C# and VB. Soon after that was also
when Dan Haywood and I started work on the book
Better Software Faster to provide guidance to
development teams looking for new techniques to
accelerate quality.
Q: You also edited two books for SIGS and Cambridge
University Press—Object Development Methods (1994)
and Developing Business Objects (1998). Can you
share some tips from this experience?
A: Both these books were collaborations with other
authors who were contributing to the
commercialization of object-oriented approaches to
software development. The early ’90s was a time when
new object-oriented methods were being published
almost every month! The situation was very confusing
to project managers who just wanted to adopt best
practice, rather than have to assimilate scores of
different notations and processes then choose one
or, possibly worse, invent a new one by combining
different set of features from other published
methods. The industry’s response to this situation
eventually saw the emergence of a standard notation,
UML (the Unified Modeling Language), first published
in draft form in 1997 by Graby Booch, Jim Rumbaugh
and others. In 1994 though my aim for the Object
Development Methods book was that it would show the
commonality of concepts and principles in all
object-oriented methods, regardless of the notation
people used to display them. I think this was
reasonably successful and helped build consensus
towards a standard approach. Developing Business
Objects continued this work but including a wider
scope of where object technology had been applied in
business systems.
Q: Your most recent book, “Better Software Faster,”
is so practical and it’s clearly evident that both
you and Dan have considerable “real-world”
experience to share. There is so much useful advice
on how to engage and work on object-oriented,
enterprise-based development projects. You use
Together, TogetherSoft’s killer development app, and
your sample distributed Java application is
excellent—supporting both local and Internet-based
clients. There’s an interesting foreword too by
Edward Yourdon—he has been recommending this book to
his friends apparently.
Will you talk about this book in general, what led
you to write this book with Dan, and what you’re
your main objectives with the book?
A: There were two clear goals we had when setting
about this book. The first was to explain the
“single-source model” approach and how this can
improve quality and efficiency. This was not
necessarily dependent on people using Together
(providing they had an environment that would keep
design and implementation in step). In many ways
it’s much more important to me that people
understand the power of this concept and implement
it in other areas, rather than just go out and use
Together. The lack of a single-source approach is,
in my opinion, the most important reason why
model-driven approaches and CASE tools in particular
have not been used by even a majority of software
developers. The second goal was to give Together
users practical advice on how to get the best out of
the product and guidance in defining their
development process.
In fact we identified four principles that sum up
the message of the book: maintain just one
single-source model; use a minimum “meta-model” (in
other words have a simple definition of the model
behind your development process); use a
“perturbation” change model (in other words an
iterative approach based on small changes to a valid
system); and continuously measure quality. These
principles are applicable whatever development
environment you use – or indeed if you are only
planning to use a text editor and a compiler!
Dan and I had a good partnership on this book
because while I’m always looking for these lofty
goals and principles, Dan is the pragmatic details
man and an excellent teacher of complex aspects of
software. He produced an excellent Java
implementation of our case study project –
downloadable from the book’s web site (www.bettersoftwarefaster.com)
so readers can use it with the book. He also
provided a lot of detail, some in appendices to the
book, on how to configure Together which have really
been appreciated by Together users that I’ve spoken
to. Dan’s deep knowledge of J2EE has also meant that
we were able to cover this aspect of Java
development in one of the later chapters of the
book.
Ed Yourdon and I did some work together in 1992 but
I had not seen him since, so I was flattered when he
agreed to write the foreword to this book. Ed has
done a lot of work recently on legal disputes
between firms involved in large software projects
that have gone wrong. As he points out in the
foreword the very existence of these cases means
that software engineering has a long way to go in
improving its processes, so that the very common
sight of large software projects ending in tears
does not continue to happen. Ed picked out this
paragraph from the first chapter of the book to
highlight one key strategy for avoiding software
disasters.
"By discovering what the business imperatives are,
the team can deliver enhancements to the current
capabilities within the specified period while also
preparing for longer term requirements. If the team
follows this route, it will need a flexible and
lightweight development process with rapid feedback
loops to report holdups and roadblocks—and it will
need to plan contingencies accordingly."
Why read the rest of the book, asked Ed, if the
answer to the problem is a straightforward as that?!
Fortunately he found enough in the rest of the book
to consider it essential reading. The fact that Ed
is now recommending the book to business analysts,
project managers, designers and developers who are
not using Together, as well as to those that are, is
a great encouragement to us.
Q: It’s often the case that the analysts’ detailed
UML diagrams are overlooked or shelved by developers
in the attempt to make deadlines. What’s the impact
of such poor synchronization between the UML model
and code and can you share you views on
TogetherSoft’s software and how it addresses these
kinds of issues?
A: You’re right Stephen, this is a common problem.
When delivered software is out of step with the
analysis and design documents there are two serious
consequences. Firstly the information that has been
carefully and expensively gathered in such
documentation is seriously devalued, since the
implementation is not consistent with it. This may
result in incorrect functionality or delays caused
by rework when users or testers discover
inappropriate system behavior. Secondly it means
that the higher level documentation is of only
little value to the development team and subsequent
maintainers of the system since, even if the
documentation is consistent with the implementation,
developers can’t assume it is, and they have to go
and examine the source code any way – a much more
time-consuming and error-prone process. Together’s
LiveSource™ technology is the key to overcoming this
problem. By actually deriving the higher level
models from the source code and storing additional
design documentation directly with source code
files, there is no separate synchronization process
to be carried out.
There is still the need of course for different
levels of detail in the full set of diagrams and
documents maintained by the development team. But
rather than maintaining several models at different
levels of abstraction, we advocate a “single-source”
approach. This means that the different levels of
class diagram for example are all derived from the
same source files but with different filters and
selections made on the diagrams by the designers, to
ensure each diagram communicates with its intended
readers. Diagrams produced to explain the detailed
design for particular mechanisms in a system look
very different from diagrams that show an overview
of the components or the principle classes of
business objects discovered in analysis. Yet all
these diagrams can be derived from the same set
version-managed files.
This approach works best with relatively small teams
– that means more than a pair but less than a dozen
developers – and one in which the different
specialists, analysts, designers, database or user
interface experts and programmers are all working
together in the same team or teams, rather than
separating out say analysis teams and coding teams.
This makes sense in iterative life cycles where
projects often have very short timescales (less than
9 months) and requirements and business priorities
that may change through the life of the project.
Such project require an “agile” process so that the
team can respond to changing circumstances but still
deliver quality software with useful and effective
documentation.
Q: Can you comment on the iterative and incremental
approach—the perturbation change model?
A: Sure. A perturbation is a small change away from
a stable state – like moving a block by rolling it
end over end. Each step in the process is a stable
state and progress is made by temporarily
“perturbing” the equilibrium. In an iterative
development process it is important that the
starting point for change is a stable point. In
other words all the tests that have been defined for
requirements implemented up to that point should
pass, all the unit tests should pass and other
quality measures that have been defined (such as
measured complexity and coding standards) are also
acceptable. Then when a change is made to the
system, say to implement a new requirement, to
improve its performance or to improve the integrity
of its design, the quality measures can be reapplied
and the system be brought back to its consistent
working state. This contrasts greatly with a
development process that separates out requirements
definition, design, coding and testing into separate
phases of the project.
Most modern software development processes are
compatible with this perturbation model, but they
also tend to define a lot of other aspects like
project phases and roles which can cloud the issue.
That’s why we’ve tried to explain development in
terms of a simple continuous process that remains
the same throughout the whole project: start from a
state where the system (or model) is conformant with
the quality measures, introduce a small change and
get it back to the consistent state as soon as
possible.
Q: Does this affect the way the project is managed
as well as what the developers do?
A: It certainly does. We devoted one chapter of the
book to project planning and monitoring because we
think that project managers should be using a
different planning model from the normal critical
path analysis method, which is used by tools like
Microsoft Project and is typically shown as “Gantt”
or “Pert” charts. This method is essential to
planning something like a building or a bridge where
the dependencies between the tasks are extremely
significant. You can’t put the roof on the house for
example until you’ve built the walls to hold it up!
On the other hand the order in which features of a
software system are delivered to users are not
constrained to the same degree, meaning that the
users or business representatives can determine the
priorities for the implementation of different
features based on an assessment of risk and benefit.
We can also allow these priorities to be changed as
the project progresses. This makes the development
plan flexible, while still enabling close monitoring
of progress and continuously adjusted estimates of
completion dates.
Q: Sounds as though you’ve had experience of
projects that don’t always deliver to plan. Have
some of the projects that you have worked on had
problems like these and what lessons were learned by
the project that you can pass on?
A: You’re right Stephen! I don’t think anyone who
has worked in this industry for any length of time
would not have seen some project or other suffering
from budget or schedule overruns. I remember one
large project where everyone was working really hard
to meet its first customer delivery deadline – not
for the system going live but still a very important
milestone. Right up to a week before the deadline
the project manager thought he was going to get a
system out on time. However at that point they
realized that it was not going to happen and they
declared a one month slip. How could the management
have been so out of touch with the reality on the
ground? Well in fact the project did not complete
that milestone until a full 5 months later! The main
problem was that this was the first occasion when
the project had tried to integrate all the software
being delivered by the different teams. Everyone
thought they could finish their piece in time but it
wasn’t until they started to put them together that
they could see where the holes were. Continuous
integration of software, and continuous unit and
functional testing of software, have been important
aspects of software development processes that I
stress to my clients, and particularly since the
experience on that project. Whether the project
learned the lesson I’m not sure, but certainly many
other projects have learned from their problems.
Q: I understand that you’re now moving on from
TogetherSoft? What does the future hold for you now
do you think?
A: That’s right. I’ve really enjoyed my time working
at TogetherSoft with a tremendously talented bunch
of people, both in the UK and throughout their
offices worldwide, but it is now time to move on. I
intend to remain available as a consultant to
clients using Together as their development platform
but I also want to consult more widely with
enterprises defining and improving their development
processes. I would not be surprised to find that I
become involved again in the management of a
UK-based professional services organization but
actually at this stage I haven’t made any final
decisions about new businesses. It’s one of the
bonuses of life that one never knows what’s just
around the corner!
Q: Thank you for sharing your valuable insights with
us today and we look forward to hearing you speak at
conferences, reading your books, and articles.
A: Thank you Stephen. It’s been a pleasure. |
|
|
