Few management tools appear as simple and obviously useful
as measurement. You establish numeric goals, take actions, and measure how the
actions affect progress toward goals. Based on what the measures reveal, you adjust
your actions. You continue in this way. Simple. You analyze measurements to determine
what works and what doesn't. Despite organizational complexity, you learn confidently,
your managerial decisions backed up by hard data. Obviously useful. Right?
Look more closely, however, and this clear picture begins
to blur. Soon you find examples of measurement disaster. Look again and you discover
startling disagreement among recognized experts about the value of organizational
measurement. First, you encounter an expert who claims that measurement is indispensable,
a nearly absolute good, almost a miracle cure. Next, you find an expert who concludes
that measurement is inescapably harmful, a danger to the survival of the Western
world, a seductive trap. Keep looking and you also see that, far from sorting
out their differences, disagreeing experts seem to be largely ignoring each other.
Your warm feelings of confidence in organizational measurement give way to cold
This book is my
attempt to sort it all out, to determine which organizational factors allow measurement
to work successfully, and which force measurement programs to fail or do damage.
Be forewarned, however: This is not a cookbook. You will not find quick-and-easy
answers here. Measurement in organizations is not a quick-and-easy subject, and,
as I will show, the idea that measurement is quick-and-easy causes quite a bit
of trouble for people who try to implement measurement systems. What you will
find in this book, I hope, is a way to deeper understanding of organizational
measurement and a framework that will be useful as you consider measurement opportunities
in your own organization. I hope that you find these things here, because the
foremost goal of this book is to help managers and designers of real systems of
This book is different
from some other books you might have read on measurement or related subjects.
Perhaps the most important difference is in its emphasis on the behavioral aspects
of measurement situations. There are no technical descriptions of specific measurements
here and no details of measurement analysis or graphing techniques. Instead, the
focus is on people and how they react when they are part of organizational systems
that are being measured.
Another difference between this book and some others is
its attention to measurement systems that don't work very well. Some books on
measurement so strongly advocate its use that they look almost exclusively at
success stories. They profess to tell you how to get it right, but they supply
little or no detail about the consequences or likelihood of getting it wrong.
Partly, this is because stories of management failures are harder to find than
accounts of successes, for obvious reasons: People like to claim credit for successes
and forget failures. But you can learn a lot from failure. So, I've worked to
find examples of failure and devoted a significant portion of this book to examining
the examples in search of a common pattern of failure. Understanding the pattern
of failure can help us avoid it.
Yet another way that this book is different is in some of the tools it uses
to explore issues surrounding measurement. Some tools used in this book are borrowed
from economics, and if you are not an economist you may not have seen them before.
I use them because they add value to the discussion by drawing attention to aspects
of measurement situations (especially costs) that get left out of many analyses.
As these tools are introduced, I explain them, provide advice on how to interpret
them, and try to point out their implicit limitations.
are at the heart of this book:
How should measurement be used to improve
the efficiency and effectiveness of organizations?
Why do real organizations often use measurement inappropriately, thereby causing
measurement programs to fail or do harm?
What are the practical
implications of the answers to the first two questions?
To answer these questions, I've taken three approaches.
First, I've conducted an extensive review of written materials
related to measurement and performance management in fields ranging from economics
to management theory to statistics to theater. By drawing on so many fields, we
can make use of the special insights available in each. The many references to
other source materials in this book, and its extensive bibliography, should assist
any reader in search of more information on the subject.
Second, I've constructed
a model similar to models used by some economists. The model is a way of capturing
the pattern of dysfunction, a mechanism that allows us to vary aspects of a measurement
situation and see what happens. As such, it is a way to determine which situations
make success possible and which situations make failure inevitable. Answers to
central questions are derived primarily from the model.
Third, I've conducted interviews with eight people who are
recognized experts in the use of measurement to manage a particular organizational
activity: computer software development. The content of expert interviews adds
richness to this book's recommendations and explanations. The interviews also
provide a check on assertions about the current state of thinking about measurement;
the check is especially important at points where allegedly common ways of thinking
are shown to contain flaws or contradictions. In addition, the interviews permit
discovery of expert beliefs that are consistent with the model's explanations
of measurement failure -- alarmingly, the seeds of failure seem present in the
advice of some experts.
My use of the word "expert" in this book is deliberate,
but the word should be interpreted in a specific sense. I call these eight people
experts because they are recognized repositories of state-of-the-art knowledge
about software measurement -- not because they agree with me (some of them don't).
The fact that some experts' answers contain elements consistent with what I call
a dysfunctional pattern does not mean that they are not experts, or that they
cannot add value to a discussion of organizational measurement. So, throughout
this book, I use quotes from all of the interviews to help me make points. I see
no necessary contradiction or double standard in using the opinions of experts
with whom I disagree on some points to make points on which we agree.
Who are these experts? Those who agreed to be identified
David N. Card, Software Productivity Solutions
Tom DeMarco, Atlantic Systems Guild
Capers Jones, Software Productivity Research
John D. Musa, AT&T Bell Laboratories
Daniel J. Paulish, Siemens Corporate Research
Lawrence H. Putnam,
Quantitative Software Management
E.O. Tilford, Sr., Fissure
One expert wished to remain anonymous and is labeled Expert X when cited or
discussed in this book. Names of experts are used throughout except in Chapter
Sixteen, where expert answers to specific questions are analyzed and directly
compared. Names are omitted in that chapter to prevent the appearance of ranking
of experts, which is not one of the aims of this book.
Students of software measurement will recognize most, if
not all, of these names. People who are less familiar with these experts may wish
to know that this group, on average, has produced extensive book, periodical,
and conference proceedings publications specifically on the subject of software
measurement. Their experience in the software field ranges from fifteen to thirty
years, and all have spent much of their careers working on measurement or related
issues. These experts are a subset of all such experts in the world, but they
are a relatively large subset in this new field. I estimate that there are no
more than twenty other experts in the world who would be regarded by most practitioners
as being on a par with these.
A Strategy for Reading This Book
Readers of this book will no doubt share an interest in
organizational measurement as a management tool. But their deeper motivations
may vary. Some will be interested because they need information they can use to
improve the performance in their own organization; such readers may be rushing
to satisfy specific needs for solutions that work right now. Other readers will
be attracted by a general desire for understanding of organizational phenomena;
these readers may be less hurried and may value more leisurely exposition and
more detailed explanations. Where a reader falls on the spectrum between specific
and general interest will influence what he or she gets out of each of the chapters
of this book. I encourage you to think about where you fit on this spectrum, and,
depending on what you decide, to vary your approach to reading this book. I have
advice on how to vary your approach, but first I need to explain how the book
This book is organized
into nineteen chapters. Chapters One through Three introduce measurement issues,
especially issues that surround measurement failure. Chapters Four through Nine
directly address the first of the big questions highlighted above by constructing
the model that is central to this book's treatment of measurement in organizations.
Chapters Ten through Thirteen address alternative forms of management and some
practical conclusions of the model about management styles and the organization
of work. Chapters Fourteen through Sixteen turn to the second big question by
extending the model to consider why measurement dysfunction arises and persists.
Chapters Seventeen and Eighteen explore broader implications of the previous chapters'
conclusions. And, finally, in Chapter Nineteen, an epilogue provides some summary
thoughts on the book as a whole.
My advice on how to read this book is quite simple. For deepest understanding,
read all of the chapters in detail. For a quicker tour with a more practical focus,
skim Chapters Four through Eight (which present the details of the model used
in this book), and carefully read Chapter Nine (a recap of Chapters Four through
Eight). If you are especially pressed for time, read Chapters One through Three
("An Introduction to Measurement Issues," "A Closer Look at Measurement
Dysfunction," and "The Intended Uses of Measurement in Organizations"),
Chapter Seventeen ("The Measurement Disease"), and the concluding chapter.
I suggest this last, very brief route through the book, in part, because I think
it will suffice to hook you -- to bring you back to the book at a later date,
when you have more time available for reading.
What This Book
Is Not About
This book does
not deal with all important measurement-related issues. As I noted earlier, there
is little here on the technical aspects of measuring, analyzing, or graphing specific
measurements. Perhaps the most significant topic not addressed here, however,
has to do with a moral dimension of the measurement problem. Aside from considerations
of efficiency, feasibility, and cost, there are situations in which measurement
is not appropriate for ethical reasons. Philosophical questions concerning privacy,
fairness, and the like could be addressed in a book on measurement in organizations.
But treating these issues adequately would require many more pages. My decision
to exclude the bulk of moral issues from this book reflects my desire to treat
capably a manageable slice of the measurement subject; it is not an indicator
of how important I believe ethical issues to be.
To people who are interested in the moral aspects of measurement in organizations,
I am pleased to report that by the end of this book I have established a case
for the importance of ethical behavior purely on efficiency grounds. I show that
ethical behavior should be practiced in many organizational contexts not because
such behavior makes the world a better place (although that is also a fine reason)
but because ethical behavior makes things work better. If there is a single message
that comes from this book, it is that trust, honesty, and good intention are more
efficient in many social contexts than verification, guile, and self-interest.
To some, this conclusion may not seem profound. In my view, questions of the appropriate
mixture of selfish individualism and selfless cooperation in a civilized society
and its institutions are both profound and of the greatest practical importance.
1. Where I've embedded references in the text, I list the foremost
authorities first and then revert to chronological order.
This excerpt from Measuring
and Managing Performance in Organizations [ISBN:0-932633-36-6] appears by permission of
Dorset House Publishing. Copyright © 1996 by Robert
D. Austin. All rights reserved. See
The material contained in this file may be shared for
noncommercial purposes only, nonexclusively, provided
that this Copyright Notice always appears with it. This
material may not be combined with advertisements, online
or in print, without explicit permission from Dorset
House Publishing. For copies of the printed book or
for permissions, contact Dorset House Publishing, 1-800-342-6657,
New: 3143 Broadway, Suite 2B, New York, NY 10027 USA. Additional
rights limitations apply, as presented in the Legal
Disclaimer posted at http://www.dorsethouse.com/legal.html.