Few management tools appear as simple and obviously useful as measurement. You establish numeric goals, take actions, and measure how the actions affect progress toward goals. Based on what the measures reveal, you adjust your actions. You continue in this way. Simple. You analyze measurements to determine what works and what doesn't. Despite organizational complexity, you learn confidently, your managerial decisions backed up by hard data. Obviously useful. Right?

Look more closely, however, and this clear picture begins to blur. Soon you find examples of measurement disaster. Look again and you discover startling disagreement among recognized experts about the value of organizational measurement. First, you encounter an expert who claims that measurement is indispensable, a nearly absolute good, almost a miracle cure. Next, you find an expert who concludes that measurement is inescapably harmful, a danger to the survival of the Western world, a seductive trap. Keep looking and you also see that, far from sorting out their differences, disagreeing experts seem to be largely ignoring each other. Your warm feelings of confidence in organizational measurement give way to cold misgivings.

This book is my attempt to sort it all out, to determine which organizational factors allow measurement to work successfully, and which force measurement programs to fail or do damage. Be forewarned, however: This is not a cookbook. You will not find quick-and-easy answers here. Measurement in organizations is not a quick-and-easy subject, and, as I will show, the idea that measurement is quick-and-easy causes quite a bit of trouble for people who try to implement measurement systems. What you will find in this book, I hope, is a way to deeper understanding of organizational measurement and a framework that will be useful as you consider measurement opportunities in your own organization. I hope that you find these things here, because the foremost goal of this book is to help managers and designers of real systems of measurement.

This book is different from some other books you might have read on measurement or related subjects. Perhaps the most important difference is in its emphasis on the behavioral aspects of measurement situations. There are no technical descriptions of specific measurements here and no details of measurement analysis or graphing techniques. Instead, the focus is on people and how they react when they are part of organizational systems that are being measured.

Another difference between this book and some others is its attention to measurement systems that don't work very well. Some books on measurement so strongly advocate its use that they look almost exclusively at success stories. They profess to tell you how to get it right, but they supply little or no detail about the consequences or likelihood of getting it wrong. Partly, this is because stories of management failures are harder to find than accounts of successes, for obvious reasons: People like to claim credit for successes and forget failures. But you can learn a lot from failure. So, I've worked to find examples of failure and devoted a significant portion of this book to examining the examples in search of a common pattern of failure. Understanding the pattern of failure can help us avoid it.

Yet another way that this book is different is in some of the tools it uses to explore issues surrounding measurement. Some tools used in this book are borrowed from economics, and if you are not an economist you may not have seen them before. I use them because they add value to the discussion by drawing attention to aspects of measurement situations (especially costs) that get left out of many analyses. As these tools are introduced, I explain them, provide advice on how to interpret them, and try to point out their implicit limitations.

Three Central Questions

Three questions are at the heart of this book:

To answer these questions, I've taken three approaches.

First, I've conducted an extensive review of written materials related to measurement and performance management in fields ranging from economics to management theory to statistics to theater. By drawing on so many fields, we can make use of the special insights available in each. The many references to other source materials in this book, and its extensive bibliography, should assist any reader in search of more information on the subject.[1]

Second, I've constructed a model similar to models used by some economists. The model is a way of capturing the pattern of dysfunction, a mechanism that allows us to vary aspects of a measurement situation and see what happens. As such, it is a way to determine which situations make success possible and which situations make failure inevitable. Answers to central questions are derived primarily from the model.

Third, I've conducted interviews with eight people who are recognized experts in the use of measurement to manage a particular organizational activity: computer software development. The content of expert interviews adds richness to this book's recommendations and explanations. The interviews also provide a check on assertions about the current state of thinking about measurement; the check is especially important at points where allegedly common ways of thinking are shown to contain flaws or contradictions. In addition, the interviews permit discovery of expert beliefs that are consistent with the model's explanations of measurement failure -- alarmingly, the seeds of failure seem present in the advice of some experts.

My use of the word "expert" in this book is deliberate, but the word should be interpreted in a specific sense. I call these eight people experts because they are recognized repositories of state-of-the-art knowledge about software measurement -- not because they agree with me (some of them don't). The fact that some experts' answers contain elements consistent with what I call a dysfunctional pattern does not mean that they are not experts, or that they cannot add value to a discussion of organizational measurement. So, throughout this book, I use quotes from all of the interviews to help me make points. I see no necessary contradiction or double standard in using the opinions of experts with whom I disagree on some points to make points on which we agree.

Who are these experts? Those who agreed to be identified are

One expert wished to remain anonymous and is labeled Expert X when cited or discussed in this book. Names of experts are used throughout except in Chapter Sixteen, where expert answers to specific questions are analyzed and directly compared. Names are omitted in that chapter to prevent the appearance of ranking of experts, which is not one of the aims of this book.

Students of software measurement will recognize most, if not all, of these names. People who are less familiar with these experts may wish to know that this group, on average, has produced extensive book, periodical, and conference proceedings publications specifically on the subject of software measurement. Their experience in the software field ranges from fifteen to thirty years, and all have spent much of their careers working on measurement or related issues. These experts are a subset of all such experts in the world, but they are a relatively large subset in this new field. I estimate that there are no more than twenty other experts in the world who would be regarded by most practitioners as being on a par with these.

A Strategy for Reading This Book

Readers of this book will no doubt share an interest in organizational measurement as a management tool. But their deeper motivations may vary. Some will be interested because they need information they can use to improve the performance in their own organization; such readers may be rushing to satisfy specific needs for solutions that work right now. Other readers will be attracted by a general desire for understanding of organizational phenomena; these readers may be less hurried and may value more leisurely exposition and more detailed explanations. Where a reader falls on the spectrum between specific and general interest will influence what he or she gets out of each of the chapters of this book. I encourage you to think about where you fit on this spectrum, and, depending on what you decide, to vary your approach to reading this book. I have advice on how to vary your approach, but first I need to explain how the book is arranged.

This book is organized into nineteen chapters. Chapters One through Three introduce measurement issues, especially issues that surround measurement failure. Chapters Four through Nine directly address the first of the big questions highlighted above by constructing the model that is central to this book's treatment of measurement in organizations. Chapters Ten through Thirteen address alternative forms of management and some practical conclusions of the model about management styles and the organization of work. Chapters Fourteen through Sixteen turn to the second big question by extending the model to consider why measurement dysfunction arises and persists. Chapters Seventeen and Eighteen explore broader implications of the previous chapters' conclusions. And, finally, in Chapter Nineteen, an epilogue provides some summary thoughts on the book as a whole.

My advice on how to read this book is quite simple. For deepest understanding, read all of the chapters in detail. For a quicker tour with a more practical focus, skim Chapters Four through Eight (which present the details of the model used in this book), and carefully read Chapter Nine (a recap of Chapters Four through Eight). If you are especially pressed for time, read Chapters One through Three ("An Introduction to Measurement Issues," "A Closer Look at Measurement Dysfunction," and "The Intended Uses of Measurement in Organizations"), Chapter Seventeen ("The Measurement Disease"), and the concluding chapter. I suggest this last, very brief route through the book, in part, because I think it will suffice to hook you -- to bring you back to the book at a later date, when you have more time available for reading.

What This Book Is Not About

This book does not deal with all important measurement-related issues. As I noted earlier, there is little here on the technical aspects of measuring, analyzing, or graphing specific measurements. Perhaps the most significant topic not addressed here, however, has to do with a moral dimension of the measurement problem. Aside from considerations of efficiency, feasibility, and cost, there are situations in which measurement is not appropriate for ethical reasons. Philosophical questions concerning privacy, fairness, and the like could be addressed in a book on measurement in organizations. But treating these issues adequately would require many more pages. My decision to exclude the bulk of moral issues from this book reflects my desire to treat capably a manageable slice of the measurement subject; it is not an indicator of how important I believe ethical issues to be.

To people who are interested in the moral aspects of measurement in organizations, I am pleased to report that by the end of this book I have established a case for the importance of ethical behavior purely on efficiency grounds. I show that ethical behavior should be practiced in many organizational contexts not because such behavior makes the world a better place (although that is also a fine reason) but because ethical behavior makes things work better. If there is a single message that comes from this book, it is that trust, honesty, and good intention are more efficient in many social contexts than verification, guile, and self-interest. To some, this conclusion may not seem profound. In my view, questions of the appropriate mixture of selfish individualism and selfless cooperation in a civilized society and its institutions are both profound and of the greatest practical importance.