How to Measure Anything Workbook: Finding the Value of Intangibles in Business

By Douglas W. Hubbard

The invaluable companion to the new edition of the bestselling How to Measure Anything

This companion workbook to the new edition of the insightful and eloquent How to Measure Anything walks readers through sample problems and exercises in which they can master and apply the methods discussed in the book.

The book explains practical methods for measuring a variety of intangibles, including approaches to measuring customer satisfaction, organizational flexibility, technology risk, technology ROI, and other problems in business, government, and not-for-profits.

• Companion to the revision of the bestselling How to Measure Anything
• Provides chapter-by-chapter exercises
• Written by industry leader Douglas Hubbard

Written by recognized expert Douglas Hubbard—creator of Applied Information Economics—How to Measure Anything Workbook illustrates how the author has used his approach across various industries and how any problem, no matter how difficult, ill defined, or uncertain can lend itself to measurement using proven methods.

• Language: English
• Publisher: Wiley
• Release date: Feb 26, 2014
• ISBN: 9781118860397

Book preview

PART I

Questions

CHAPTER 1

The Challenge of Intangibles

LEARNING OBJECTIVES

Describe different definitions of intangibles.

Explain why measurements matter.

Explain the different reasons for measurements.

Explain the purpose of a decision-oriented framework for measurement.

Describe the key steps in the Applied Information Economics approach to measurement.

Explain why we use power tools for measurement.

Provide an outline to the remainder of the book.

CHAPTER OVERVIEW

Chapter NaN proposes that anything can be measured and explains why measurements are critical to business, government, and life, and it outlines the case for this in the book. In business, there is an unlimited array of so-called intangibles like management effectiveness, research productivity, and public image, each of which turns out to be measurable by methods that are simpler than the reader may think.

The book proposes a decision-oriented framework for measurement. It is preferable to use quantitative models over unaided subjective intuition. For each decision, there are many micro-decisions about what to measure.

A method called Applied Information Economics (AIE) is introduced as a universal approach to measurement problems. The complexities of measurement can be simplified with a power tools approach to statistics.

QUESTIONS

Two common understandings of the word intangible apply to things that can be measured but are not _______, and things that cannot be measured at all. One purpose of this text is to argue that the second type of intangible _______.

Physically touchable; does not exist

Physically touchable; is the best-working definition of the word

Analyzed; does not exist

Analyzed; is unavoidable to some degree

None of the above

To which type of decisions does this book apply?

Government policy

Personal decisions

Business planning

All of the above

Which of the following is true of intangibles?

They defy measurement entirely.

The term is essentially a misnomer because anything is measurable.

They have little direct impact on decisions.

They are rarely measurable and relatively unpredictable.

Which of the following statements is true of intangible variables?

One can measure the tangible effects of seemingly intangible variables; if a variable literally had no detectable effect, then it would not be relevant to any decision.

The low-cost measurement of intangible variables usually requires the application of state-of-the-art statistical techniques.

Immeasurable variables can only be assessed intuitively.

The routine treatment of intangible variables as unimportant in prevalent decision models suggests the effect of allegedly immeasurable factors is, in most cases, negligible.

None of the above

If you are trying to figure out how a specific measurement process should work, how should you use this book?

Check for your specific problem in the index. If you find it, skip to that chapter. If you don’t find it, that issue is not addressed.

The book is purely theoretical and doesn’t address specific measurement problems.

The steps described in the book apply to any measurement problem and are presented sequentially by chapter. Read the whole book and apply the steps within.

None of the above

Which purpose of measurement is emphasized the most in this book?

Measurements for resale

Measurements for curiosity or entertainment

Measurements that support decisions

None of the above

Why do managers need to have a method to analyze options for reducing uncertainty about decisions?

There are so many possible things to measure.

Only some measurements impact the outcome.

Measurements can be costly and time-consuming.

All of the above

Which of the following statements is true?

The value of a measurement is a direct function of the cost to obtain it.

Measurements can be done only on things with no uncertainty.

If a thing is difficult to measure, it’s probably not as important to a decision.

The value of a measurement is partly a function of the uncertainty associated with that variable.

Why are decision makers in organizations often less informed than they could be?

They presume that some things are totally immeasurable.

They use traditional statistical methods to describe an intangible quantity.

They don’t rely enough on experience and expert opinion.

They don’t isolate and eliminate all uncertainty.

What is/are example(s) of a micro-decision to which the author refers?

Small, inconsequential investments

Small corrections to projects

The choice about what to measure about a decision and how much to measure it

All of the above

Which of the following statements about quantitative models is false?

They don’t have uncertainty.

They tend to be more reliable than intuition.

Studies indicate that they perform better than human judgment alone.

They help to optimize the reduction of uncertainty.

Which is a valid reason for wanting to measure something?

Upper management has suggested that it’s a good idea.

It’s easy to do.

It ultimately informs a decision of some kind.

Any uncertainty in a decision process can lead to catastrophic consequences.

While the author stresses that anything can be measured, why does he not also say that everything should be measured?

The second statement is practical only when applied to micro-decisions.

The second statement is contrary to basic economic measurement principles.

The second statement applies only to cheap measurements.

The second statement applies only to an organization’s core values.

True or False: Those who work in business tend to employ careful measurement methods more often than those working in the physical sciences.

True

False

Which of the following are examples of power tools as described in the book?

Proofs based on probability axioms

Spreadsheets

Tables

A list of mathematical laws of statistics to memorize

All of the above

B and C

What is the purpose of power tools as described in the book?

To teach us how to mathematically derive fundamental principles of statistics

To take apart each problem and make sure we understand every detail of the process

To make statistics practical to a wider audience

To help us quickly look up formulas

How do most scientists handle statistical measurement problems required for

Reviews for How to Measure Anything Workbook

[jpsnow · Rating: 3 out of 5 stars]

What I enjoyed most about this book was how it revealed multiple ways to attack a measurement problem. Even for the most complex or unclear problems, there are ways to bound and frame the approach sufficient for informing decisions. The book is also heavy, zooming from simple examples to the full span of a college-level statistics course. For the reader interested in making measurement and estimation practical, this book is worthy read and could serve as a ready reference.

[shdawson · Rating: 4 out of 5 stars]

We do not know unless we first measure.

[f.lee-1 · Rating: 1 out of 5 stars]

All the undeserved optimism of a TED talk condensed into a book. The author wants people to think more quantitatively as they go about trying to find tools to measure and presumable optimize anything in life. Now this is wonderful if you're measuring the size of a railroad, but when you're measuring something latent like utility, happiness levels, or personality traits, some theories need to be developed about the connection between your tool of measurement and the thing quantity you wish to measure. How do you know that the intangible you're measuring actually exists? How do you know that your measurement tool isn't actually giving you completely spurious results that have nothing to do with the intangible you want to measure? The author never addresses this, but provides a variety of techniques you can use to measure your potentially nonexistent intangibles.

[brikis98 · Rating: 4 out of 5 stars]

As an engineer, this book makes me happy. A great discussion of how to break *any* problem down into quantifiable metrics, how to figure out which of those metrics is valuable, and how to measure them. The book is fairly actionable, there is a complementary website with lots of handy excel tools, and there are plenty of examples to help you along. The only downside is that this is largely a stats book in disguise, so some parts are fairly dry and a the difficulty level jumps around a little bit. If you make important decisions, especially in business, this book is for you.

Some great quotes:

Anything can be measured. If a thing can be observed in any way at all, it lends itself to some type of measurement method. No matter how “fuzzy” the measurement is, it’s still a measurement if it tells you more
than you knew before. And those very things most likely to be seen as immeasurable are, virtually always, solved by relatively simple measurement methods.

Measurement: a quantitatively expressed reduction of uncertainty based on one or more observations.

So a measurement doesn’t have to eliminate uncertainty after all. A mere _reduction_ in uncertainty counts as a measurement and possibly can be worth much more than the cost of the measurement.

A problem well stated is a problem half solved.
—Charles Kettering (1876–1958)

The clarification chain is just a short series of connections that should bring us from thinking of something as an intangible to thinking of it as a tangible. First, we recognize that if X is something that we care about, then X, by definition, must be detectable in some way. How could we care about things like “quality,” “risk,” “security,” or “public image” if these things were totally undetectable, in any way, directly or indirectly? If we have reason to care about some unknown quantity, it is because we think it corresponds to desirable or undesirable results in some way. Second, if this thing is detectable, then it must be detectable in some amount. If you can observe a thing at all, you can observe more of it or less of it. Once we accept that much, the final step is perhaps the easiest. If we can observe it in some amount, then it must be measurable.

Rule of five: There is a 93.75% chance that the median of a population is between the smallest and largest values in any random sample of five from that population.

An important lesson comes from the origin of the word experiment. “Ex- periment” comes from the Latin ex-, meaning “of/from,” and periri, mean- ing “try/attempt.” It means, in other words, to get something by trying. The statistician David Moore, the 1998 president of the American Statistical Association, goes so far as to say: “If you don’t know what to measure, measure anyway. You’ll learn what to measure.”

Four useful measurement assumptions:
1. Your problem is not as unique as you think.
2. You have more data than you think.
3. You need less stated that you think.
4. And adequate amount of new data is more accessible than you think.

Don’t assume that the only way to reduce your uncertainty is to use an impractically sophisticated method. Are you trying to get published in a peer-reviewed journal, or are you just trying to reduce your uncertainty about a real-life business decision? Think of measurement as iterative. Start measuring it. You can always adjust the method based on initial findings.

In business cases, most of the variables have an "information value" at or near zero. But usually at least some variables have an information value that is so high that some deliberate measurement is easily justified.

While there are certainly variables that do not justify measurement, a persistent misconception is that unless a measurement meets an arbitrary standard (e.g., adequate for publication in an academic journal or meets generally accepted accounting standards), it has no value. This is a slight oversimplification, but what really makes a measurement of high value is a lot of uncertainty combined with a high cost of being wrong. Whether it meets some other standard is irrelevant.

When people say “You can prove anything with statistics,” they probably don’t really mean “statistics,” they just mean broadly the use of numbers (especially, for some reason, percentages). And they really don’t mean “anything” or “prove.” What they really mean is that “numbers can be used to confuse people, especially the gullible ones lacking basic skills with numbers.” With this, I completely agree but it is an entirely different claim.

The fact is that the preference for ignorance over even marginal reductions in ignorance is never the moral high ground. If decisions are made under a self-imposed state of higher uncertainty, policy makers (or even businesses like, say, airplane manufacturers) are betting on our lives with a higher chance of erroneous allocation of limited resources. In measurement, as in many other human endeavors, ignorance is not only wasteful but can be dangerous.

If we can’t identify a decision that could be affected by a proposed measurement and how it could change those decisions, then the measurement simply has no value.

The lack of having an exact number is not the same as knowing nothing.

The McNamara Fallacy: The first step is to measure whatever can be easily measured. This is OK as far as it goes. The second step is to disregard that which can’t easily be measured or to give it an arbitrary quantitative value. This is artificial and misleading. The third step is to presume that what can’t be measured easily isn’t important. This is blindness. The fourth step is to say that what can’t easily be measured really doesn’t exist. This is suicide.

First, we know that the early part of any measurement usually is the high-value part. Don’t attempt a massive study to measure something if you have a lot of uncertainty about it now. Measure a little bit, remove some uncertainty, and evaluate what you have learned. Were you surprised? Is further measurement still necessary? Did what you learned in the beginning of the measurement give you some ideas about how to change the method? Iterative measurement gives you the most flexibility and the best bang for the buck.

This point might be disconcerting to some who would like more certainty in their world, but everything we know from “experience” is just a sample. We didn’t actually experience everything; we experienced some things and we extrapolated from there. That is all we get—fleeting glimpses of a mostly unobserved world from which we draw conclusions about all the stuff we didn’t see. Yet people seem to feel confident in the conclusions they draw from limited samples. The reason they feel this way is because experience tells them sampling often works. (Of course, that experience, too, is based on a sample.)

Anything you need to quantify can be measured in some way that is superior to not measuring it at all.
—Gilb’s Law

[danjust · Rating: 5 out of 5 stars]

This is the only book about measurement for business which has any substance. But anyone who is just a clerk or not really dealing with deicions under uncertainty probably got this thinking it would be about some accounting procedure. This is really for the manager and leader who is a scientist at heart and wants to run things on the basis of reason and evidence. Moderately high IQ required.