The Science of Organizational Change: How Leaders Set Strategy, Change Behavior, and Create an Agile Culture

By Paul Gibbons

"The best book on change I've ever read..." (Bank CEO)

"The best book on change in fifteen years, perhaps longer..." (Organization Development consultant)

Leaders need guidance on leading change grounded in the latest science, not 20th-century myths. In this updated 2019 edition of The Science of Organizationa

• Language: English
• Publisher: Phronesis Media
• Release date: May 20, 2019
• ISBN: 9780997651249

Paul Gibbons

Paul Gibbons has a 40-year career straddling international business and academia. His research and writing explore how philosophy and science can be used to enlighten contemporary business thinking, debunk myths and pseudoscience, and solve practical business problems, including changing culture, developing leaders, and using analytics and evidence to make strategic decisions. Paul's academic background, starting in math, then in economics, neuroscience, psychology, and philosophy, allows him to bring perspectives to business not typically found in traditional business books. His consulting career, mostly in Europe, included founding an award-winning "teal" Organization Development consulting firm, Future Considerations. Paul has coached dozens of CEOs - on strategy, change, and talent issues. His change experience includes clients such as Comcast, Shell, PwC, BP, Barclays, KPMG, British Airways, HSBC, Nokia, The Body Shop, Comcast, the NHS, and UK Ministers. He was the change management lead on a $1 billion program for the UK's Department of Work and Pensions. Paul has appeared in Microsoft's Distinguished Author Program and at Google and appeared in the Wall Street Journal and the Financial Times. He is a Fellow of the Royal Society of Arts. He now writes, hosts the award-winning Think Bigger Think Better philosophy podcast, plays competitive poker, chess, and bridge, and raises two boys in Colorado.

Book preview

Paul_SoC_ebook_cover.jpg

Volume I

of the

Leading Change in the Digital Age

series

The Science of Organizational Change

How Leaders Set Strategy, Change Behavior, and Create an Agile Culture

(2019 Edition)

Paul Gibbons

Publisher: Phronesis Media

© 2019 by Paul Gibbons

Originally published as The Science of Successful Organizational Change (FT Press, 2015)

For information about buying this title in bulk quantities, or for training materials, webinars, or consulting services, please contact the author – [e-mail]

Company and product names mentioned herein are the trademarks or registered trademarks of their respective owners.

All rights reserved. No part of this book may be reproduced, in any form or by any means, without permission in writing from the author.

Cover and interior design by Dania Zafar

Illustrations by Andrés Goldstein

Printed in the United States

First Printing April 2019

ISBN-13: 978-0-9976512-3-2 (paperback)

ISBN-13: 978-0-9976512-4-9 (ebook)

Library of Congress Control Number: 2019938289

About the

Leading Change in the Digital Age

Series

This book is the introductory title in a series called Leading Change in the Digital Age. Although each book can be read without having read the others, in later books I find myself constantly referring readers to more detailed discussions in this book. If I added all the relevant content from this book into the second volume in the series, Impact, I’d have a 600-page door stop; all four would make a 1200-page book on leading change. All concepts in change are interlinked (as they ought to be!) For example mindfulness is related to engagement, decision making, team culture, debiasing, leadership, and habit change. Those mindfulness concepts are spread across all four volumes, with an introduction to the science of mindfulness in this book.

My aim in the series is, immodestly, to reinvent change for the 21st-century. I find much of what I read in the most popular change books either trivial or wrong and that has to stop if business is to play its role in forging a better world for humanity, our children and grandchildren.

The series titles are:

Volume I – The Science of Organizational Change: How Leaders Set Strategy, Change Behavior, and Create an Agile Culture (Spring 2019)

Volume II – Impact: Using Science to Change Behaviors, Hearts, and Minds (Summer 2019)

Volume III – Agile Leadership: New Frontiers in Leadership Development and Business Agility (Summer 2020)

Volume IV – Leading with Science: The Evidence, AI, and Data Revolutions (Summer 2021)

Contents

About the Author

Preface to the 2019 edition

Introduction

Chapter 1: Failed Change: The Greatest Preventable Cost to Business?

Chapter 2: From Change Fragility to Change-Agility

Chapter 3: Governance and the Psychology of Risk

Chapter 4: Decision Making in VUCA Environments

Chapter 5: Cognitive Biases and Failed Strategies

Chapter 6: Misunderstanding Human Behavior

Chapter 7: The Science of Changing Behaviors

Chapter 8: The Science of Changing Hearts and Minds

Chapter 9: Leading with Science

About the Leading Change in the Digital Age Series

Bibliography

Acknowledgments

Praise for

The Science of Organizational Change

If you are tasked with change inside a large organization, this is a must-read. Gibbons has written the single most insightful book on organizational change. You'll laugh at the myth-busting and consultant-bashing while taking copious notes on everything else.

—Miki Reilly-Howe, Managing Director, Department26

Organizational change is a huge industry filled with buzzwords and fads and suffering from an unacceptably high failure rate. Paul Gibbons applies scientifically founded, rigorous thought and practical wisdom to this charlatan-filled domain, and produces actionable, sensible, evidence-based insights that can make change efforts much more likely to succeed and organizations much more agile and effective.

—Jeffrey Pfeffer, Thomas D. Dee II Professor of

Organizational Behavior, Stanford Business School

The best book on change I have read. Paul Gibbons draws from his extensive experience in change management in big businesses and blends it beautifully with his knowledge of philosophy, psychology, neuroscience and even derivative trading to produce a highly readable science-based and groundbreaking study of what has gone right and wrong in managing change in the business arena.

—David Bennett, former CEO of Alliance & Leicester PLC

In my 25 years of working with teams and organizations, this is the first book that actually uses science as the foundation of how organizations learn and develop rather than platitudes and well-worn but erroneous beliefs. One of Gibbons’ strengths is his willingness to tell it like it is, no matter how sacred the cow. Businesses and consultants will use this work for many years to come.

—Curtis Watkins, Master Coach

"Paul Gibbons rethinks change management with a 21st-century approach that exchanges cargo cult management for an evidence-based approach built on neuroscience and complexity sciences. Time is overdue to replace Kotter’s change model for something better suited for a complex world where change is continuous and not a one-time event, and where creative change continuously drives organizational improvement. Paul has succeeded with this, and in the process distills the best research into a book with a framework and ideas that will resonate with the modern leader and the Agile/Lean community. Buy it, read it, and place it on the bookshelf next to The Halo Effect, Switch, and The Fifth Discipline—in easy reach for rereading."

—Rolf E. Häsänen, Founder, Value at Work

Paul Gibbons has made a valuable contribution to the store of knowledge on change strategy and strategic decision making. By applying the latest findings from the science of decision making to his 25 years of practical in-the-trenches experience counseling executive teams, Gibbons has enabled anyone engaged in strategic decision making to raise their game.

—Dan Sweeney, Director, IEE, Daniels College of Business,

University of Denver

Few people bring Gibbons’ expertise, breadth of scholarship, depth of understanding, and range of experience to that most important of business practices: leading change.

—Robert Entenmann, Global Head E-business

for a major European bank

To my parents for giving me a love for science and learning, and to Conor and Luca for reminding me why it matters.

About the Author

Paul Gibbons has a 40-year career straddling international business and academia.

His research and writing explore how philosophy and science can be used to enlighten contemporary business thinking, debunk myth and pseudoscience, and solve practical business problems, including changing culture, developing leaders, and using analytics and evidence to make strategic decisions.

Paul’s academic background, starting in math, then in economics, neuroscience, psychology, and philosophy, allows him to bring perspectives to business not typically found in traditional business books.

His consulting career, mostly in Europe, included founding an award-winning teal Organization Development consulting firm, Future Considerations. Paul has coached dozens of CEOs – on strategy, change, and talent issues. His change experience includes clients such as Comcast, Shell, PwC, BP, Barclays, KPMG, British Airways, HSBC, Nokia, The Body Shop, Comcast, the NHS, and UK Ministers. He was the change management lead on a $1 billion program for the UK’s Department of Work and Pensions.

Paul has appeared in Microsoft’s Distinguished Author Program and at Google and appeared in the Wall Street Journal and the Financial Times. He is a Fellow of the Royal Society of Arts.

He now writes, hosts the award-winning Think Bigger Think Better philosophy podcast, plays competitive poker, chess, and bridge, and raises two boys in Colorado.

Preface to the 2019 edition

This book was first published in 2015 as The Science of Successful Organizational Change. I never liked the addition of the word successful. It struck me as superfluous and wordy; and, as you will see, a fair percentage of change fails. We need to talk about the science, or lack of, there too!

Despite nearly 40 years in business, the book was my first serious management book and looking back five years later; there isn’t anything I’d take back. I managed to avoid saying anything too daft. While the book didn’t receive Gladwell-like acclaim, it hovered around the top five on change management, organizational change, and leading change for five years. This noob was delighted.

Even those topics, brand new in 2013, behavioral science in change, evidence-based management, the psychology of risk, VUCA, decision making under uncertainty, cognitive biases and their effect on strategy, and change mythology, have stood up very well. The mission of the book, to corral the latest research in the human sciences in a book on change, was fulfilled. The challenging style, the debunking of learning styles, culture surveys, resistance to change, Kubler-Ross, Lewin’s change model, Myers-Briggs, Kotter’s change model, and dozens of more change ideas won some fans tired of same-old, same-old change books and annoyed some defenders of the old guard.

Some of the new ideas have stuck. Behavioral science in business is the new black, so much so that there is now a role called the Chief Behavioral Officer. The best-known cognitive biases, the confirmation bias, and the availability bias, have found their way into management education. Two-speed businesses, an agile out and a stable core in are a thing. Evidence-based management is blossoming. What hasn’t stuck particularly, that I can discern, is the material on psychology of risk – which I still think is fundamental to the governance of change programs. That, too, may be changing. Cognitive scientist and former poker champ, Annie Duke, wrote Thinking in Bets in 2018. She does an excellent and engaging job of illustrating why business decision making is more like poker than like chess – the subject of Chapter 3 within (Governance and the Psychology of Risk.)

One thing I did would change is the order of the chapters. I used a top-down logic – first organization capability building (agile enterprises), then strategy, then tactics (what people call change management.) In a book on change, I think people hunger for the behavioral and influencing material, and under that structure, they wait until chapter six to get that. The early chapters may be of most interest to very senior executives (how do you manage risk on multiple programs, how do you allocate capital, how do you avoid cognitive biases in strategy decisions?)

I suggest new readers skip to the chapters in which they are most interested.

As serendipitous proof of how much we need to debunk our field, I received an email from the most prestigious business school in the world (starts with H) on the day before this preface was written. This three-paragraph email, advertising a course in accelerating change, had no fewer than SEVEN statements that were pure nonsense or grossly misleading, either outright untruths or frothy appeals to fear. It takes a special skill to fit that many untruths in so few paragraphs – artistry more usually cultivated on Twitter.

Examples? The email ad talked about silencing naysayers. Srsly? Is that really the approach in the 21st-century? The 70% change failure rate was presented as fact when that is one of the smelliest of myths – the failure rate is about 40%. If you didn’t take their course, you would be one of the goons in the 70%, and if you did, you would join the elite. I would reproduce the email here, but lawsuits are no fun at all.

As you will see within, we need to question the ideas of change and leadership gurus much harder. Just because a guru, Kotter, Drucker, or Welch says something, does not make it true. In the management world, popularity is a proxy for expertise – a dangerous thing! Beyond the specific science in this book, there is the general notion of the scientific mindset: not Einstein, nor Newton, nor Hawking, nor Dawkins, nor Friedman, nor Krugman got a free pass in their scientific fields, let alone in their proclamations outside their area of expertise. Yet daily, some nonsense quote will appear on LinkedIn or Twitter, with a pretty image in the background, with the name of a management guru affixed, followed by thousands of likes, without ever a wait a minute, this makes no sense, what if… The Science of Organizational Change stands for having the mindset and courage to challenge orthodoxy, no matter from which expert it comes.

Some good things happened despite trying to take on the biggest names in the field. I was invited to speak in Microsoft’s Distinguished Author Program, sandwiched between Arianna Huffington and Carol Dweck (Stanford, growth mindset.) Then came a talk Google’s People function – equally intoxicating. Moreover, I’ve been invited to speak in exotic places such as Jamaica, Kuala Lumpur, Paris, Bergen, and Gary.

My rationale for republishing this book is update some of the contents and to make it broadly available at a reasonable price. New content includes interviews at Zappos, and discussions with IBM’s leadership. Diagrams have been added and some tables updated, however, if you own the first edition, you can probably pass this one by without affecting your quality of life. However. Read on!

Within months of this publishing date, I’m publishing IMPACT: Using Science to Change Behaviors, Hearts, and Minds. That book is a sequel taking some topics much deeper. The two strategic themes are Digital Transformation and Future of Work. The big question is: those are the what of change, the content of change programs, but do they affect the how of change, specifically how we lead it?

I’ve taken liberties with my voice that made it much more fun to write. It is more irreverent and more fun: quotes from Tupac, Homer, and Joyce in the same chapter, and some wisdom from Daenerys Targaryen, Epicurus, Feynman, and Inigo Montoya. I’ve added some humor (at least I think so) – which is never found in oh so serious management literature. I’ve been able to poke fun at consultants, which, even though they are an easy target, is always fun. I think the new book advances change leadership quite a great deal while managing not to be boring.

The new book has two chapters on behavioral science with up-to-the-minute case studies. There are two chapters on influencing that include using technology platforms to build constant engagement with millennial and gig workers. It has a chapter that discusses the Future of Work, and Digital Transformation – two of the buzziest of buzzwords in businesses today.

I want to thank all the people that have become followers around the world. I will name some names in the acknowledgments – you know who you are! I also have been running a podcast called Think Bigger Think Better, something like a Sam Harris wannabe podcast, ranked top-20 in philosophy six places behind Sam. (It caters to my other passions, philosophy, science, tech, economics, and environmental issues.)If you want to use any diagrams from the book, email me [e-mail].

If you want periodic cool stuff, controversial blogs, videos, book releases, and freebies, join my newsletter list at

paulgibbons.net.

Enjoy the book! If you do, and leave a review on Amazon, send me the review, and I will send you your choice of one my Ebooks.

Paul, Colorado, April 2019

Introduction

Change management needs to evolve, from pure craft to a science-based craft – much like chemistry emerged from alchemy, or medicine from witchcraft.

How to Set 3 Million Dollars on Fire

In March 1993, the derivatives market was booming, and banks— although intoxicated by the profits—were worried about the risks of these strange, complex instruments and how to control the armies of traders making all that money. The normally reserved world of British commercial banks had been taken over by brash traders using swarthy epithets as often as the gammas and deltas of the trade. Senior management loved the income, but did not understand the math, hated the new trading-room culture of risk, vulgarity, and aggression, and were at sea with how to manage a business they did not understand.

So they called in the cavalry: consultants.

PwC had assembled a team from MIT, Harvard, and Oxford to help Barclays develop a comprehensive Risk Management Framework. I was on the team as a math guy and because, as a former trader, I spoke the traders’ language, the gammas, the deltas, and the epithets.

We worked for months interviewing senior leaders, traders, and other risk experts by day and writing our reports by night. The result was 12 ring-bound volumes of several hundred pages each. The one featuring my contributions was filled with pages of equations describing how financial instruments behaved under various stresses using advanced statistics. The other volumes were similarly detailed and dense, suggesting what strategies, systems, accounting procedures, processes, and management practices Barclays should use to manage risk.

We charged 1.8 million pounds ($2.7 million), which was a hefty consulting fee for 1993.

And then, nothing happened.

What? you say, surely not nothing? To be more precise, our findings were presented to the board of directors who nodded vigorously at all the right times. Then we presented to the executive committee, to the managing directors, to the business unit heads, and to their teams. They all nodded and applauded. No PowerPoint slide was left unturned. Almost none of our recommendations became real business change.

Although they found our logic compelling, and our recommendations sound, Barclays failed to mind the gap, the one between agreeing with something and doing it. Barclays might as well have lit a bonfire on Lombard Street with the three million bucks (which, given London’s spring weather, would have been better use of their capital).

I was crushed. From Master of Universe Consultant to snake-oil peddler on my first project. The project team remains the smartest and most professional I had ever known. What went wrong? There seemed to be three questions:

How could our recommendations not have been implemented when Barclays was so worshipful of them?

If our recommendations were as good as the bankers and we thought, what else, besides more PowerPoint, should we have done to get them adopted?

When were they going to blow the whistle on consultants for charging huge fees, producing no results, and hopping off to the nextassignment?

Reports in Drawers and Personal Change

Little did I know that such epic fails were more the rule than the exception in strategy consulting; they have a name, the report in a drawer. Over the next 18 months, I worked on several strategy projects that soon decorated executive shelves and bottom drawers.

This professional epic fail paralleled one in my personal life.

As a teenager in 1980, I worked in cancer research before submitting myself to the sleepless nights of medical education. My research project involved studying the biochemistry of cancer to understand the effect of Vitamin A on skin cancers through its effect on DNA and RNA synthesis—by treating little white mice with a carcinogen from cigarette smoke. Despite this, and since the age of 14, I smoked a pack of Marlboro Red per day. At the lab, I would squirt the cigarette extract, watch the mice get cancer, and grab a quick smoke between experiments. While working at Barclays, I still frequented the parking lot for smoke breaks, so for almost 20 years, I had ignored all the science, some of which I produced firsthand, which told me I was killing myself one cancer stick at a time.

The link between the failed project at Barclays and my death wish was not lost on me. There must be a link between how I systematically defied in-your-face science, and how Barclays effectively ignored our advice on risk management.

This birthed a tremendous hunger: How do people change, and how do businesses make real change happen? How do good ideas get acted upon in the real world, and how do reports find their way from bottom drawers into hearts and minds? This seemed to be a problem at the root of human happiness, business prosperity, and how we manage ourselves as a society: the gap between intentions and actions, between knowing and doing, between espoused values and embodied values.

The equation seemed to be: CHANGE = E × X

E seemed to be expertise, knowledge, research, statistics, advice, reasons, rationality, and clear thinking. My strategy colleagues and I were good at all that. X was the bit that stumped me completely, that I knew nothing about—the special sauce that combined with reasons produced change. X had eluded me in my personal life, and now in my professional life. I wanted to make a difference, not just espouse grand theories, and to be someone who did not just talk a good game but could play ball. I wanted X.

I changed gears. For almost two decades, I lived, ate, and breathed organizational change. My immersion was obsessive: in its academic disciplines (psychology, sociology, Organization Development, and Organizational Behavior), training in Daryl Conner’s change toolkit, Californian self-actualization workshops, training as a counselor, working as a change manager in dozens of businesses, and teaching the advanced change management program to management consulting partners. (When people ask me how long it takes to become an OD/ change consultant, I generally say about a decade to be competent, another decade to be masterful.)

From the Laboratory to the Sweat Lodge

Then another problem reared its head. Little did I know that leaving the solid bedrock of science and reason for the world of change meant journeying to the opposite end of the spectrum—a world where ideas were much harder to test. At first, I accepted perspectives in books such as Gladwell’s The Tipping Point, or Goleman’s Emotional Intelligence uncritically, never wondering how much meat there was on the sandwich, or whether eating marshmallows really predicted success. It does not – I later found out that the charming story about children and marshmallows (Mischel’s research on emotional intelligence), presented as research fact by popular media, is questionable. It is a great story, however not eating the marshmallows was related to socio-economic class (the poor kids grabbed the candy.) Once you control for that, it predicts nothing about future success.

How the media popularizes and exaggerates research in psychology is the subject of Chapter 6, Misunderstanding Human Behavior. I loved the writings of 1990’s change gurus—for example, John Kotter (Harvard) and Tom Peters (McKinsey)—and at that time, I accepted their ideas uncritically because of their reputations. I was to learn later that both my heroes had feet of clay.

My new change colleagues and I talked about feelings, spirituality, socially constructed realities, presencing, living systems metaphors, ancient wisdom, consciousness, cultural memes, spiritual values, and stakeholder engagement. Then, as I traveled farther down the change rabbit hole, I found even funkier notions such as quantum leadership, conscious organizations, right-brained leadership, and leading from Source. The least funky of these, right-brained leadership, gets plenty of attention, but as I was to learn later, there is no such thing as a right-brained person or a right-brained leader. Psychological dispositions (such as being logical, or creative) are unrelated to hemispheric dominance. This myth, too, is exploded in Chapter 6.

So hungry was I for deeper insights, I attended Organization Development (OD) workshops (also attended by household names in that field) that began with an attunement to let in Spirit, and workshops to reveal the collective unconscious (Jung) of a business.

Clients even paid for this stuff. One workshop, hosted by consummate change professionals and attended by senior executives from a leading consumer goods company, used a labyrinth (a room-sized carpet that looks like a maze) to evoke insight and creativity. The premise was that if one walked around this ancient sacred structure with a question in mind, insight and creativity would emerge. I just got dizzy. Clients, traditionally discerning business people, suppressed whatever reservations they might have had.

I was now a stranger in a strange land. There was more to producing organizational and personal change than reasons and smarts, yet change theories had no science to back them up, and most of my fellow practitioners disdained science in favor of other ways of knowing – which began to occur to me a justification for believing in all sort of strange things.

I had found that the change world (Organization Development and change management) had an overlap in values and methods with humanistic psychologists, pop psychologists, therapists, 1960’s counterculture, and New Age spirituality. As great people-people, they made for outstanding facilitators, but I always had nagging doubts about how reliable methodologies guided by those underlying belief systems might be. I had explored the most extreme realms, hungering for meaning, for personal answers, and answers to the question, How do people and businesses change? On my search for meaning and insight, in those extreme realms, I had a lot of company; the demand for esoteric approaches from senior business people is enormous. In addition to labyrinths, I hosted Native American drumming workshops, encounter groups, Lego-play workshops, monastic retreats at monasteries, ropes courses, trust falls and trust walks, improvisational and standard theater sessions, yoga and tai-chi, psychodrama, and workshops with concert violinists and pianists. The clients for these workshops were all in the top 100 corporations in the world and household names. I would wager that 100% of companies in the Global 500 have used and still use at least one of those techniques.

While experimenting with those methods, I began to hunger for proof. Clients were happy and came back for more, so I left behind happier workplaces—but where was the evidence, was the change sustainable, and did business results follow from the increased engagement?

In parallel with the esoteric personal development and alternative workshops, I became an orthodox change practitioner – gaining multiple certifications in various methodologies. This change management looked more traditional: stakeholder engagement, team alignment, strategy facilitation, communications planning, organization design, and change-leader coaching. At the apex of my big-change experience, I became the change management leader on a project with a billion-dollar price tag, as well as dozens of systems implementations, strategy rollouts, restructurings, and HR transformations.

Yet, how much better proven were the more conventional tools I favored, such as business cases, process-mapping, organization performance models, risk registers, stakeholder analyses, and criterion matrices, than my friends’ labyrinths? Was it just a matter of taste? Although I became skeptical of some of the more esoteric approaches, I had equally little evidence to prove what I did.

The most shocking thing is that during more than 30 years in business, at the most senior levels, in the world’s biggest companies, dispensing consulting advice, no client ever asked me whether there was evidence to support the models, frameworks, tools, methods, and ideas I proposed using. Never. I worry, silly me, about using methods on billion-dollar projects that are based on beliefs for which there is skimpy evidence. Do we just use what looks good, what is in fashion, and what gurus say works? Do we use what methods we know best rather than the best methods? How do we evaluate methods side by side in this science-free world?

In 1999, I was passionate about spirituality. In those days, emotional intelligence was the rage – I wondered whether people bringing the spirituality to work would make more engaged and human workplaces, whether by bringing their whole selves to work, they would be more fulfilled. Value-centered leadership would become the norm. At the US Academy of Management, we formed a division called Management Spirituality and Religion. All people, we thought, had a spiritual dimension to their lives—either religion, spiritual practice, a deep sense of values, or a commitment to some kind of humanism. Our messianic quest was to introduce this idea to the Academy and into businesses. My contribution was to try to square the circle between the ineffability, unmeasurability (and sometimes wooliness) of spiritual ideas, and science. My research was (aridly) called Spirituality at Work: Definitions, Assumptions, and Validity Claims. (On my website for the curious.) I tried then to bring some hard science to where my passions lay, and to the softest part of the soft end of OD.

This book, in some ways, extends that project—can we bring some harder-edged concepts and some robust validity testing to the tools and ideas used by change leaders? Can we prove a certain kind of workshop works? Are our models and metaphors sound and useful?

Defenders of the Faith —

How to Prove Something Works

"When we work backward from results to figure out why those things happened, we are susceptible to a variety of cognitive traps, like assuming causation when there is only a correlation, or