Data for All

By John K. Thompson

Do you know what happens to your personal data when you are browsing, buying, or using apps? Discover how your data is harvested and exploited, and what you can do to access, delete, and monetize it.

Data for All empowers everyone—from tech experts to the general public—to control how third parties use personal data. Read this eye-opening book to learn:

• The types of data you generate with every action, every day
• Where your data is stored, who controls it, and how much money they make from it
• How you can manage access and monetization of your own data
• Restricting data access to only companies and organizations you want to support
• The history of how we think about data, and why that is changing
• The new data ecosystem being built right now for your benefit

The data you generate every day is the lifeblood of many large companies—and they make billions of dollars using it. In Data for All, bestselling author John K. Thompson outlines how this one-sided data economy is about to undergo a dramatic change. Thompson pulls back the curtain to reveal the true nature of data ownership, and how you can turn your data from a revenue stream for companies into a financial asset for your benefit.

Foreword by Thomas H. Davenport.

About the Technology

Do you know what happens to your personal data when you’re browsing and buying? New global laws are turning the tide on companies who make billions from your clicks, searches, and likes. This eye-opening book provides an inspiring vision of how you can take back control of the data you generate every day.

About the Book

Data for All gives you a step-by-step plan to transform your relationship with data and start earning a “data dividend”—hundreds or thousands of dollars paid out simply for your online activities. You’ll learn how to oversee who accesses your data, how much different types of data are worth, and how to keep private details private.

What’s Inside

• The types of data you generate with every action, every day
• How you can manage access and monetization of your own data
• The history of how we think about data, and why that is changing
• The new data ecosystem being built right now for your benefit

About the Reader

For anyone who is curious or concerned about how their data is used. No technical knowledge required.

About the Author

John K. Thompson is an international technology executive with over 37 years of experience in the fields of data, advanced analytics, and artificial intelligence.

Table of Contents

1 A history of data
2 How data works today
3 You and your data
4 Trust
5 Privacy
6 Moving from Open Data to Our Data
7 Derived data, synthetic data, and analytics
8 Looking forward: What’s next for our data?

• Language: English
• Publisher: Manning
• Release date: Aug 8, 2023
• ISBN: 9781638351931

Book preview

Inside front cover

Analytical Maturity Model

The Analytical Maturity Model illustrates how companies leverage and use data to derive intelligence and to make decisions related to customers, marketing, operations, strategy and more.

In Praise of Data for All...

John Thompson’s book, Data for All, is one of the more interesting ones I have read recently. It is one of the most honest, direct, pull-no-punches sources on one of the most important personal issues of our time. That issue is, ‘Should I undertake extraordinary efforts to prevent companies—tech and otherwise—from getting access to my personal data?’ I think the answer is yes. I’ve already changed some of my own behaviors after reading the book, and I suggest you do so as well. You have more to lose than you may think.

—Thomas H. Davenport,

Distinguished Professor, Babson College and Fellow,

MIT Initiative on the Digital Economy,

author of Competing on Analytics and The AI Advantage

John provides a cogent and concise treatment of the problem of ‘digital exhaust’ and what to do about the nearly infinite traces of ourselves that we create in the course of everyday activity. His voice is of someone who has lived with and thought about data from before it was trendy to do so. While John is appropriately blunt about the challenges ordinary people face in controlling their data, he is neither cynical or pessimistic, and therefore provides an actionable path toward greater agency around the data we generate.

—Thomas A. Finholt,

Dean, School of Information,

University of Michigan

How much time do you spend thinking deeply about the data you create, who is using that data, and how? It’s hard to know where to even start. John Thompson’s Data for All provides a wide-ranging non-technical overview of the past, present, and future of data, of topics including data privacy, open data, and data analytics. And for those looking to take control of their own data futures, this book is the best version of a ‘People’s Data Manifesto’ I’ve seen to date.

—David Steier,

Distinguished Service Professor,

Carnegie Mellon University

John Thompson and I have been discussing, debating, and collaborating on data and analytics projects for over 25 years. Data for All concisely summarizes and accurately outlines the coming data revolution and, at the same time, provides guidance as to how each individual and company can prepare themselves to own, manage, protect, and profit from their data. The world of data is changing rapidly. All of us should be ready to benefit from it, and this book is a practical guide to doing just that.

—Larry Smarr,

Distinguished Prof. Emeritus,

UC San Diego Department of Computer Science & Engineering

John Thompson is distinctively qualified by experience and insight to explain how the data economy works and why you need to know how it works. Like it or not, the data we generate critically conditions the terms on which we live and work and deal with each other and with the institutions—especially the private corporations—who capture and take advantage of that data. Data for All delivers an accessible yet comprehensive education about the world that we and our computers have created.

—William Janeway,

University of Cambridge, and Warburg Pincus

John tackles a topic that is more pertinent than ever—understanding the world of data, what data is being collected about you, and how that data might be used to influence what cereals you buy, what routes you take to work, and for whom you vote. John’s pragmatic style is designed to up-skill everyone in becoming data and decision literate so that we can make informed decisions in an imperfect world of constant disruption. The survival of the human race depends on us mastering that skill.

—Bill Schmarzo,

Customer Data Innovation, Dell Technologies

John Thompson is uniquely qualified to explain the complexities and value of truly understanding data. He accurately puts data in context and explains how the use of data has revolutionized our lives and the growth of businesses. We have moved into an era where data has become the most important asset on the planet. Many companies like Google and Amazon depend on data to grow their revenue. Ironically, for these data-focused companies, the customer and users of these systems become the product that help these companies grow into global leaders. John provides an important perspective so that business leaders and citizens understand the importance of understanding how data is central to our lives. Being able to understand the context and uses of data will help everyone make better and more informed decisions. We are in the midst of a data revolution that requires this book to truly understand what data means to you. Therefore, this book is critical to gaining an understanding of data in context.

—Judith Hurwitz,

Chief Evangelist at Geminos, Data & AI thought leader, and Author

In Data for All, industry veteran John Thompson has written the quintessential book for the digital generation, recognizing how society is at a crossroads in who owns data and who creates data. John offers practical advice on how we can restore the balance in power between companies who exploit our data and us as individual owners of our data. With its masterful storytelling, I hope this book will become required reading in high schools and business schools.

—Cindi Howson,

Chief Data Strategy Officer of ThoughtSpot and host of The Data Chief podcast

Data for All

John K. Thompson

Foreword by Thomas H. Davenport

To comment go to liveBook

Manning

Shelter Island

For more information on this and other Manning titles go to

www.manning.com

Copyright

For online information and ordering of these  and other Manning books, please visit www.manning.com. The publisher offers discounts on these books when ordered in quantity.

For more information, please contact

Special Sales Department

Manning Publications Co.

20 Baldwin Road

PO Box 761

Shelter Island, NY 11964

Email: [e-mail]

©2023 by Manning Publications Co. All rights reserved.

No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in any form or by means electronic, mechanical, photocopying, or otherwise, without prior written permission of the publisher.

Many of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designations appear in the book, and Manning Publications was aware of a trademark claim, the designations have been printed in initial caps or all caps.

♾ Recognizing the importance of preserving what has been written, it is Manning’s policy to have the books we publish printed on acid-free paper, and we exert our best efforts to that end. Recognizing also our responsibility to conserve the resources of our planet, Manning books are printed on paper that is at least 15 percent recycled and processed without the use of elemental chlorine.

ISBN: 9781633438774

brief contents

  1 A history of data

  2 How data works today

  3 You and your data

  4 Trust

  5 Privacy

  6 Moving from Open Data to Our Data

  7 Derived data, synthetic data, and analytics

  8 Looking forward: What’s next for our data?

Appendix A. Timeline of the development of the Internet

Appendix B. A Declaration of the Independence of Cyberspace by John Perry Barlow

Appendix C. Washington Post-Schar School Trust in Technology Survey

Appendix D. Data Sources about you and your world

Appendix E. A complete list of 2021 consumer data privacy legislation

Appendix F. Open Data by US State

contents

Front matter

foreword

preface

acknowledgments

about this book

about the author

about the cover illustration

  1 A history of data

1.1   A concerning situation

Life cycle of a video, picture, text, email, or file

All your online actions create permanent records of your activity

Intelligent choices

1.2   An example: Genetic testing and reporting

Genetic sequencing

1.3   The beginnings of modern data

Commercial data and analytics: Data as a valuable commodity

How our data rights and right to compensation were diverted

Let’s jump forward to the late 20th century in the UK 8

1.4   Modern data in the present day

The dangers of social media

People are waking up to the abuse of our data

And here we find ourselves

1.5   The new and current view and value of data

Data is the new sand

Data is the new sun

Data is the new gold

Data is the new currency

Data is the new plastic

Data is the new bacon

Data is the new future

1.6   Wrapping up

  2 How data works today

2.1   Where does data originate?

2.2   The life cycle of data

Location services

Do you like being tracked and monitored?

2.3   The past is the past, but the future has not been written yet

2.4   On my way for the day... A Grand Day Out ...

Your working day: Do you know who is watching you?

Browsing data: What are you looking at?

Tangential interests vs. core interests

Infrequent or episodic transactions

Small repetitive transactions

Data and the power of multiple data sources

Fresh or stale, just like bread

But day-old bread has its uses too

2.5   Whose interests are being served by leveraging your data?

This can only continue if we allow it

Is it really possible to exit platforms?

Is it worth what you and we are paying?

Why is the online world different than the offline world?

2.6   Who are you aligned with?

What is in our best interests?

Many paths to liberation: Beginning to control your data. Detoxing your data

2.7   Final thoughts

  3 You and your data

3.1   Origins of the internet and World Wide Web

3.2   Current views and attitudes toward data

3.3   Some people don’t have the luxury of thinking about data

People who live in poverty

People who are living at subsistence levels

Reducing poverty and why it matters

3.4   How the general population thinks about data today

3.5   How do you think about data today?

You own your data

Think about how your data is being treated

Think about your children

Worldwide data creation

Thinking about how to manage your data

Reaping a monetary reward from companies using your data

Carrot and stick

3.6   Green shoots and new beginnings

3.7   Final thoughts

  4 Trust

4.1   Forces that are working against our best interests

4.2   Trust

Infotainment is not news, and alternative facts do not exist

Citizenship and our duty to the objective truth

4.3   Trust in government

4.4   Trust in business and business leaders

Trust in technology companies

Why do people feel they need to be connected to technology?

Trust in media companies

4.5   Trust is lost; time for a change

4.6   Technology and media companies are making bank from your data

What is the value of data, and how much can it mean to each individual?

4.7   Governmental regulations

The European Union

Australia

The United States

4.8   Effect of data laws around the globe

4.9   Final thoughts

  5 Privacy

5.1   Privacy defined

5.2   Privacy throughout history

The internet has not eliminated privacy

Privacy crosses sociology, psychology, and basic human conditions

The need for privacy compared to the right of privacy

5.3   Psychology and privacy

5.4   We need privacy like we need sleep

5.5   Privacy and secrecy

5.6   Two sides of privacy

5.7   Privacy and human behavior

5.8   Privacy precepts

5.9   Poor privacy policies

5.10 Enlightened privacy policies and related data protection

5.11 Privacy laws and regulations

5.12 Privacy and data ownership

5.13 Privacy and technology

5.14 Privacy and trust

5.15 Final thoughts

  6 Moving from Open Data to Our Data

6.1   Data from many sources drives value

6.2   Data and analytics at dinner parties

6.3   Data can be used as a weapon

6.4   The horse is out of the barn, let’s go riding ...

6.5   New and modern approaches to data

Open Data defined

Open Data’s beginnings

Open Data today

Governmental Open Data policies

Open Data: US federal and state governments

6.6   Data exchanges

Types of data exchanges

6.7   Data intermediaries, data pools, and data unions

6.8   Data commons

6.9   Final thoughts

  7 Derived data, synthetic data, and analytics

7.1   Data lineage

7.2   Forms of data

Natural or raw data

Aggregated data

Derived data

Synthetic data

Simulated data

Optimization data

Machine-generated data, Internet of Things data

7.3   Analytics and data

Analytics continuum

Stage 0: Descriptive statistics

Stage 1: Predictive analytics

Stage 2: Prescriptive analytics

Stage 3: Simulation

Stage 4: Optimization

7.4   Augmented intelligence

7.5   Data scientists and statisticians

7.6   Final thoughts

  8 Looking forward: What’s next for our data?

8.1   Where do we go from here?

National and state laws will lead and guide the changes

The new data ecosystem will create and realign flows of money

Innovation and change will create friction and opportunity

It may be hard to see, but change is well underway

8.2   A day in the life of your data... well, actually two days

Data you create each day will not change

All the data we create in our daily lives: An example

New data streams from our roads and cars

8.3   What’s different in 2025?

8.4   Data intermediaries (DIs)

Creation of DIs

A new regulatory environment for data

Managing your data in 2025

8.5   Dimensions of data access

8.6   What DIs will do for you

Setting revenue maximization as your primary objective

Many objectives

Setting multiple objectives

8.7   Dimensions of data monetization

How much money will your data dividend be?

Data ownership and licensing

Beyond the cash, what is the value to you?

8.8   So what do we do today?

8.9   Final thoughts

Appendix A. Timeline of the development of the Internet

Appendix B. A Declaration of the Independence of Cyberspace by John Perry Barlow

Appendix C. Washington Post-Schar School Trust in Technology Survey

Appendix D. Data Sources about you and your world

Appendix E. A complete list of 2021 consumer data privacy legislation

Appendix F. Open Data by US State

index

front matter

foreword

John Thompson’s book, Data for All, for which I am pleased to write a few words of introduction, is one of the more interesting ones I have read recently. I am not yet entirely sure that I agree with everything he writes—there are some pretty strong statements about individuals and companies in here—but I am very glad I read it. It is one of the most honest, direct, pull-no-punches sources on one of the most important personal issues of our time. That issue is, Should I undertake extraordinary efforts to prevent companies—tech and otherwise—from getting access to my personal data?

I think the answer is yes, but my own thinking has evolved somewhat on this issue. I love John’s personal and historical stories in this book, so I will provide one too. In 1998 I saw the movie An Enemy of the State with Will Smith in the starring role and an excellent cast overall. I highly recommend seeing the movie, even if you are not interested in data privacy. But if you are, it raises a particularly interesting set of issues. Will Smith plays a lawyer named Bobby Dean. He accidentally ends up with some highly damaging information about a government intelligence agency. A corrupt member of that agency is able to use all of the data about Dean to track him down instantaneously, tap into all his calls, plant all sorts of false information about him, and generally make his life miserable.

In 1998 that wasn’t really possible; data couldn’t be captured, integrated, and acted upon instantaneously. Today, however, the Will Smith character would have left a much richer data trail, and the communications and computing power available to the intelligence agency would make it relatively easy to track and find him (and perhaps to shoot him dead with a drone!).

The situation is somewhat similar with today’s tech companies, or just any company that wants to sell you something. Like the intelligence industry near the turn of the century, their tools aren’t really that good yet. I constantly receive targeted and personalized ads that are of no interest to me at all. I am often retargeted with offers for goods I once searched for or stumbled upon online. The assumption isn’t very sophisticated—if I was once interested, I must still be. Even if I actually bought the thing, the advertiser doesn’t seem to be aware, or assumes I want two.

But like the intelligence industry, advertisers will eventually get much better at targeting their messages to consumers based on what they know about them. At some point we will undoubtedly all be shocked, embarrassed, or creeped out by the things that are known about us and acted upon by commercial organizations. As Thompson points out, tech companies—and sometimes just retailers and consumer goods companies—know who and where you are, what you’ve bought in the past (at their site and elsewhere), what you look at on social media, what you say to Alexa, what you email to your friends and colleagues, etc. It’s clearly too much.

And the tech industry has never been sympathetic or empathetic to our privacy concerns. As then-Sun Microsystems CEO Scott McNealy famously replied in 1999 when a group of reporters asked him about data privacy, You have zero privacy anyway. Get over it. Ten years later, Eric Schmidt, at the time the CEO of Google, said in a CNBC interview in 2009 that If you have something that you don’t want anyone to know, maybe you shouldn’t be doing it in the first place. That’s a pretty scary thought, given that probably everyone has a few things they wouldn’t want anyone to know. Most tech executives have become a bit more savvy in what they admit about privacy, but outside of Apple, which has wisely embraced privacy as a marketing tool, no one in Silicon Valley seems to take the issue seriously.

In short, John Thompson is correct to warn you about the perils of your online data, and it’s clear that—at least in the US—nobody is going to solve this problem but you. This book gives you the information and the recommendations necessary to make things better. Now very few of us will want to go off the grid entirely, and it’s really hard to do anyway. Evan Ratliff, a writer for Wired, tried (with a big cash reward if he succeeded) to do it for a month in 1999, took many extraordinary measures, and was found in 23 days.

But as Thompson details in this book, there are many steps you can take to reduce the exposure of your data and maintain some degree of privacy. I’ve already changed some of my own behaviors after reading the book, and I suggest you do so as well. You have more to lose than you may think.

—

Thomas H. Davenport,

  Distinguished Professor, Babson College and Fellow,

  MIT Initiative on the Digital Economy,

  author of Competing on Analytics and The AI Advantage

preface

Recorded data (day-tuh) has been part of the human experience for thousands of years. The first piece of recorded and validated data is from the area now known as the Democratic Republic of Congo. The 20,000-year-old Ishango Bone—found near one of the sources of the Nile—seems to use matched tally marks on the thigh bone of a baboon for recording counts. Counts of what, we are not certain, but it is clear that marks on the bone were made with a purpose and over time.

Five thousand years ago, the Sumerians took the art and science of accounting to a new level. All manner of transactions, interactions, and exchanges between people across the cities of Mesopotamia were recorded using cuneiform on small clay tablets that could be held easily in one hand. The thousands of clay tablets can be considered the first handheld devices! Those small, terra-cotta clay tablets illustrate the extent and widespread use of data to record transactions of all types.

People have traded and transacted for nearly all of human history. I am sure that there are data and records that have yet to be discovered that predate even the Ishango Bone. At this point in time, we have discovered that our ancestors were interested in knowing and recording how many vessels of wine were bought and sold, how many heads of livestock were traded, and we have learned what transactions represented exchanges executed on that day and the promises that were made in the market each day for future trades and transactions, and much more.

In addition to the basic numbers of items traded and the prices of those items, other related data was recorded on those tablets, including the conditions that would govern the proportions and amounts, which would change based on market conditions and outcomes of external factors and events. The conditions recorded are the precursors of modern trading systems—puts and calls, short and long sales, and more.

Our forebearers realized that creating a written record that all parties were present for, and agreed to, was the most effective way to ensure that the details of any and all transactions and covenants were accurately documented and certified by not only the primary parties to the transaction, but also additional interested parties and perhaps impartial third-party observers. And for at least tens of thousands of years, we have incontrovertible proof that humans have created data to support our memory of the facts of interactions, promises, terms, conditions, timing, restrictions, correspondence, transactions and more.

The Sumerian clay tablets show that current and future transactions of considerable complexity were impressed upon the soft clay and hardened for all history. Those early records that were lost to history and found centuries later are a harbinger of the durability of data. These clay tablets are a corollary to the data we create and carelessly dump onto the internet and record for posterity on the World Wide Web.

I think back to my years as a college student, and even back to some of the things that I thought and said in high school, and I am grateful that there were no easy ways to record audio or photographs of those foolish thoughts or acts. I was trying on concepts and positions. I was encouraged by teaching and challenged by adults. I tried arguments for the sake of hearing how they worked or landed.

One example of the exuberance and folly of youth was that I represented myself in traffic court at 16. I am certain that my arguments were not persuasive, or logical. Clearly, they were not successful. I am sure that there are written records of the interactions between the judge and me, and I am equally confident that I would be embarrassed to read what I said and posited, but I didn’t have any experience in court, and I tried arguments that I am sure amused the judge. I know that I did not offend him, but in the end I was directed to leave the court room and pay my fines.

Think back to your youth—did you do or say things that you may have regretted soon after? Have you done things that you are grateful are not documented in pictures, video, and audio? I am sure that this is the case for many of us.

Our children are not so lucky as to be given the chance to allow youthful folly to be so easily relegated to the dustbin of history. They will have an indelible and enduring record of some truly unfortunate moments that illustrated their early attempts to try out different courses of action or possible approaches to situations, or simply their regrettable lapses of judgment. Young people should be allowed to explore thoughts and ideas without those explorations haunting them for the rest of their lives. I am not advocating for covering up misdeeds or shirking responsibility for acts that are illegal, unethical, or harmful. What I am saying is that one episode of trying out a discredited philosophy from the pages of history or a new and novel ideology should not brand a person for the remainder of their lives.

A few of my college friends were enthralled by Marx, Engels, socialism, communism, and other early twentieth century movements, governments, and social experiments. We were taught history and we tried on these arguments and positions for the sake of learning and experimentation. Some of those late-night discussions went far down various rabbit holes. If those discussions were to come to light via video, it would take a significant amount of explaining to lay out the context of those verbal exchanges. And we would have to explain our truly unfortunate fashion choices as well, and that would likely be more embarrassing than explaining some late-night diatribe about the value of labor, or about property rights as a basis of an economic system.

The point to be made here is that data has a long life; longer than you can imagine at first glance. Even data that was created to mark the trade of a handful of livestock 20,000 years ago can be read, remembered, and analyzed today. With our ubiquitous computing networks and modern electronic infrastructure, the audio files you record, the pictures you take, and the videos you record and send can, and might, outlive generations of your descendants. It’s something to keep in mind as you snap pictures, share documents, and record voicemails.

acknowledgments

It all started with Jennifer H. Thompson saying, You should write a book. My response was, I have no time. As with many things in life, she persisted, and now writing is a pure joy. I cannot conceive of not writing. As with all good things in life, she knows where true joy and fulfillment can be found and enjoyed. Thank you, Jennifer. You are my partner in all things in this life and maybe others as well. We have grown, smiled, cried, and laughed all along. You are the shining light of my life. Thank you for agreeing to take my phone number on the corner of Southport and Belmont on that sunny July day in Chicago. Thank you for being the love and light of my life.

Irene Ellis was my maternal grandmother. I don’t know what she saw in me, and she never told me, but all I know is that she believed in me and loved me when it felt like no one else did. From my earliest memories, she was always there with a kind smile and a hug. She was gentle, interested, and interesting to me. Thank you for showing me that all you need is one person to believe in you to make it possible to believe in yourself, and for the chocolate-covered cherries.

Kathryn Thompson is a force. Creative, intelligent, empathetic, sympathetic, and deeply caring. I am amazed at the breadth of your creativity and range of interests. Our time together is always a joy, and I love to spend time talking and laughing. You have been so generous in teaching me about how the world works and how you see life.