Big Data: the Revolution That Is Transforming Our Work, Market and World

By PAT NAKAMOTO

"Big Data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it." – Dan Ariely

Within 2 days we produce the same amount of data generated by at the beginning of the civilization until 2003", said Eric Schmidt in 2010. According to IBM, by 2020 the world will have generated a mass of data on the order of 40 zettabyte (1021Byte). Just think, for example, of digital content such as photos, videos, blogs, posts, and everything that revolves around social networks; only Facebook marks 30 billion pieces of content each month shared by its users. The explosion of social networks, combined with the emergence of smartphones, justifies the fact that one of the recurring terms of recent years in the field of innovation, marketing and IT is "Big Data".

The term Big Data indicates data produced in massive quantities, with remarkable rapidity and in the most diverse formats, which require technologies and resources that go far beyond conventional data management and storage systems. In order to obtain from the use of this data the maximum results in the shortest possible time or even in real time, specific tools with high computing capabilities are necessary.

In the United States, the term has become a key concept in 2009 when President Obama created the position of Chief Technology Officer as part of his America's Innovation Strategy.

But what does the Big Data phenomenon mean? Is the proliferation of data simply the sign of an increasingly invasive world? Or is there something more to it?

Pat Nakamoto will guide you through the discovery of the world of Big data, which, according to experts, in the near future could become the new gold or oil, in what is a real Data Driven economy.

• Language: English
• Publisher: PAT NAKAMOTO
• Release date: Jul 17, 2018
• ISBN: 9781981151165

Book preview

Introduction

Big Data is like teenage sex: everyone talks about it, nobody really knows how to do it, everyone thinks everyone else is doing it, so everyone claims they are doing it.

– Dan Ariely

A new class of assets

Every day in the world, we create 2.5 exabytes of data (2.5 quintillion bytes a day). The pace with which the human race is producing data has become overwhelming; just think that the 90% of existing data has been created only during the last five years [IBM sources]. 

This data is recorded everywhere: for example, climate sensors, post on social media, video and digital images, purchases transaction records, and cell-phone GPS signals. Over the last few years, technological evolution has led to a remarkable increase in the number of devices capable of automating operations, both in the productive world and in private life. Such devices generate a huge amount of data every day; and its size is expected to grow exponentially in the coming years. Estimates speak of unprecedented growth, defined as a "torrent flowing into every area of ​​the global economy" (The Economist, 2010). In terms of numbers, the expected growth is around forty percentage points for next year.

Just think, for example, of digital content such as photos, videos, blog posts, and everything that revolves around social networks; only Facebook marks 30 billion pieces of content each month shared by its users (Manyika, et al., 2011). The explosion of social networks, combined with the emergence of smartphones, justifies the fact that in recent years one of the most recurring term in the field of innovation, marketing and IT is "Big Data".

In the United States, the term has become a key concept in 2009 when President Obama created the position of Chief Technology Officer as part of his America's Innovation Strategy. The term Big Data indicates data produced in massive quantities, with remarkable rapidity and in the most diverse formats, which requires technologies and resources that go far beyond conventional data management and storage systems. In order to obtain the maximum results in the shortest possible time - or even in real time - from this data, specific tools with high computing capabilities have become essential. Big Data is already a reality, and the growth of the IT industry will mean it is produced in an exponentially larger amount. Just think that eBay has three distributed storage systems for a total capacity of 90PB; Walmart’s computer systems each hour process near to one million online shopping transactions, which are recorded on databases that contain 2.5 petabytes of data; Facebook manages 50 billion photos in its databases, and of course, these numbers are growing steadily.

But what does the Big Data phenomenon mean? Is the proliferation of data simply the sign of an increasingly invasive world? Or is there something more to it?

Most of the research up to now has focused on the question of the volume of data. You surely already have heard about the incredible amount of information constantly stored in our business data centers. It is obvious that this aspect captured the attention of most researchers, yet you have to realize that data storage is just the surface of the problem.

In the economic sphere, the awareness of how Big Data's smart management can help to improve decision-making and performance is growing. Business-Intelligence certified software developers and software providers working on Big Data are increasing, transforming the management of this data into a rapidly expanding industry with actors in both the private and open source communities. Private field contributions already exist through companies such as SAP, Oracle, HP and IBM, but the decisive impetus for innovation in Big Data management is coming from the open source world, through the use of software such as Apache Hadoop, Cascading, Apache HBase, MongoDB, Apache Storm and others.

What is the real game of Big Data? Converting large, heterogeneous and rapidly variable data into usable information through patterns of analysis: this is the Big Data uppermost challenge. It is therefore essential to understand what the real opportunities are, and what are the implications that can arise from the constant use of technologies capable of collecting and analyzing large amounts of data; only in this way will it be possible to understand the core value of Big Data.  Which, according to experts, in the near future could represent a new gold or oil, becoming a real asset class, in what we could call a Data Driven economy.

Chapter 1.  Introduction to Big Data

In God we trust. All others must bring data.

– W. Edwards Deming

Notes to this Chapter

The aim of this Chapter is to introduce the concept of Big Data and its main features,