D3: Modern Web Visualization: Exploratory Visualizations, Interactive Charts, 2D Web Graphics, and Data-Driven Visual Representations (English Edition)

By Victor M Garcia Sanabria

Picking the most appropriate tool is essential whenever you develop a dynamic chart online. D3.js is a clear winner here. The author of this book set out to delve deeply into the software engineering community's most widely used charting library.

From data requests to scale and style, and even some experiment charts, you'll have a firm grasp on everything you need to know to create your charts. You can make static charts much more engaging by incorporating elements such as tooltips, interactions, legends, and selectors.

The book explains how to make a simple chart in easy-to-understand steps. Each chapter in the book will deepen your understanding of the different components of charts and the options that D3.js provides for making them. The book dives deeply into the code and examines various charting examples for different use cases. This book will help you learn to create charts by constantly referencing all chapters.

• Language: English
• Publisher: BPB Online LLP
• Release date: Nov 28, 2022
• ISBN: 9789355511027

Book preview

CHAPTER 1

Getting Started

The world is now changing so fast, so it is hard to know what technology one needs to learn. By the time a technology is mastered, it could have been rendered obsolete by another. There are some " rules of thumb " that one can follow, but in general, there will be a little bit of faith involved. Here are some potentials guidelines to follow:

Learn a classic: Traditional, or classic technologies, as we call them, are immortal. They have been around for long enough to convey a sense of security, so people keep learning it, if only for that reason. Also, there is already so much code written with it that you will very easily find a project to work on. Examples of this can be C, C++, or Java.

Learn a supported tech: Even if it is an emerging or new technology, if it has been invented by Google, Microsoft, Apple, or Facebook, it is a good idea to learn it. They will continue investing in it for a long time. Examples of these can be .Net, React, Swift, or Go.

If it is not supported by a giant, make sure it is by the public: It may not be supported by any of the top firms in Silicon Valley, but languages such as Python or JavaScript are right on the top of the curve. They are up-trending, so to speak. They have so many users that it is self-supported. A simple google search will tell you which are the top languages in GitHub (one of my favorite rankings) or Stack Overflow (see the following note).

Languages for upper tech trends: If the language is new and not supported by a giant, it may be a bit risky. That can be mitigated if it is a technology destined to make a modern problem easy to solve. In the world of Data Science, R is the classic (which has seen an uptrend with the obvious increase of Data Scientists), and Python is the one winning the battle, but there are upcoming techs that are destined for Data Analysis, like Julia, that may be worth a shot. As with all bets, if it so happens that Julia starts picking up and you already have a couple of years of experience, chances are you will be offered a nice sum of money for your services.

Note: If you do not know what Stack Overflow is, I seriously advise you to google it right now. It is going to be one of your favorite resources very soon, and it is the number one Q&A website for code, after all.

But my ultimate suggestion is that you learn for the sake of learning because, in a constantly evolving world, the skill you should be the most comfortable with is learning itself. When I was presented with the same dilemma, that is exactly how I made the decision. I had to choose a technology for chart design, to create a new reporting tool. It has to have flexibility, as it was a new project and those are mutating in nature.

Structure

In this chapter, we will cover the following topics:

How to read this book?

Who is this book for?

Before we start

What is D3?

Make a new directory

Starting a server

Creating our index.html

Copying an existing chart

Visualizing our first chart

Objectives

After studying this chapter, you will be able to:

Understand D3.js and its advantages

Explain the way the book targets its audience and see if you fit in it.

Set up a work environment

Get our first visualization up and running

Understand the main procedure of a simple chart

How to read this book?

So, we know we want to learn D3.js, but how to learn it best? As per our experience, there are two rules we will normally follow:

It is the teacher’s job to flatten the learning curve. As we mentioned in the previous section, the world changes so fast that learning should be the main skill you need to master. So, this book will be more useful if it teaches you fast, and you can start working away quickly and experiment on your own.

Show how useful the tech can be and how it can solve problems. Learn by solving those problems. As you become better, you will find that your visualizations improve when you ask the right questions, and to know which questions are better to ask, you need to learn the core and meaning of them.

For that purpose, we are dividing the book into two sections. In the first section, we will dive into code very fast. We will teach you how to get familiar with using D3.js resources and implementing examples straight away. We will cover most of the things D3.js was designed to do, and we will learn some tips and tricks. When you have finished that section, you will be comfortable developing charts on your own and solving most of the challenges you will have to face as a D3.js developer. In the second section, we learn how to make specific types of charts using D3 and when it is right to use them. This is the fastest way to become a good data visualization specialist and even the right path for data analysts. We will continuously refer to the first section, as in all cases, we will be learning by developing examples in which one aspect of what I am trying to teach is prominent, and therefore, easy to understand.

For all cases, we completely recommend that you follow along and try to code things yourself. I will also pose some interesting challenges in the chapters that will help you find out if you have understood the main takeaways.

Who is this book for?

Even though the book covers the basics of D3.js from the beginning, it is not within the scope of the book to cover JavaScript fundamentals. Unfortunately, we would be diverting from the aim of the book if we had to stop to explain JavaScript and doing no one any favors. We will keep the code simple so that it is easy to follow, but if you get stuck at any point due to basics, please stop to check the part you do not know as well. We are going to be using ES6, D3.js v4, and v5. They are similar enough. We will get into these differences later on. So, it is clear that we are presuming some knowledge of JavaScript in order for you to be able to follow along.

So, who is this book directed at?

Front-end developers, who need to include charts in their work or even develop new charts that nobody else can make.

Data Analysts and Data Scientists, who want to write their own charts but do not want to do it in Python because they prefer to have them on Web technology.

Data Engineers, who want to create their own dashboards.

Other data people, who just want to get better at visualizations. If that is your case, then presumably, the second half of the book is going to be more interesting. Hopefully, once you are finished, you will have a new appetite for the world of coding.

Anyone with interest in charts.

Anyone with interest in data.

Anyone who loves learning.

Before we start

We have already made a case for D3.js, but before we move to the first chapter, let us make sure that you are aware of the pros and cons of D3.js, especially in comparison with the rest of the existing alternatives. Due to its comprehensive nature, D3.js is generally an option that will do what you need. Whether D3.js will do what you need the way you need it and in the timeframe that you have is a different story, and it will largely depend on your skill and experience.

What other options are there? There are so many options in the market that it would be impossible to mention them all, but I will try to make a comparison of the main ones and their pros and cons versus D٣. Please refer to the following table:

Table 1.1: Visualization libraries

Surprisingly enough, most of these are built on D3. D3.js allows you to define each element in your chart, design new types of charts and guarantee full control over appearance and behavior. This is why it is the most flexible on the list. C٣.js and NVD٣.js are two good options even to get started with D٣.js. These contain prebuilt examples of D٣, allowing you to have more or less the same amount of control, with some work done for you. The only thing you are sacrificing is the way the implementation happens. They are also compatible with D٣, so you would be able to use them together if you wanted.

Charts.js is the most popular choice out there for people who want to have an open-source chart library but only want their charts and want to do no chart design. It requires no effort to get a chart running. Obviously, the amount of control you have is far more contained, but they are increasing functionality, and it is possibly the most time-effective option. The main problem is that once you get started, you are stuck with them and have to start from scratch if you want something else. Plus, they all look the same, and everyone will know you went for the easy option. Very much "straight out of the shelf" work.

Plotly is some sort of intermediate solution. It contains many charts, with lots of flexibility. It is a library with a lot of data science options, and it even has a Python version. It is a very good candidate for quick dashboards, as it has other associated tools. It also has a premium version that is supported, which is what you will eventually end up with. A very good solution for start-ups. Even if it does not provide the flexibility that D٣.js will, it is worth navigating into their documentation to see if their integration and customization are sufficient for you.

ECharts and Highcharts are my two proposed solutions if you want something different; they are very easy to play with and with a lot of support. ECharts is like an upgraded version of Charts.js, even though it has no commercial connection with them. It is a Chinese tool (some of its documentation is in Chinese only) with many options. It has a very powerful API that allows a lot of control over the charts’ appearance (it even has a template builder incorporated, but alas, it is one of those things that remain to be translated to English), and you can render the chart in both SVG and bitmap without losing functionality. Highcharts has been a professional solution for a long time, providing similar functionality to that of eCharts (having man-prebuilt chart types) and allowing a lot of control over it. It only has a premium version, and it is the only paid library that you should recommend.

If you have read until here and you still like the idea of learning D3, there is only one thing left to say: congratulations!

Now that we have an overview of all the available options, it should be clear why I recommend D3.js. My favorite feature is its versatility and its capacity to adapt to anything you need. If you work on it long enough, you will have had time to develop your own library derived from D3, and you will be a unique asset to any visualization company.

We are now ready to start diving into D3 and the world of visualization.

What is D3?

D3 stands for Data-Driven Documents. Indeed, the origin of your chart will always be the data, which is also the way you should plan your chart. This does not mean that you cannot use D3.js without a data set. In fact, if you have ever used jQuery (unless you are very new to front-end development, you probably have), you will find that D3 has much in common with it to the point of making itself redundant. They both work by selecting certain nodes of the DOM.

Note: DOM stands for Document Object Model. You should be very familiar with this term by now, but if you are not, I suggest you start doing so right away. Here is a good reference if you do not know where to start: https://developer.mozilla.org/en-US/docs/Web/API/Document_Object_Model/Introduction

This makes D3 perfect in combination with modern frameworks, such as React or Vue, and in substitution for jQuery. People are starting to leave it behind anyway.

The official resources for D3 are available online at the following links:

Official site: https://d3js.org/

Blocks, the biggest D3 repo out there: https://bl.ocks.org/

The only thing inside them that is relatively important is that it was created by Mike Bostock, to whom I thank for such a marvelous piece of work. However, both the official site and Blocks are handy pages that we may refer to in the future. Among other things, they are full of good examples that we can copy.

Copying is, indeed, the first and most common way of learning. For our first example, we are going to get an existing visualization from one of these sites, and we are going to make it work on our local machine.

Let us get started!

Make a new directory

Let us start by creating a folder to work on. You can do so in many ways. Navigate to someplace you are comfortable in your machine and create a new directory.

Both in Windows and in Mac (and I am assuming it would be similar on Linux distributions), you can create it using your mouse (right-click). If you prefer, you can also do it with the command line:

Windows

In Windows 10:

shift + right click / Open PowerShell window here

Then, in the PowerShell,

md -Name newDirName

Where NewDirName is the name, you want to give to your new directory.

cd NewDirName

Will get you inside the newly created folder.

MAC OS

In Mac, use Spotlight: Cmd + Spacebar to search for the Terminal (which is the equivalent of the Power Shell or the Command Line Tool in Windows).

Then, navigate to the location you want to start your project in.

Now:

mkdir yourChosenFolderName

And now, to navigate inside:

cd yourChosenFolderName

This is going to be where we keep all of our project files. We can have several projects if we want to, but for the moment, it is better if we stick to just one.

Starting a server

Maybe I should have announced this before, but I wanted to get you on code as soon as possible. In order to run our code, we are going to have to run a virtual server on our machine; otherwise, the charts will not render. In case you did not know, browsers treat files downloaded from a server and local files differently (although it might not look so at first glance). This makes it completely essential to start a virtual server in order to see our D3 charts.

I know two ways to do so (there may be more, these are the ones I know). If you are a Python enthusiast like myself, you can do so using Python, provided you have it installed on your machine, that is. At the end of this chapter, you may find a guide on how to install Python if that is your choice.

Assuming you are in the correct folder (you should have navigated to it in the previous step), you may start the server by typing the following command into the console (whether it is the PowerShell or the Mac Terminal).

python -m http.server 8000

where 8000 is the port, you wish to open.

You may try it by opening a Web browser and inputting the following URL:

localhost:8000

or whatever port you choose.

One important note here. This command is only valid in Python 3. For Python 2, the command is different, but right now, you should not be using Python 2, especially if you are going to have to install it fresh.

The other way I know is using node.js. In all fairness, you will have to go through the same trouble of installing the node, similarly as you would do with Python, but since the node is JavaScript, I believe the target audience of this book may prefer this option. Even being a backend language, it may be something you use in the future. Again, you may follow the installation tutorial at the end of the chapter if you do not have it already installed and want to do it yourself. In this case, you will also need to have installed a module called http-server (this is also described in the tutorial at the end of the chapter).

To start the server in node, input the following command into the PowerShell or Terminal:

http-server path/to/your/project/folder

You may then check the server in localhost:8080 (read the console, the port may be different).

To stop the server in Mac and Windows, press control + C.

Creating our index.html

It is now time to create our index file, and, for the first time, we are going to resort to our IDE. I am going to use Visual Studio Code for mine, but any IDE should work. Some popular choices include Sublime Text, Webstorm (this one is not free), or Brackets. If you love traditions, you can also use Vim or Nano, but I think modern editors are better equipped for Web development.

Create a new file and name it index.html. If you are using VS Code (like me), you may go to the side menu, open explorer (top icon), and hit the new file icon. You can also use Cmd + N on Mac and Ctrl + N on PC. If you are a terminal fan, you can input touch index.html in Mac, or

New-Item -Path . -Name index.html -ItemType file

In Windows PowerShell, either way, you will have to open it, so this might be a good moment to start using the editor and leave the console for installations.

Copying an existing chart

For our first example, we are going to copy an existing chart. If you are worried about copying someone else’s work, simply do not. We are infringing no copyright laws, and we are doing something all developers do. This is showcase code we are copying, and we are not selling it; we are learning. Plus, for this example, it does not matter where we go.

Make the following google search: site: bl.ocks.org horizontal bar chart

It does not matter which result you go for, and they will all be similar. Here is where you will start noticing why I mentioned bl.ocks.org earlier. You may see the example of the chart and all the code and data used to generate it.

Depending on the example you land on, there are two ways in which the code may be reading the data:

It may be on the code itself, in a script tag.

It may be on a different section, which means the code needs to be on a separate file.

We will see both cases now:

Case 1: Data is in the same file

If the data shows in the same file, it must be under a variable. This is the simplest case, as we can just parse it as an object and use Object.values(data) to convert it to an array, which, in turn, will allow us to use all the useful array methods that JavaScript has.

Case 2: Data is in a different file

Even though there may be a lot of times in bl.ocks.org in which you find data in the same file, it is very rare that you find this in real life. When you develop, you are very likely going to obtain your data either from a separate file or from a service provider. We will talk a lot about this in detail in Chapter 3: Treating Data, but for the moment, I want to call your attention to the function that is being used to call the data file. Depending on whether you are using d3.v4 or d3.v5, you will see either a Promise (version 5) or a d3 function (version 4). You will still find a lot of examples of v3, but for the topic, v3 and v4 use the same system.

I have created a GitHub repository with my own examples so that you can follow along without copying someone else’s work, although it is still something I encourage you to do, as it is the best way to learn.

Go to https://github.com/vik000/d3/tree/master/01/sameFile to see an example of a bar chart in version 5 where the data is on the same file and to https://github.com/vik000/d3/tree/master/01/separateFile to see an example of the data on a different file (also in version 5, which will be the official version of the whole book).

Visualizing our first chart

Now copy the index file of the first example (in the same file) and copy it into the index.html that you created earlier, then save it. Make sure your server is running (or run it now if you did not do it then, as explained in the second point of this chapter), and now go to the web browser to see the result (remember, localhost:yourchosenport).

Congratulations, you should now be seeing your first chart!

More precisely, you should be seeing something like this:

Figure 1.1: My first chart

Code revision

It is now time to understand what is going on in the file.

en>

UTF-8>

viewport content=width=device-width, initial-scale=1.0>

No magic in this section. You may recognize the html5 usual configuration. In this case, we do not even have CSS files.

In the body, we may see that the first script tag is as follows:

The latest version of d3 is 7, but throughout the book, we will be using version 5, and we will be referencing versions 4 and 6. This is because there are many more resources available for older versions, and most developers are not using version 7 yet. You may find this on the very first page of the d3 website: https://d3js.org/

You may next find div tag:

container width=800 height=300>

This is just a container where our chart is going to lie. I have set a width=800 and a height=300. This is arbitrary, but the size needs to fit our chart’s purpose.

Our next item is the data variable:

var data = {

0 : {

country: Brazil,

titles: 5

},

1: {

country : Germany,

titles : 4

},

2: {

country : Italy,

titles : 4

},

3: {

country : Argentina,

titles : 2

},

4: {

country : Uruguay,

titles : 2

},

5: {

country : France,

titles : 2

},

6: {

country : England,

titles : 1

},

7: {

country : Spain,

titles : 1

}

};

If you like football, you will quickly recognize that this data corresponds to the list of World Cup winners.

This variable is set as an object, which is not the most comfortable format to work with. That is why in the next line, we have transformed it into an array:

data = Object.values(data);

Up to this point, we have seen no d3, but you may remember that d3 is not a framework, but a library, so do not worry; we are getting there.

var containerWidth = +d3.select(‘#container’).attr(‘width’);

var containerHeight = +d3.select(‘#container’).attr(‘height’);

In these two lines, we are using our first d3 method. This allows us to select items in the DOM by either id (#) or class(.). Remember I said it was similar to jQuery?

We are storing the container div’s width and height in variables. This is important because we are then adjusting the chart to the container size, and that means that if we change that, the chart will adapt. This does not mean the chart’s responsive. It is not. It will adapt to the container size, but if you have a dynamic container, the chart will not follow unless you refresh the site or run the script again. In a future chapter, I will show you a trick to create a responsive substitute for charts. It is not perfect, but it will work in most situations.

var margin = {top:20, bottom:20, left:60, right:10};

var width = containerWidth - margin.left - margin.right;

var height = containerHeight - margin.top - margin.bottom;

Every time you work with d3, you will see these three lines (or an adaptation of them). First, we are setting margins for our chart. These are generally leaving a bit of space for the axes. We will see that in a minute.

var x = d3.scaleLinear().range([0, width]);

var y = d3.scaleBand().range([height, 0]).padding(0.1);

The next two lines are one of the most important in a d3 file. They represent scale functions. They are so important that we will dedicate a whole chapter to them. Their job is to correlate real dimensions to those in the browser window.

var svg = d3.select(#container).append(‘svg’).attr(‘id’, ‘svg’).attr(‘width’, containerWidth).attr(‘height’, containerWidth);

This variable selects the container and appends a svg tag to it. It also adds a few attributes to the tag, such as the width and the height. You may notice that we are using the variables we created earlier: containerWidth and containerHeight, which came from the container div.

This would be a very good moment to explain how d3 works but bear with me, and we will see all of this in the upcoming chapter.

x.domain([0, d3.max(data, d=>{return d.titles})]);

y.domain(data.map(d=>{return d.country}));

These two lines are part of the scale definition process, but this step cannot be taken before the data is fully parsed. This will be very common in most, if not all, of the charts you make.

var bars = svg.append(‘g’).attr(‘id’, ‘barGroup’).attr(‘class’, ‘bars group’).attr(‘transform’, `translate(${margin.left}, ${0})`);

We finally arrive at the bar drawing. You may see we select the svg (assigned to the variable we named svg earlier), append a g tag, which is an svg group tag, and assign several attributes to it, the same way we did earlier with the svg. You may start seeing a pattern in the way d3 works.

bars.selectAll(‘rect’).data(data).enter()

.append(‘rect’).attr(‘class’, ‘bar’)

.attr(‘x’, x(0))

.attr(y, function(d) { return y(d.country); })

.attr(width, function(d) { return x(d.titles); })

.attr(height, y.bandwidth() )

.attr(fill, #69b3a2)

This is going to be the hardest part to understand.

We get the bars variable we created in the previous step, and we use the selectAll method. This is a d3 method that returns all found tags instead of just one (which is what select does). Then, we use the data method (also a d3-specific method), and we pass the data argument (in array format). Finally, we call the enter function another d3-specific method.

What we have done up to now is select all bars and assign them the data through the enter function. I know we have no bars yet; how are we assigning anything? Although it is counter-intuitive, you must think that D3 is data-driven and that we are going to generate our bars based on the existing data, so it would make even less sense that we even had bars to select without having the data. The enter function is doing the