Learn Multithreading with Modern C++

By James Raynard

Multithreading is used in many areas of computing, including graphics processing, machine learning and Internet stores. An understanding of threads is essential to make full use of the capabilities of modern hardware.

 

C++ now provides direct support for threads, making it possible to write portable multithreaded programs which have well-defined behaviour, without requiring any external libraries.

 

This book thoroughly covers the basics of threading and will prepare you for more advanced work with threads. Source code is available for all the examples. No previous knowledge of threading is required, but you should be comfortable with programming in C++ at an intermediate level.

 

The book begins with the concepts of multithreading, then shows how to launch threads in C++. We look at the problems which can occur with multiple threads and how to avoid them.

 

C++ provides tools which allow us to work at a higher level of abstraction than system threads which share data; we cover condition variables and promises with futures. We also look at asynchronous, lock-free and parallel programming, including atomic variables and the parallel algorithms in C++17. We end by implementing a concurrent queue and thread pool, which brings together the material you have learnt.

• Language: English
• Publisher: James Raynard
• Release date: Jul 29, 2022
• ISBN: 9798201658069

Book preview

Preface

This book is based on a very successful online course which I created for Udemy: Learn Multithreading with Modern C++. A number of students requested copies of the slides used in the videos.

The source code for the examples is available for download from my github repository.

Chapter One

Concurrency Introduction

What does concurrency mean?

Concurrency

Concurrency generally means two or more things happening at the same time. For example, you could be reading this book while listening to some music.

In computing, concurrency normally means that the activities are conceptually different. Some programs take a long time to run, perhaps because they are fetching data from the Internet, or doing a lengthy calculation. Often, the program will provide some kind of feedback to the user, such as an hourglass or a progress counter. This lets the user know that the program is still running, and also gives some idea of what the program is doing. A program which performs these two activities at the same time is an example of computational concurrency.

At the operating system level, concurrency has been standard in operating systems for many years. For example, when I record videos on my laptop, a program records my words of wisdom. PowerPoint is open and waiting for me to go to the next slide in the lecture. The mail program occasionally checks for new messages, and so on. The computer is able to run several programs at the same time, or at least it appears to.

However, until fairly recently, most computers only had one processing unit. And if you only have one processing unit, how can you run different programs at the same time? The answer lies in something called "time slicing".

Time Slicing

With time slicing, each program gets to run on the processor for a little bit of time. Then it stops, and another program gets to run. Then that program stops to allow other program to run, and so on.

If, for example, I am working on a document while waiting for a compilation to finish, we could have a scenario like this:

- The compiler runs for a short period of time, then pauses

- The mail program runs for a short period of time, then pauses

- The compiler runs again from where it left off, then pauses

- The word processor runs for a short period of time, then pauses

If all this switching between tasks happens quickly enough, it will appear that all these programs are running at the same time.

Cinema films (movies) rely on a similar illusion: they present a sequence of static images so quickly that the human eye interprets them as continuous motion.

Task Switching

A cynic will tell you that there is no such thing as a free lunch, and unfortunately this applies to task switching.

The reason is that when a program is running, the processor needs to have some information about it. The processor contains memory cells, known as "registers; these contain the values of local variables, the program’s current stack position, which instruction the program is currently executing, and so on. When the processor starts running another program, all this context" has to be loaded up for the new task.

Modern processors also have "cache" memory, which stores instructions and data which are being used, in larger quantities than would fit into the registers. If the program has not run recently, this information will not be present in the cache and needs to be loaded up again.

These "context switches" take time, during which the processor cannot execute any instructions for any program. This will slow down the program.

Hardware Concurrency

In today’s world, computers usually have several processors. In fact, even phones now have multiple processors. These are known as "cores or processor cores".

These multiple processor cores can all be working away, doing different things at the same time. They could even be running different parts of the same program at the same time.

These cores all execute independently of each other. They are known as "hardware threads, because each processor core follows its own execution path or thread" through the code. There is one hardware thread per processor core.

Software Concurrency

Modern operating systems allow a program to perform multiple activities at the same time at the software level, using time slicing. These are known as "software threads".

Programmers often refer to software threads as threads and hardware threads as cores.

Computers often have more software threads executing than there are processor cores. This is generally not a problem, because usually threads do not run all the time.

Quite often, a thread has to stop and wait for some event to occur or some result to become available. For example, a thread may be waiting for another thread to finish a calculation, or it may be waiting for the user to press a key. While one thread is waiting, another thread can run and do some useful work.

C++ Support for Concurrency

C++ did not have any standard support for concurrency until C++11. As with many standard C++ features, it only provides the basics for programmers to write their own library; however, what it does have is very efficient. C++20 offers some more sophisticated features, which are not yet fully supported by compilers.

C++’s concurrency implementation is fairly low-level, compared to other languages. The basic building block is the std::thread class; this maps onto a software thread which is managed by the operating system.

std::thread is similar to the thread class in the Boost library, but there are some important differences:

- No thread cancellation

- Different argument semantics

- Different behaviour on thread destruction

Threads and Tasks

The terms thread and task are often used interchangeably. In this book, thread will be used to mean either a software thread, or an object of the C++ std::thread class.

A task is a higher-level abstraction, representing some work that should be performed concurrently. This work could be done by a single thread, or possibly by several threads.

Chapter Two

Concurrency Motivation

Why is concurrency important?

Concurrency Motivation

Concurrency first appeared on mainframes in the 1960’s, but it was not until the 1990’s that the world of workstations and personal computers began to regard it as an important technology. This was due to a number of developments in the world of computing, which were happening around that time.

The Rise of the Internet

Previously, most servers were connected to local networks which served offices or departments; typically, they were only required to handle a few dozen or maybe a few hundred clients at most.

Internet servers can have thousands or even millions of  simultaneous users. Clearly, the amount of work that a server could do at any one time needed to increase dramatically.

Internet servers use threading to increase server throughput.

The Rise of Windows

Users on personal computers could now access several running programs at the same time. They expected to be able to interact with any program at any time, even if the program was busy doing something else.

This meant that Windows programs had to be able to respond to user interaction events while continuing with their work.

Windows applications use threading to provide "separation of concerns".

Fast Numeric Computing

Games and multimedia applications were becoming very popular at this time. These typically involve a lot of mathematical calculations. If the game cannot update its display fast enough, the game will appear jerky, and the audio and visual may suffer from gaps or distortion.

Scientists and engineers were increasingly using computer simulations, which also require rapid mathematical computation.

Games and simulations use threading to provide separation of concerns and to speed up calculations.

The Rise of Multi-core Processors

Computers began to appear which had multiple processors.

Programmers use threading to make effective use of the hardware.

Single-threaded Servers

Before multithreading, a server performed each activity as a separate "process" (a process is a single instance of an executing program)

- The server waits for a client to connect to it

- When a client connects, the server creates a new " child " process to handle the client

- The server waits for the next connection

There is a single server process, and a number of child processes which are dealing with clients. These processes are all separate running applications, each with their own address space. There is no direct communication between these processes.

Usually, the server usually needs to communicate with the child process, perhaps to exchange data with it or to control it in some way. This requires establishing some form of communication channel between the two processes.

Setting up these communication channels has overhead, and there is also a certain amount of overhead when creating a new process. On Windows systems in particular, creating a new process is slow and resource intensive. This overhead will reduce the amount of work that the system can do, which reduces its scalability.

Separation of Concerns

Imagine you are editing a large document, using a single-threaded application, and you start an action which takes a long time to complete. This could be compiling an index for the document, or calculating page numbers, or even - and remember this is the 1990’s! - saving the document to a floppy disk.

While this is happening, the user interface of the program will be completely unresponsive. It will not respond to keystrokes or mouse clicks. If you cover up its window with another application and then uncover it, the program window will have been replaced by a grey box.

The Windows system is sending events to the program, telling it that the user is interacting with it, but the program cannot respond, because it is busy performing the action. All the GUI events are stuck in a queue, waiting for the program to respond to them.

Eventually, the action ends. The program reads all the events in the queue, and processes them instantly. Depending on what you did when you were frantically trying to get the program to respond, all sorts of things could happen. You may find that you have erased your document, or overwritten it with garbage and saved it. This is obviously not desirable!

Separation of concerns improves the user's experience of GUI programs. More generally, it may improve performance, particularly on multi-processor systems. It also follows good coding practice, because it keeps all the code which does one thing in the same place.

In this example, all the GUI event handling code is in a separate thread from the document processing code. This makes the code easier to understand and work with; it also allows different programmers to work on different parts of the code at the same time, without interfering with each other's work.

Fast Numeric Computation

Numeric computation was mainly restricted to people who worked in Universities or Research Institutes and could justify buying a "supercomputer or transputer". These were specialized hardware devices with large numbers of processors. They had a parallel processing architecture which allowed the processors to coordinate with each other.

These computers were programmed using specialized languages, such as Occam or Parallel Fortran. These divided up the program and ran different parts of it on different processors at the same time. The computers, and indeed the languages they used, were extremely expensive.

Nowadays, we can buy a normal computer, or even a phone, which supports parallel programming in hardware, and use a standard language to write parallel code.

Effective Use of Hardware

In the 1990’s, chip designers reached the limit of what they could achieve by making bigger and more complex processors which ran faster and faster. They were starting to be affected by the limitations of the laws of Physics.

Electrons move through silicon at around 5-10% of the speed of light (roughly 4,000 miles or 7,000km in a second.) With a processor cycle time of 1 nanosecond (a frequency of 1 GHz or 1,000 million cycles per second), this translates to a quarter of an inch or half a centimetre per cycle.

Given that modern processor chips can be 3 inches or 5cm across, then it will take several cycles for an electron to go from one side of the processor chip to the other. This limits the speed at which data can move around and instructions can be processed.

The other problem is heat generation. When electrons move through silicon, they generate heat, and this increases as the chip gets bigger and faster. If designers had continued making single chip processors, then chips would now be running at the temperature of molten steel or the surface of the sun. Which is really not something you want to keep on your desk!

Instead, designers started producing "cores", in which there are several processors on the same silicon chip.

A single-threaded program can only run on one core at a time. If you have a modern computer which only runs one program, and it only uses one of the available cores, then you are clearly not making full use of the system’s power.

Concurrency is the Answer!

Concurrency gives our program the ability to do more than one thing at the same time.

For servers, we can have a child thread which runs inside the server process. This avoids having to create a new process every time a client makes a connection. Because the server and the child are