The History of Visual Magic in Computers: How Beautiful Images are Made in CAD, 3D, VR and AR

By Jon Peddie

If you have ever looked at a fantastic adventure or science fiction movie, or an amazingly complex and rich computer game, or a TV commercial where cars or gas pumps or biscuits behaved liked people and wondered, “How do they do that?”,  then you’ve experienced the magic of 3D worlds generated by a computer.

3D in computers began as a way to represent automotive designs and illustrate the construction of molecules. 3D graphics use evolved to visualizations of simulated data and artistic representations of imaginary worlds.

In order to overcome the processing limitations of the computer, graphics had to exploit the characteristics of the eye and brain, and develop visual tricks to simulate realism. The goal is to create graphics images that will overcome the visual cues that cause disbelief and tell the viewer this is not real.

Thousands of people over thousands of years have developed the building blocks and made the discoveries in mathematics and science to make such 3D magic possible, and The History of Visual Magic in Computers is dedicated to all of them and tells a little of their story.

It traces the earliest understanding of 3D and then foundational mathematics to explain and construct 3D; from mechanical computers up to today’s tablets. Several of the amazing computer graphics algorithms and tricks came of periods where eruptions of new ideas and techniques seem to occur all at once. Applications emerged as the fundamentals of how to draw lines and create realistic images were better understood, leading to hardware 3D controllers that drive the display all the way to stereovision and virtual reality.

• Language: English
• Publisher: Springer
• Release date: Jun 13, 2013
• ISBN: 9781447149323

Book preview

Jon PeddieThe History of Visual Magic in Computers2013How Beautiful Images are Made in CAD, 3D, VR and AR10.1007/978-1-4471-4932-3_1© Springer-Verlag London 2013

1. Introduction

Jon Peddie¹ 

(1)

Jon Peddie Research, Tiburon, CA, USA

Abstract

Figure 1.1 shows the basic construction of a 3D graphics computer. That is also the general organization of this book, with each block more or less representing a chapter (there is no chapter on memory, but memory is discussed in multiple chapters).

The book traces the earliest understanding of 3D and then the foundational mathematics to explain and construct 3D. From there we follow the history of the computer, beginning with mechanical computers, and ending up with tablets. Next, we present the amazing computer graphics (CG) algorithms and tricks, and it’s difficult to tell the story because there were a couple of periods where eruptions of new ideas and techniques seem to occur all at once. With the fundamentals of how to draw lines and create realistic images better understood, the applications that exploited those foundations. The applications of course can’t do the work by themselves and so the following chapter is on the 3D controllers that drive the display. The chapter that logically follows that is on the development of the displays, and a chapter follows that on stereovision.

1.1 Introduction

It was difficult to write this book because it is impossible to trace a single line of development to generating beautiful realistic 3D images with a computer. There is the computer itself to talk about, the basic math needed, the display and its controller, the software algorithms used to generate curves and textures, and the applications that tie it all together. In addition, 3D is in so many places, automotive design, movies, architecture, games, molecular design, and simulation/visualization of imaginary worlds, and atomic bombs. Furthermore, in tracing these wondrous and exciting discoveries, I had to delve into seemingly obscure items and events because although they were not directly representations of the use of 3D they laid the foundation for the development or exploitation of 3D (Fig. 1.1).

A303032_1_En_1_Fig1_HTML.gif

Fig. 1.1

Basic block diagram of a 3D graphics computer

Consumers, started to become aware of the simulation of worlds and fantasy realms initially, with the introduction of movies like TRON in 1982. Consumers is the metaphor for the common person who is not a scientist or engineer or working in the technical side of the computer industry. Computer graphics (CG) and 3D in computers had actually been around since the early 1960s, but hidden away in government, automotive, aerospace, and university labs.

Between those two mileposts clever computer programs, known as algorithms, were developed. Those algorithms implemented mathematical functions that made it possible to simulate the real and imagined world within the restrictions of the computers of the day. It’s a credit to the genius of the developers that most of those same algorithms are still in use today or are the foundation for even more sophisticated programs.

In this chapter, I will introduce you to some of the foundations and terms of 3D and CG. You won’t be able to get a high-paying exciting job in the CG industry by reading this book, but you’ll probably be a lot smarter and aware of how things work than your friends and family (assuming they are not in the CG business).

If you have ever looked at a fantastic adventure or science fiction movie, or an amazingly complex and rich computer game, or a TV commercial where cars or gas pumps or biscuits behaved liked people and wondered, How do they do that? The answer is with 3D on a computer—it’s magic (Fig. 1.2).

A303032_1_En_1_Fig2_HTML.jpg

Fig. 1.2

A computer generated architectural rendering of an interior (© Koncept Info Technologies)

How about an airplane, car, bridge, skyscraper, or mobile phone; how are they deigned? You know a computer is used, but that’s like saying its magic, it doesn’t tell you anything.

The beautiful and amazing images you see in the movies and video games, the soaring skyscrapers, dazzling automobiles, and the fantastic airplanes that you’ve seen and admire are all products of 3D models and images developed on computers. The 3D content produced on computers is all around us, and in fact, our lives often depend on the data communicated in 3D. In addition to the glorious products described above and seen by most consumers, 3D on computers is also responsible for advances in medicine, astronomy, weather prediction, chemistry and pharmaceuticals, and surgical techniques to name but a few of the invasive aspects of the use of 3D capable computers and programs.

Writing a book about 3D in computers is almost as insane as writing a book about electricity—where do you stop? Electricity is ubiquitous, involved, and integrated with our lives. Likewise, 3D in computers has reached into so many aspects and activities that it is almost impossible to know where to stop.

Here is a brief (although it may not seem so brief) list of some of the places 3D is used in computers

Computer-aided design—CAD (Architecture, mechanical, etc.).

Games (PCs, Consoles, handhelds)

Geoscience

Molecular modeling

Movies and TV

Visualization and simulation

Virtual reality

The goal of this book is to tell a story about how we have gotten to where we are. In the process of the telling, I have tried to recognize the many contributors to the field and at the same time acknowledge the difficulty in doing so for such a book would be a library of books by all those people and their archivists.

The whole idea of 3D on a computer is to create a realistic simulation of the real world. Such simulations are for entertainment as in the case of movies and games or for scientific investigation as in the case of CAD, geophysical exploration, or dangerous and expensive things like nuclear bombs and space launches.

One thing that will come of reading this book, even if you don’t read every word, is that you will look at movies, TV, video games, advertisements, and photographs with a new understanding of what’s behind the images, and with any luck, you will develop a new appreciation for the creation of those images. You might even be inspired to try it yourself, and I very much encourage that. You can use many low cost programs today.

1.2 Geometry and Art

3D in computer graphics has two primary elements: geometry and art. There are subsets of these broad categories, and I’ll try to cover them, but basically in 3D on a computer you are trying to generate a model, a 3D model, of something (not necessarily in the real world), and once you’ve got the geometry figured out, then you want to make it look interesting or realistic—the art of it.

One of the main themes of this book was the introduction of something new to the field that moved the science or art further or overcame a blockade. Therefore, evolutionary steps, like memory going from a 55 nm process to 22 nm, or a GPU increasing its processors from 300 to 1,800 are not considered. This will annoy some who feel those steps are significant and who have devoted a portion of their lives to making it happen, but this is not intended to be a book on the history of technology or computers, or the world, just 3D on computers and major steps in that development.

Almost every product we see today, movie, TV show, computer game uses 3D, 3D on a computer.

3D is the measurement of the primary three directions in space, up, sideways and in or out or Y, X, and Z, or F1, F2, and F3 as depicted in Fig. 1.3.

A303032_1_En_1_Fig3_HTML.gif

Fig. 1.3

Basic representation of 3D space with 3-Axis

3D is a mathematical concept used in the physical world to create things in the physical world. That is, you take the math of 3D to describe something, say this book, and then you build it using the mathematical data. When someone says an object is x-inches wide, and y-inches tall, and z-inches deep they have described a 3Dimensional object—they have used 3D (Fig. 1.4).

A303032_1_En_1_Fig4_HTML.gif

Fig. 1.4

Three axis or dimensions to describe the size of a book

A303032_1_En_1_Fig5_HTML.jpg

Fig. 1.5

Car model courtesy of Nvidia (Advanced Rendering Center), rendered with mental ray®

The math is simple when describing a cubic thing like a book; it gets a lot more complicated when describing curved surfaces like the fenders of your car (Fig. 1.5).

And it gets trickier still when describing something that moves and changes shape while it changes like water, smoke, hair, or science fiction shape-shifters and transformers (Fig. 1.6).

A303032_1_En_1_Fig6_HTML.jpg

Fig. 1.6

Smoke and hair—all simulations created in a computer (© Jos Stam and Henrik Jensen & Andrew Selle and Michael Lentine, respectively)

However, in our incredibly sophisticated lives we take these presentations, products, and entertainment vehicles for granted, this magic is part of everyday life.

Someone once said the best way to destroy a miracle is to explain it, but when it comes to 3D, the miracle just gets better. The History of Visual Magic in Computers represents a fascinating journey by people driven to bring abstract ideas to life—to see what they think.

1.3 The History of Visual Magic in Computers

Tracing the historical development of 3D in computers is difficult because like any historical development there is no straight path. Rather the development consists of rivulets, and stops and starts, and evolutions and a lot of parallel work in unrelated fields. Therefore, throughout this book you will find various attempts at showing these developments, and sometime a little redundancy due to the overlapping and interlocking aspects of the work, discoveries, and implementations.

Figure 1.7 shows an overview of the development of 3D in computers, and the non-linear, cross-referenced depiction of that diagram serves as a perfect metaphor of the problem and the richness of the story.

A303032_1_En_1_Fig7_HTML.gif

Fig. 1.7

The History of Visual Magic in Computers traces a complex and exciting path

A303032_1_En_1_Fig8_HTML.gif

Fig. 1.8

Penetration of 3D into computing platforms

Today we live in a world immersed in 3D from representations on our TV to perspective views of roads on in-dash car GPS displays, to game consoles that are as good as the movies and all the way to faithful reproductions of all the body’s parts and functions.

The penetration of 3D into computers began in the early 1980s and by 2008; it was 100 % on every major platform with a programmable processor as illustrated in Fig. 1.8.

Computer graphics, including 3D, started with vector animation in the 1960s. To create animations of 3D images people photographed sequential drawings made on pen-plotters, one frame at a time, in 1969. It was very tedious and time consuming, and full of mistakes and errors causing lots of re-work—it was hell, and at the same time exciting as hell to be the first ones making such things. I remember vividly watching a giant 6 by 12, flatbed plotter, draw the outline of a car and was amazed, even though I helped design the thing. Pictures of it at the time were super-secret, so I don’t have an example to show you I’m sorry to say.

1.4 Looking Forward

The following chapters will trace the development of the computers, the software, (and the math behind it) that runs on them, and the most important components in the computer used to generate the images and effects.

One of my favorite axioms (some call it Peddie’s law) is, in computer graphics too much is not enough. Another take-away I hope you’ll get from this book is that in CG what we’re trying to create is perfect replication of the, or a, world. Sometimes it done for maximum accuracy as in the case of designing bridges and nuclear weapons, and sometimes for fantasy as in the movies and game. In that pursuit goal, there isn’t enough computing horsepower today—and that statement is true regardless of when you read this book.

1.5 Computer Graphics References and Links to Resources

If this book, which I hope you’ll find interesting, stimulates your curiosity and interest in computer graphics I’ve listed a few places you can go for additional information. In addition, in several chapters there are other references for further reading and investigation.

1.5.1 May the Pixel Be with You

Nan’s Computer Graphics Page http://www.cs.rit.edu/~ncs/graphics.html

Index of Course Sections – History of Computer Graphics https://design.osu.edu/carlson/history/lessons.html

An Historical Timeline of Computer Graphics and Animation http://sophia.javeriana.edu.co/~ochavarr/computer_graphics_history/historia/

Computer Graphics – Fall 2011 http://www.cs.ucsb.edu/~mturk/cs180/links.html

Graphics | VizWorld.com http://www.vizworld.com/graphics/

Ken Perlin’s homepage http://mrl.nyu.edu/~perlin/

Computer software and computers – the best resources are on CompInfo http://www.compinfo.co.uk/#cad

THE resources for computer software and computers for corporate users with extensive links to technologies, manufacturers, support, news, magazines, events, books etc.

Computer Graphics – Guide to the Literature http://depts.washington.edu/englib/eld/fulltext/ComputerGraphics.pdf

3D Computer Graphics :: General Information http://mkrus.free.fr/CG/info.html

CS5611 Advanced Computer Graphics Course Information Page http://www.csl.mtu.edu/cs5611/www/Home.html

Resources Overview – Computer Science COMP-175: Computer Graphics – Research Guides at Tufts University http://researchguides.library.tufts.edu/computer_science_175

Research Guides. Computer Science COMP-175: Computer Graphics. Resources Overview.

ACM SIGGRAPH Web Resources: Learning on the Web http://www.siggraph.org/cgresources/displaycategory.php3?category_id=1

Computer Graphics Tutorials Are Currently The Most Favored Source of Information Regarding The Study http://www.sooperarticles.com/technology-articles/computer-graphics-tutorials-currently-most-favored-source-information-regarding-study-800663.html

In today’s world when computers play such an important role in our day to day life, how can we forget the role of computer graphics played in computers and other related items. In the current scenario everything except text and sound every term can be broadly included in computer graphics.

IGI Global: International Journal of Creative Interfaces and Computer Graphics (IJCICG) (1947–3117) (1947–3125): Ben Falchuk, Adérito Fernandes Marcos: Journals http://www.igi-global.com/journal/international-journal-creative-interfaces-computer/1161

International Journal of Creative Interfaces and Computer Graphics (IJCICG) (1947–3117) (1947–3125): Ben Falchuk, Adérito Fernandes Marcos: Journals

Program of Computer Graphics: Resources http://www.graphics.cornell.edu/resources.html

Computer Graphics Information on the Web http://www.graphics.cornell.edu/online/links.html

Ron Fedkiw – Stanford University http://physbam.stanford.edu/~fedkiw/

Vladlen Koltun – Stanford University http://vladlen.org/

See also the Wikipedia pages for –

Computer graphics

Visualization

Information Visualization

Jon PeddieThe History of Visual Magic in Computers2013How Beautiful Images are Made in CAD, 3D, VR and AR10.1007/978-1-4471-4932-3_2© Springer-Verlag London 2013

2. Getting to 3D

Jon Peddie¹ 

(1)

Jon Peddie Research, Tiburon, CA, USA

Abstract

Getting to 3D has required several steps, which were not a logical process done with an end goal in mind. Rather it was the natural outcome of one discovery leading to another, but driven by intellectual curiosity, genius, and occasionally chance discovery. When tracing the history of anything there has to be time and basic distance measurement. From that, we trace the development of basic geometry, and find that the triangle is foundation of all computer graphics. Even before triangles, we had to be able to count, and the first numbering systems date back to 5000 BCE. A system of numbers requires rules, rules that will support predictability and repeatability. In India, Panini established the Sanskrit grammar, and the grammar known as Ashtadhyayi, which was beginning of linguistics. That was necessary so we could share our counting and designs with other people. Thales of Miletos brought the science of geometry from Egypt to Greece, three centuries before Euclid. Pythagoras known for the Pythagorean Theorem used those concepts. After Thales introduced deductive reasoning in the 300s BCE, Euclid organized the teachings of Pythagoras into his own great work, The Elements. Then we had to learn how to use zero, and from there negative numbers, on to matrix math and transformations. It took close to 6,000 years to get to the point where we understood 2D geometry. The next step was to extend it to 3D. That wasn’t as easy as it sounds and Heron of Alexandria, mastered it in Egypt in the first century.

2.1 Introduction

In this chapter I’ll show how 3D graphics has been able to be realized though the pioneering discoveries and efforts of the Egyptians, the Renaissance artists, heretics, and geniuses. One of the best books on the subject is The Psychology of Graphic Images, by Manfredo Massironi [1]. The book explores the nature of one of the most ancient tools for nonverbal communication: drawings. They are naturally adaptable enough to meet an incredibly wide range of communication needs, and how exactly do they do their job so well? We’ll explore some of those topics, with the view to 3D.

In this chapter, we will look at the foundation mathematics that makes 3D possible and the men who discovered those principles (and no need to worry, there won't be a test); the block diagram shows where the mathematics fits in the scheme of things (Fig. 2.1).

A303032_1_En_2_Fig1_HTML.gif

Fig. 2.1

The math is the foundation of all 3D graphics

2.2 The Foundation of 3D

Computers use 3D to mimic the real world. We make constructions basic on geometry and math to create believable and recognizable forms of nature, fantasy, and reality.

The face in Fig. 2.2 (Mr. 3D guy) is computer generated. Created in 2005 by Takayoshi Sato as part of a master class in computer graphics, in Tokyo, it is good, but not good enough to fool you into thinking it was a photograph. Part of that has to do with a phenomenon known as the Uncanny Valley, explored in the chapter on Developing the Application.

A303032_1_En_2_Fig2_HTML.jpg

Fig. 2.2

Mr. 3D guy, a computer-generated image of a face (Courtesy of Takayoshi Sato and Sheen Hara)

It is possible today to make such perfect pictures, but the point of this picture is to show you how one is constructed; it’s all done with triangles (Fig. 2.3).

A303032_1_En_2_Fig3_HTML.gif

Fig. 2.3

Triangle mesh for computer-generated image of a face; a head of less than 2,500 triangles driven by 36 bones (Courtesy of Takayoshi Sato and Sheen Hara)

A303032_1_En_2_Fig4_HTML.gif

Fig. 2.4

Georges Seurat – The Side Show (1888) – detail showing pointillism technique (Copyright free, image is in the public domain)

Computer graphics and 3D is fundamentally built on the geometry of triangles, which in turn are built on (or of) lines, and they are built from points. However, it took a long time, over 5,000 years for humanity to get to the point where we could understand, exchange information about, and make use of geometry. Therefore, to appreciate the development of 3D in a computer I think it is necessary to appreciate the foundations of its development. (As a side note, one of the discussions today is that computer graphics is fundamentally a coloring problem: what color should each dot be on a screen?) Determining each color of a pixel however, is dramatically different from Georges Seurat (1859–1891) and pointillism [2] (Fig. 2.4).

Linear algebra and projective geometry are involved in CG and 3D. 3D also includes vectors as well as surfaces. Fundamentally, we’re dealing with points, lines, and polygons for digital computer graphics. I’ll try to bring this all together throughout this book.

It took centuries for man to understand his place on the earth, to be able to measure it and understand it, all of it embodied in what we call today math or Geometry. The history of the discovery of place and measurement is an interesting topic on itself, and a book I’d like to write 1 day. But for the purposes of understanding how one does or gets 3D on a computer I have limited the historical review of the math to a brief discussion of the following events.

2.3 The Calendar

When tracing the history of anything you have to refer to a calendar. The calendar chosen for this book is the Gregorian, The numbering of years per the Gregorian calendar is currently dominant in many places around the world, in both commercial and scientific use. For decades, it has been the worldwide standard, recognized by international institutions such as the United Nations and the Universal Postal Union. This is due to the great influence of the Western world on science, technology and commerce, as well as the fact that the solar Gregorian calendar has, for a long time, been considered to be astronomically correct. There is no religious basis or bias for the use of this calendar system in this book.

2.3.1 Notation

The notation used in the book for time will be CE and BCE. It refers to Common Era and is used in place of A.D. BCE means Before Common Era. In the absence of CE after a date it may assumed to mean CE.

2.4 The First 3D – ∼5000–3000 BCE

In computer graphics, we have evolved to the stage where the elementary element we can use for representation of a geometric component is a triangle. With that in mind, the pyramids present themselves as a fine place to begin the discussion of the math of 3D being the most predominant example of the oldest implementation of 3D (Fig. 2.5).

A303032_1_En_2_Fig5_HTML.jpg

Fig. 2.5

Pyramids represented the first successful implementations of 3D mathematics (© Historylink101.com)

The pyramids have inspired others in the field of 3D as well as archaeology and you will find a reference to a live version of it in the Epilogue (page 26).

2.5 Learning to Count (2500–500 BCE)

Even before triangles, we had to be able to count. One might think people would have a very good number sense, but as it turns out, people do not. Experiments have shown that the average person has a number sense that is around four.

People groups in the world today that have not developed finger counting have a hard time discerning the quantity four. They tend to use the quantities one, two, and many-which would include four. The number sense is something many creatures in this world have as well as well as we do; even crows seem to be able to count up to four. Although, as suggested, our human ability is not much better than the common crow. We are born with the number sense, but we have to learn how to count.

Ancient Egyptians since at least 4000 BCE had an understanding of fractions, however they did not write simple fractions as 3/5 or 4/9 because of restrictions in notation. The Egyptian scribe wrote fractions with the numerator of 1. They used the hieroglyph A303032_1_En_2_Figa_HTML.gif an open mouth above the number to indicate its reciprocal. The number 5, A303032_1_En_2_Figb_HTML.gif written as a fraction 1/5 would be A303032_1_En_2_Figc_HTML.gif .

The method, found on several papyruses, is why we refer to this technique as Egyptian fractions.

Counting is one of the basic functions of a computer and of course, the foundation for all arithmetic. And although we take it for granted today, it wasn’t as easy as it might seem, it had to be figured out.

2.6 Numbering System (5000–460 BCE)

A system of numbers requires rules, rules that will support predictability and repeatability. And back 6,000 years ago that wasn’t easy or obvious.

Several societies developed counting techniques, but one of the most profound was that done in Mesopotamia by the Babylonians. The Babylonians developed a numbering system about 5,000 years ago. They were famous for their astrological observations and calculations (aided by their invention of the abacus). They used a sexagesimal (base-60) positional numeral system inherited from the Sumerian (5200–4500 BCE) and Akkadian civilizations, which is, and why we have a 360° circle and 60-min hours. This, in turn led to the development of angles, and of course, angles are the foundation of triangles. So this was one of the first developments that helped us get to 3D, but it was a slow and sometimes arduous path with obstacles created by religion, ignorance and fear, and in some case political obstruction.

Just getting a common set of symbols and usage or grammar took a long