Epipolar Geometry: Unlocking Depth Perception in Computer Vision

By Fouad Sabry

What is Epipolar Geometry

Epipolar geometry is the geometry of stereo vision. When two cameras view a 3D scene from two distinct positions, there are a number of geometric relations between the 3D points and their projections onto the 2D images that lead to constraints between the image points. These relations are derived based on the assumption that the cameras can be approximated by the pinhole camera model.

How you will benefit

(I) Insights, and validations about the following topics:

Chapter 1: Epipolar geometry

Chapter 2: Optical aberration

Chapter 3: Focal length

Chapter 4: Camera lens

Chapter 5: 3D projection

Chapter 6: Vanishing point

Chapter 7: Distortion (optics)

Chapter 8: Parallel projection

Chapter 9: Collinearity

Chapter 10: Fundamental matrix (computer vision)

(II) Answering the public top questions about epipolar geometry.

(III) Real world examples for the usage of epipolar geometry in many fields.

Who this book is for

Professionals, undergraduate and graduate students, enthusiasts, hobbyists, and those who want to go beyond basic knowledge or information for any kind of Epipolar Geometry.

• Language: English
• Publisher: One Billion Knowledgeable
• Release date: May 14, 2024

Book preview

Chapter 1: Epipolar geometry

Epipolar geometry is the geometry behind three-dimensional perception. There are a variety of geometric linkages between the 3D points and their projections onto the 2D images that lead to limitations between the image points when two cameras observe a 3D scene from two different places. These connections are formed from the idea that the cameras can be represented by a pinhole camera.

Two pinhole cameras, both focused on point X, are shown in the diagram below.

Using actual cameras, The plane of focus lies behind the picture plane, It creates an image that is symmetrical with respect to the lens's focal point.

Here, however, By imagining an image plane in front of the camera's focal point (i.e.

lens' optical center to create an image that is not distorted by the mirror image.

OL and OR represent the centers of symmetry of the two cameras lenses.

Both cameras' focal points are indicated by the symbol X.

Points xL and xR are the projections of point X onto the image planes.

The 3D world is captured by each camera as a 2D image. The pinhole camera model perfectly describes this transformation from three dimensions to two, which is known as a perspective projection. It is common practice to represent this projection process using rays that leave the camera and focus at its center. One image point is represented by each ray.

Since the cameras' lenses have different optical centers,, There is a single point in the image plane of the other camera that each focal point projected onto.

These two apexes of the image, denoted by eL and eR, epipoles, often known as epipolar points.

Both epipoles eL and eR in their respective image planes and both optical centers OL and OR lie on a single 3D line.

The line OL–X is seen by the left camera as a point because it is directly in line with that camera's lens optical center.

However, This line is perceived as a line in the image plane of the correct camera.

That line (eR–xR) in the right camera is called an epipolar line.

Symmetrically, the line OR–X is seen by the right camera as a point and is seen as epipolar line eL–xLby the left camera.

The location of point X in three-dimensional space determines the epipolar line, i.e.

as X shifts, Both pictures will have a series of epipolar lines drawn on them.

Since the 3D line OL–X passes through the optical center of the lens OL, the corresponding epipolar line in the right image must pass through the epipole eR (and correspondingly for epipolar lines in the left image).

The epipolar point is the origin of all epipolar lines in a given image.

In fact, Since the epipolar point can be located anywhere in space, any line that passes through it is considered an epipolar line.

As a contrasting image,, think on the evidence X, OL & OR that form a plane called the epipolar plane.

The epipolar lines are the lines formed where the epipolar plane meets the picture plane of each camera.

No matter where X is, every epipolar plane and epipolar line must pass through the epipole.

Two crucial realizations follow from knowing the two cameras' relative positions::

Assume the projection point xL is known, and the epipolar line eR–xR is known and the point X projects into the right image, on a point xR which must lie on this particular epipolar line.

This necessitates that, along a known epipolar line, for every point in one image, the corresponding point must be shown in the other image.

This provides an epipolar constraint: the projection of X on the right camera plane xR must be contained in the eR–xR epipolar line.

Every X, for instance.

X1, X2, X3 on the OL–XL line will verify that constraint.

This allows us to determine if two points are the same 3D point.

The fundamental or essential matrix connecting the two cameras can also characterize epipolar restrictions.

If the points xL and xR are known, their rays of projection are also well-known.

The projection lines must cross at X if the two picture points represent the same 3D point.

Since we know the locations of these two landmarks in the image, we can use them to determine X, the use of triangles, or triangulation.

If the picture planes of the two cameras are parallel to one another, the epipolar geometry is simplified.

Here, though,, the epipolar lines also coincide (eL–XL = eR–XR).

Furthermore, the epipolar lines are parallel to the line OL–OR between the centers of projection, and the horizontal axes of the two pictures can be aligned in practice.

That's to say, for every dot in a single image,, You need only scan horizontally to locate its counterpart in the opposite picture.

If the cameras can't be set up like this, The cameras' image coordinates can be modified so that it appears as though they are all pointing towards the same plane.

Image correction refers to this procedure.

Pushbroom cameras use a collection of one-dimensional CCDs to create a continuous picture strip, or image carpet, rather than the traditional frame camera's two-dimensional CCD. This sensor's epipolar shape is very dissimilar to that of traditional pinhole projectors. To begin, the epipolar line of the pushbroom sensor is curved like a hyperbola instead of being straight. Second, there is no such thing as a pair of epipolar curves.

{End Chapter 1}

Chapter 2: Optical aberration

Aberration is a feature of optical systems, such as lenses, that allows light to be dispersed across a certain region of space rather than being focussed to a single point. This phenomenon is known in the field of optics..

1: Imaging by a lens with chromatic aberration.

In addition, a lens that has a lower chromatic aberration

Image formation optical systems that are subject to aberration will result in the production of images that are not sharp. Optical instrument manufacturers are required to make adjustments to their optical systems in order to compensate for aberration..

The techniques of geometrical optics can be utilized in order to do an analysis of aberration. Some of the general characteristics of reflected and refracted rays are discussed in the articles that are devoted to reflection, refraction, and caustics..

Reflection from a spherical mirror.

Reflected rays (green) that are not directed toward the focal point are produced by incident rays (red) that are directed away from the center of the mirror.,

F.

Because of spherical aberration, this is the case..

An ideal lens would allow light from any point on an object to pass through it and converge at a single point in the picture plane (or, more generally, the image surface). This would be the case if the lens were perfect. Actual lenses, on the other hand, do not precisely concentrate light on a single spot, even when they are constructed to perfection. Aberrations of the lens are the term used to describe these deviations from the idealized performance of the lens..

The two categories of aberrations are known as monochromatic and chromatic aberrations. When light is reflected or refracted, monochromatic aberrations can develop. These aberrations are created by the geometry of the lens or mirror, and they can occur in either of these two processes. The name comes from the fact that they are visible even when employing monochromatic light..

Chromatic aberrations are brought about by dispersion, which is the change in the refractive index of a lens that occurs regardless of the wavelength. It is because of dispersion that distinct wavelengths of light come to focus at different spots in specific locations. The use of monochromatic light does not result in the appearance of chromatic aberration because.

In terms of monochromatic aberrations, the most common ones are:

Defocus

Spherical aberration

Coma

Astigmatism

Field curvature

Image distortion

In spite of the fact that defocus is theoretically the lowest-order of the optical aberrations, it is not typically considered to be a lens aberration. This is because it can be rectified by shifting the lens (or the picture plane) in order to bring the image plane closer to the optical focus of the lens..

These aberrations are not the only factors that can cause the focal point to shift; the piston and tilt effects are further examples of such effects. When an otherwise perfect wavefront is altered by piston and tilt, it will still create a flawless, aberration-free image; the only difference is that it will be displaced to a new position. This is