Photogrammetric Processing

Overview#

Photogrammetric processing is the computational pipeline that transforms overlapping photographs into accurate, georeferenced spatial products -- maps, elevation models, orthophotos, and three-dimensional point clouds. The pipeline begins with raw imagery and ends with deliverables that meet specified positional accuracy standards.

At its core, the process solves a geometric problem: given a set of two-dimensional images captured from different positions in space, reconstruct the three-dimensional coordinates of the objects photographed. This requires knowing (or solving for) two sets of parameters -- the interior orientation of the camera and the exterior orientation of each photograph -- and then applying the mathematical relationship between image coordinates and ground coordinates to compute positions.

"Photogrammetric processing involves determining the position and orientation of each photograph at the time of exposure, and then using overlapping photographs to compute three-dimensional ground coordinates of points appearing in the images." -- Wolf, Dewitt & Wilkinson, Elements of Photogrammetry with Applications in GIS (4th Ed.), Ch. 11, p. 227

The processing workflow has evolved from analog optical-mechanical instruments (stereoplotters) through analytical plotters to today's fully digital softcopy systems. Modern photogrammetric software -- whether classical (SOCET GXP, MATCH-AT, Trimble Inpho) or SfM-based (Pix4D, Agisoft Metashape, DroneDeploy) -- automates much of the pipeline, but understanding the underlying mathematics remains essential for quality control and professional competence.

Interior & Exterior Orientation#

Before any three-dimensional measurement can be made, both the internal and external geometry of each photograph must be established.

Interior Orientation (IO)

Interior orientation recovers the camera's internal geometry at the time of exposure. For a calibrated metric camera, IO parameters are taken from the calibration certificate. For film cameras, IO also involves measuring fiducial mark positions on each photograph to establish the image coordinate system through an affine or projective transformation.

The IO parameters are:

• Calibrated focal length ($c$): The perpendicular distance from the perspective center to the image plane
• Principal point ($x_0$, $y_0$): The foot of the perpendicular from the perspective center to the image plane
• Lens distortion coefficients: Radial ($K_1$, $K_2$, $K_3$) and decentering ($P_1$, $P_2$)

Exterior Orientation (EO)

Exterior orientation defines the position and attitude of each photograph in the ground coordinate system at the instant of exposure. Six parameters are required:

EO can be determined by two methods:

1. Indirect -- Solved computationally through aerotriangulation using ground control points and tie points measured across overlapping images.
2. Direct -- Measured in real-time using onboard GNSS (for position) and IMU (for attitude), a method called direct georeferencing.

"The exterior orientation of a photograph defines the spatial position and angular orientation of the camera at the instant of exposure. These six parameters, together with the interior orientation, completely describe the geometry needed to reconstruct the ground-to-image relationship." -- Ghilani & Wolf, Elementary Surveying: An Introduction to Geomatics (15th Ed.), Ch. 27, p. 810

Most production workflows use a combination: GNSS/IMU provides approximate EO values that serve as initial approximations for aerotriangulation, which then refines them to high accuracy.

Aerotriangulation (AT)#

Aerotriangulation is the process of determining the exterior orientation of every photograph in a block and, simultaneously, the ground coordinates of tie points -- all from a minimal number of surveyed ground control points (GCPs). It is the most critical computational step in the photogrammetric workflow because it establishes the geometric framework for all subsequent products.

Key Concepts

• Ground control points (GCPs): Surveyed points with known ground coordinates ($X$, $Y$, $Z$) that are identifiable in the imagery. They constrain the solution to the real-world coordinate system.
• Tie points: Points that appear in the overlap zone of two or more photographs. They connect photographs together geometrically but have no known ground coordinates -- their coordinates are solved as unknowns.
• Pass points: Tie points located in the overlap between adjacent flight strips, tying strips together laterally.
• Check points: GCPs withheld from the adjustment and used independently to verify accuracy.

Control Requirements

The number and distribution of GCPs needed depends on the block size, the accuracy required, and whether GNSS/IMU data is available. With GNSS-assisted AT, the position of each exposure station is observed by the onboard GNSS receiver, dramatically reducing the number of ground control points needed.

"The use of airborne GPS control in aerotriangulation has dramatically reduced the requirement for ground control points. In many projects, control is needed only at the perimeter of the block." -- ASPRS, Manual of Photogrammetry (6th Ed.), Ch. 11, p. 481

Modern AT is almost entirely automated. Software performs automatic tie point extraction using image matching algorithms (feature detection and correlation) to identify thousands of tie points across the block, replacing the manual point measurement that once consumed days of operator time.

Bundle Block Adjustment#

The bundle block adjustment is the mathematical engine of aerotriangulation. It simultaneously solves for the exterior orientation parameters of every photograph and the ground coordinates of every tie point using the method of least squares.

The Collinearity Equations

The mathematical foundation is the collinearity condition: at the instant of exposure, a ground point, its corresponding image point, and the perspective center of the camera all lie on a straight line. This geometric relationship is expressed as:

$x - x_0 = -c \cdot \frac{m_{11}(X - X_0) + m_{12}(Y - Y_0) + m_{13}(Z - Z_0)}{m_{31}(X - X_0) + m_{32}(Y - Y_0) + m_{33}(Z - Z_0)}$

$y - y_0 = -c \cdot \frac{m_{21}(X - X_0) + m_{22}(Y - Y_0) + m_{23}(Z - Z_0)}{m_{31}(X - X_0) + m_{32}(Y - Y_0) + m_{33}(Z - Z_0)}$

where:

• $(x, y)$ are image coordinates of the point
• $(x_0, y_0)$ is the principal point
• $c$ is the calibrated focal length
• $(X, Y, Z)$ are ground coordinates of the point
• $(X_0, Y_0, Z_0)$ is the exposure station position
• $m_{ij}$ are elements of the rotation matrix $\mathbf{M}$, which is a function of $\omega$, $\phi$, $\kappa$

Each measured image point contributes two equations (one for $x$, one for $y$). A typical block with hundreds of photographs and thousands of tie points yields a system of tens of thousands of equations. The system is nonlinear, so it is linearized using Taylor series expansion and solved iteratively using least squares adjustment.

"The bundle adjustment is a simultaneous least-squares solution for the exterior orientation parameters of all photographs in a block, the ground coordinates of all tie points, and optionally, the interior orientation parameters of the camera." -- Wolf, Dewitt & Wilkinson, Elements of Photogrammetry with Applications in GIS (4th Ed.), Ch. 17, p. 395

Adjustment Outputs

The bundle adjustment produces:

• Refined exterior orientation parameters for every photograph
• Ground coordinates of all tie points
• Statistical quality metrics: residuals, RMS errors, standard deviations of computed coordinates
• Optionally, refined interior orientation parameters (self-calibration)

Quality assessment focuses on the RMS of residuals at ground control points and check points. Large residuals indicate blunders (misidentified points), systematic errors (poor IO), or insufficient control.

Digital Elevation Models#

A fundamental product of photogrammetric processing is the digital elevation model -- a numerical representation of terrain or surface heights.

Terminology

The distinction between DTM and DSM is critical for surveyors. A DTM represents the ground surface and is used for contour generation, grading design, and volumetric calculations. A DSM includes buildings, vegetation, and other above-ground features and is used for line-of-sight analysis, canopy height modeling, and obstruction mapping.

Data Structures

Generation Methods

From photogrammetry, DEMs are generated by measuring elevations at a dense grid of points using stereo image correlation (also called image matching or dense matching). The software identifies conjugate points in overlapping images and computes their three-dimensional coordinates using the collinearity equations. Modern algorithms -- including Semi-Global Matching (SGM) -- produce point densities approaching one point per pixel.

"Digital elevation models derived from photogrammetric measurements provide the height information necessary for orthophoto production, contour generation, volumetric analysis, and three-dimensional visualization." -- Ghilani & Wolf, Elementary Surveying: An Introduction to Geomatics (13th Ed.), Ch. 27, p. 829

From LiDAR, DEMs are generated by classifying the point cloud into ground and non-ground returns and interpolating a surface through the ground points. LiDAR's ability to penetrate vegetation canopy gives it an advantage over photogrammetry for bare-earth DTM generation in forested areas.

Orthophoto Production#

An orthophoto is a geometrically corrected aerial photograph in which every pixel has been repositioned to represent its true planimetric (map) position. Unlike a raw aerial photograph, which contains displacement due to relief and camera tilt, an orthophoto can be used directly as a map -- distances and areas can be measured from it, and it can be overlaid with vector data in a GIS.

The Orthorectification Process

1. Start with the oriented imagery -- photographs whose IO and EO are known from aerotriangulation.
2. Use the DEM -- the elevation model provides the height of the terrain at every point.
3. Perform differential rectification -- for each pixel in the output orthophoto, project a ray from the map position through the DEM surface and back into the source image to determine the correct pixel value. This removes displacement due to both relief and tilt simultaneously.

The displacement of a point in a raw aerial photograph due to terrain relief is:

$d = \frac{r \cdot h}{H}$

where $d$ is the relief displacement, $r$ is the radial distance from the principal point to the image point, $h$ is the height of the point above the datum, and $H$ is the flying height above the datum. Orthorectification eliminates this displacement by accounting for the actual terrain height at every pixel.

"The orthophoto is produced by differential rectification, a process that removes image displacements caused by both camera tilt and terrain relief. The result is an image having the geometric properties of a map combined with the visual qualities of a photograph." -- Wolf, Dewitt & Wilkinson, Elements of Photogrammetry with Applications in GIS (4th Ed.), Ch. 13, p. 287

Mosaicking

Individual orthophotos are combined into a seamless orthophoto mosaic covering the entire project area. Mosaicking involves:

• Seam line placement -- choosing where adjacent orthophotos join, typically along features (roads, field edges) where the transition is least visible
• Color balancing -- adjusting brightness and contrast across images to create a uniform appearance
• Radiometric blending -- smoothing intensity differences along seam lines

The quality of the orthophoto depends directly on the quality of the DEM. Errors in the DEM produce positional errors in the orthophoto. In areas of tall structures or dense vegetation where the DEM is less accurate, orthophoto quality degrades -- a phenomenon visible as "leaning" buildings in urban areas when a DSM rather than a true DTM is used.

Point Cloud Generation#

Dense image matching has made photogrammetric point cloud generation a viable alternative -- and complement -- to LiDAR. By correlating pixels across multiple overlapping images, software can generate three-dimensional point clouds with densities comparable to airborne LiDAR.

Dense Matching Algorithms

Point clouds derived from imagery are surface point clouds (DSM) -- they represent the first visible surface, not the bare ground beneath vegetation. This is a fundamental limitation compared to LiDAR, which can penetrate vegetation canopy and record multiple returns.

Photogrammetric vs. LiDAR Point Clouds

"Image-based point clouds generated by dense matching algorithms can achieve point densities and accuracies competitive with LiDAR in open terrain. However, LiDAR retains a significant advantage in vegetated areas due to its ability to record multiple returns." -- ASPRS, Manual of Photogrammetry (6th Ed.), Ch. 12, p. 555

Feature Extraction & Mapping#

The ultimate purpose of much photogrammetric processing is to produce maps -- planimetric features, contour lines, and topographic detail extracted from the oriented imagery.

Breaklines

Breaklines are linear features that represent abrupt changes in terrain slope -- ridge lines, drainage channels, road edges, retaining walls, and the tops and toes of embankments. They are critical for producing an accurate TIN-based DTM because they force triangle edges to align with real terrain discontinuities rather than interpolating smoothly across them.

Breaklines are typically captured by an operator in a stereo viewing environment, tracing along the feature while the stereo model provides continuous three-dimensional positioning.

Contour Generation

Contours are derived from the DEM or DTM by interpolating lines of equal elevation. Contour quality depends on:

• DEM accuracy -- vertical errors translate directly to contour position errors
• Breakline enforcement -- contours must respect terrain discontinuities
• Smoothing algorithms -- raw contours from gridded DEMs can appear angular; smoothing improves cartographic appearance but must not shift contours beyond accuracy tolerances

Planimetric Compilation

Planimetric features -- buildings, roads, fences, utilities, water features -- are captured from the stereo model or from orthophotos. Stereo compilation provides three-dimensional coordinates, while orthophoto digitizing provides two-dimensional positions at orthophoto resolution.

Modern SfM-MVS Pipeline#

Structure from Motion (SfM) combined with Multi-View Stereo (MVS) has transformed photogrammetric processing, particularly for UAS (drone) applications. SfM automates what classical photogrammetry required extensive manual effort and calibrated cameras to accomplish.

How SfM Differs from Classical AT

The SfM pipeline begins by detecting keypoints (distinctive features) in each image using algorithms such as SIFT (Scale-Invariant Feature Transform). These keypoints are matched across images to establish correspondences. From the matches, the software reconstructs the relative positions and orientations of all cameras (the "structure from motion" step). A bundle adjustment refines the solution, and ground control points are incorporated to georeference the model and control accuracy.

"Structure from Motion refers to the process of recovering three-dimensional structure from a series of two-dimensional images taken from different viewpoints. Unlike classical photogrammetry, SfM does not require a priori knowledge of camera positions or calibration." -- Mikhail, Bethel & McGlone, Introduction to Modern Photogrammetry, Ch. 12, p. 448

After the sparse point cloud and camera orientations are established, MVS algorithms generate a dense point cloud by matching pixels at much higher density than the initial keypoint set. The dense cloud is then used for DEM generation, orthophoto production, and three-dimensional modeling -- the same deliverables as classical photogrammetry, produced through a more automated (but mathematically equivalent) pipeline.

Limitations of SfM

SfM's automation comes with trade-offs:

• Self-calibration instability -- without a pre-calibrated camera, IO parameters can absorb systematic errors, producing a solution that "looks right" but contains hidden deformations (the so-called "bowling effect" in corridor projects)
• GCP dependence for accuracy -- without well-distributed ground control, absolute accuracy is uncontrolled
• Texture dependence -- featureless surfaces (snow, water, uniform pavement) produce poor or no matches
• Processing time -- dense matching of large image sets is computationally intensive

For professional surveying applications, SfM-MVS workflows should always include ground control points, independent check points, and accuracy verification against ASPRS positional accuracy standards.

Key Takeaways#

• Interior orientation recovers the camera's internal geometry; exterior orientation recovers its position and attitude in ground coordinates -- both are prerequisites for all photogrammetric measurement
• Aerotriangulation determines the EO of every photograph in a block from a minimal number of surveyed ground control points, using tie points to connect photographs
• The collinearity equations are the mathematical foundation: each image point yields two equations relating image coordinates to ground coordinates through IO and EO parameters
• Bundle block adjustment solves all unknowns simultaneously by least squares, producing refined EO, tie point coordinates, and statistical quality metrics
• DEM, DTM, and DSM are distinct products: DTM is bare-earth, DSM includes above-ground features, and the choice affects all downstream products
• Orthophotos are produced by differential rectification using the DEM; their positional accuracy depends directly on DEM quality
• Dense image matching produces photogrammetric point clouds competitive with LiDAR in open terrain, but cannot penetrate vegetation canopy
• SfM-MVS automates the classical photogrammetric pipeline and enables non-metric cameras (including UAS platforms) to produce rigorous mapping products, provided ground control and quality checks are applied
• Breaklines are essential for accurate terrain modeling -- contours and surface models that ignore terrain discontinuities will misrepresent the ground surface

References#

1. Wolf, P.R., Dewitt, B.A. & Wilkinson, B.E. Elements of Photogrammetry with Applications in GIS (4th Ed.). McGraw-Hill, 2014.
2. Ghilani, C.D. & Wolf, P.R. Elementary Surveying: An Introduction to Geomatics (15th Ed.). Pearson, 2018.
3. Ghilani, C.D. & Wolf, P.R. Elementary Surveying: An Introduction to Geomatics (13th Ed.). Pearson, 2012.
4. ASPRS. Manual of Photogrammetry (6th Ed.). American Society for Photogrammetry and Remote Sensing, 2013.
5. ASPRS. Manual of Photogrammetry (5th Ed.). American Society for Photogrammetry and Remote Sensing, 2004.
6. Mikhail, E.M., Bethel, J.S. & McGlone, J.C. Introduction to Modern Photogrammetry. John Wiley & Sons, 2001.