Accuracy & Quality Control

Overview#

Accuracy and quality control are not afterthoughts in photogrammetry -- they are integral to every stage of the workflow, from flight planning through final deliverable acceptance. A photogrammetric product without documented accuracy is, from a professional surveying standpoint, of limited value.

Understanding the distinction between accuracy and precision is fundamental. Accuracy describes how close a measured value is to the true value; precision describes the repeatability or consistency of measurements. A photogrammetric dataset can be highly precise (internally consistent) yet inaccurate (systematically offset from true coordinates). Ground control and independent checkpoints exist specifically to detect and quantify both types of error.

"Accuracy refers to the degree of conformity of a measured or calculated value to its actual or true value, while precision refers to the degree of refinement with which a measurement is made or stated." -- Ghilani & Wolf, Elementary Surveying (15th Ed.), Ch. 2, p. 30

In photogrammetric mapping, the ultimate measure of quality is whether the delivered product meets the specified positional accuracy for its intended use. A topographic survey for preliminary design has different accuracy needs than an as-built survey for final payment quantities. Defining accuracy requirements at the outset of a project drives every subsequent decision: GSD, overlap, ground control density, and processing parameters.

---

Error Sources in Photogrammetry#

Photogrammetric measurements are subject to numerous error sources that propagate through the processing chain. Understanding these sources is essential for designing workflows that minimize their impact.

Camera Calibration Errors

The interior orientation parameters -- focal length, principal point offset, and lens distortion coefficients -- define the geometric relationship between the camera sensor and the lens. Errors in these parameters propagate directly into all derived measurements. Self-calibration during bundle adjustment can recover these parameters, but only if the image network has sufficient geometric strength (convergent images, varied flying heights, or oblique views).

"The accuracy of photogrammetric products depends fundamentally on the accuracy of camera calibration parameters. Errors in interior orientation propagate systematically through all derived measurements." -- Wolf, Dewitt & Wilkinson, Elements of Photogrammetry (4th Ed.), Ch. 4, p. 105

Atmospheric Refraction

Light bends as it passes through the atmosphere due to density variations. The effect is most pronounced at oblique angles and greater distances from the camera. For UAS operations at low altitudes (under 120 m AGL), atmospheric refraction is typically a minor error source. For manned aerial photography at higher altitudes, refraction corrections become significant.

Relief Displacement

Objects above the datum are displaced radially outward from the principal point; objects below the datum are displaced inward. The magnitude of relief displacement is:

$d = \frac{r \cdot \Delta h}{H}$

Where $d$ is the displacement on the image, $r$ is the radial distance from the principal point, $\Delta h$ is the elevation difference from the datum, and $H$ is the flying height above the datum. This is the fundamental reason orthomosaics require a surface model for correction -- without it, tall objects lean away from the image center.

---

Ground Control Points#

Ground control points provide the connection between the photogrammetric model's arbitrary internal coordinate system and the real-world coordinate reference system. They are the foundation of positional accuracy.

Purpose

GCPs serve three functions:

1. Georeferencing -- Transforming the model from an arbitrary system to a defined CRS
2. Constraining bundle adjustment -- Reducing systematic errors and improving geometric fidelity
3. Scaling -- Establishing correct real-world dimensions (particularly important if no direct georeferencing is available)

Design Principles

• Distribution: GCPs must be well-distributed across the project area, covering the full extent in both horizontal dimensions and the full range of elevation. Edge and corner placement is critical; interior points refine accuracy within the network.
• Redundancy: A minimum of 5 well-distributed GCPs is required for a constrained adjustment with redundancy. For larger projects, 1 GCP per 5--10 image strips or approximately 1 per 20--40 acres provides a reasonable guideline.
• Accuracy: GCP coordinates must be significantly more accurate than the target product accuracy -- ideally 3--5 times better. For a project targeting 5 cm accuracy, GCPs should be surveyed to 1--1.5 cm accuracy.
• Visibility: GCPs must be clearly identifiable in the aerial imagery. Pre-marked targets (placed before the flight) are strongly preferred over natural features identified after the fact.

"The number, location, and accuracy of ground control points are the primary factors under the photogrammetrist's control that determine the accuracy of the final product." -- ASPRS, Manual of Photogrammetry (6th Ed.), Ch. 6, p. 310

Number and Distribution Guidelines

Monumentation

For repeat surveys or long-term monitoring, GCPs should be permanently monumented (e.g., survey nails, concrete monuments, or iron pins) so they can be reoccupied. For single-use surveys, temporary painted targets or manufactured targets are sufficient.

---

Checkpoints and Independent Verification#

Checkpoints vs. Control Points

A control point (GCP) is used to constrain the photogrammetric adjustment -- it actively influences the solution. A checkpoint is a surveyed point that is withheld from the adjustment and used solely to independently assess the accuracy of the final product.

The distinction is critical: using the same points for both control and accuracy assessment produces an optimistic (and statistically invalid) accuracy statement. This is analogous to testing a regression model against its own training data.

Why Independence Matters

"Accuracy testing must be based on points that are independent of the mapping process. Checkpoints used for accuracy testing shall not be used as control points in the mapping process." -- ASPRS, ASPRS Positional Accuracy Standards for Digital Geospatial Data (2023), Section 7.3

Checkpoints should be:

• Independent: Not used in any way during the photogrammetric processing
• Well-distributed: Covering the project area representatively, not clustered
• Accurately surveyed: To the same or better accuracy standard as GCPs
• Sufficient in number: A minimum of 20 checkpoints is recommended by ASPRS for statistically meaningful accuracy statements; 25--30 for higher confidence

---

ASPRS Positional Accuracy Standards#

The ASPRS Positional Accuracy Standards for Digital Geospatial Data (2023, Edition 2) provide the authoritative framework for specifying and testing the positional accuracy of geospatial products in the United States. These standards replaced the legacy NMAS (National Map Accuracy Standard) and NSSDA approaches with a more rigorous, statistically grounded methodology.

Accuracy Classes

ASPRS defines accuracy classes based on the Root Mean Square Error (RMSE) of checkpoint residuals. The horizontal accuracy class is designated by the RMSE$_{xy}$ value; the vertical accuracy class is designated by the RMSE$_z$ value (for non-vegetated terrain).

The 95% confidence value is computed by multiplying the RMSE by 1.9600 (vertical, assuming normal distribution) or 1.7308 (horizontal radial, assuming circular normal distribution).

---

RMSE Calculations#

The Root Mean Square Error is the standard statistical measure for reporting positional accuracy in geospatial data.

Horizontal RMSE

For $n$ checkpoints, where each checkpoint $i$ has a known coordinate and a corresponding coordinate extracted from the photogrammetric product:

$RMSE_x = \sqrt{\frac{1}{n} \sum_{i=1}^{n} (x_{map,i} - x_{check,i})^2}$

$RMSE_y = \sqrt{\frac{1}{n} \sum_{i=1}^{n} (y_{map,i} - y_{check,i})^2}$

The radial horizontal RMSE combines both components:

$RMSE_r = \sqrt{RMSE_x^2 + RMSE_y^2}$

Vertical RMSE

$RMSE_z = \sqrt{\frac{1}{n} \sum_{i=1}^{n} (z_{map,i} - z_{check,i})^2}$

Worked Example

Suppose a UAS photogrammetric survey is checked against 5 independent checkpoints with the following residuals:

Horizontal RMSE:

$RMSE_x = \sqrt{\frac{0.023^2 + 0.015^2 + 0.008^2 + 0.027^2 + 0.011^2}{5}} = \sqrt{\frac{0.001518}{5}} = \sqrt{0.0003036} = 0.0174 \text{ m}$

$RMSE_y = \sqrt{\frac{0.018^2 + 0.031^2 + 0.012^2 + 0.019^2 + 0.025^2}{5}} = \sqrt{\frac{0.002151}{5}} = \sqrt{0.0004302} = 0.0207 \text{ m}$

$RMSE_r = \sqrt{0.0174^2 + 0.0207^2} = \sqrt{0.0003027 + 0.0004285} = \sqrt{0.0007312} = 0.0270 \text{ m}$

Vertical RMSE:

$RMSE_z = \sqrt{\frac{0.041^2 + 0.028^2 + 0.035^2 + 0.052^2 + 0.019^2}{5}} = \sqrt{\frac{0.006255}{5}} = \sqrt{0.001251} = 0.0354 \text{ m}$

Result: This project achieves approximately 2.7 cm horizontal RMSE$_r$ and 3.5 cm vertical RMSE$_z$, placing it in the ASPRS 5-cm horizontal and 5-cm vertical accuracy classes.

---

Accuracy vs. GSD Relationship#

A practical rule of thumb relates achievable accuracy to the ground sample distance of the source imagery. These relationships assume proper ground control and standard processing.

For example, imagery with a 3 cm GSD can typically achieve 3--6 cm horizontal RMSE and 6--9 cm vertical RMSE under good conditions.

"As a general guideline, horizontal accuracies of 1 to 2 times the GSD and vertical accuracies of 2 to 3 times the GSD can be expected from well-controlled photogrammetric projects processed with current software." -- Ghilani & Wolf, Elementary Surveying (15th Ed.), Ch. 27, p. 818

These are guidelines, not guarantees. Actual accuracy depends on numerous factors including terrain relief, ground control quality, image overlap, surface texture, and processing methodology. Projects with poor texture (water, uniform sand), steep terrain, or insufficient overlap will not achieve these ratios.

---

Quality Assurance Procedures#

Quality assurance in photogrammetry spans the entire project lifecycle. A systematic approach at each stage catches errors early, when they are cheapest to correct.

Pre-Flight Checks

• Verify camera calibration is current and parameters are loaded in processing software
• Confirm flight plan achieves required GSD and overlap for the target accuracy class
• Check GNSS base station setup and verify base coordinates against published control
• Verify GCP targets are placed, visible, and surveyed before flight
• Review airspace authorization, weather conditions, and NOTAMs

In-Flight Monitoring

• Monitor image capture in real time (if platform supports it) for gaps or failures
• Verify GNSS logging and telemetry
• Note environmental conditions (wind speed, cloud cover, sun angle) in the field log
• If conditions change mid-flight (sudden wind gusts, cloud shadows), assess whether to pause or re-fly

Post-Processing QC

• Image review: Check for blur, underexposure, overexposure, and coverage gaps before processing
• Tie point residuals: After photo alignment, review the reprojection error. Typical target: sub-pixel (< 0.5--1.0 pixel RMS)
• GCP residuals: After constrained adjustment, review GCP residuals. Values exceeding 2--3x the expected accuracy indicate problems (misidentified targets, coordinate errors, or poor geometry)
• Checkpoint analysis: Compute RMSE from independent checkpoints. This is the authoritative accuracy measure
• Visual inspection: Overlay the orthomosaic on known features (road centerlines, building footprints) to check for systematic distortion

Deliverable Review

• Verify coordinate reference system and projection are correct
• Check for seamline artifacts, color discontinuities, and ghosting in the orthomosaic
• Validate contour lines against checkpoints and known features
• Confirm point cloud classification (ground vs. non-ground) by visual inspection in cross-section
• Document all QC results in a formal accuracy report accompanying the deliverables

---

Common Pitfalls#

Bowl Effect / Doming

One of the most insidious errors in UAS photogrammetry is the bowl effect (also called doming), a systematic vertical deformation of the surface model. The surface appears to curve upward or downward, particularly at the edges of the project area. This is caused by:

• Insufficient ground control, particularly at the perimeter
• Parallel flight lines without cross-strips or convergent imagery
• Inaccurate camera calibration (particularly the radial distortion coefficients)

Mitigation: Include GCPs at the project perimeter, fly cross-strips, and use oblique images or varied flying heights to strengthen the self-calibration geometry.

Insufficient Ground Control

Using fewer than 5 GCPs, or clustering GCPs in one area of the project, leads to poorly constrained adjustments. The result may look acceptable in the area near the control but diverge significantly in uncontrolled areas.

Poor Overlap

Insufficient overlap (below 70% forward, below 60% side) creates gaps in stereo coverage and weak image matching geometry. This manifests as holes in the point cloud, noisy surfaces, and reduced accuracy. Areas of low texture (pavement, bare soil, water) require even higher overlap.

Over-Reliance on Reported RMSE

The RMSE reported by photogrammetric software on the control points (GCPs) is not an independent accuracy measure -- it is a self-consistency check. A low GCP RMSE means the model fits the control well, but it says nothing about accuracy in areas away from control. Only checkpoint RMSE from independent points constitutes a valid accuracy assessment.

"Accuracy assessment must be performed using checkpoints that are independent of the photogrammetric solution. Residuals at control points used in the adjustment are measures of precision, not accuracy." -- ASPRS, ASPRS Positional Accuracy Standards for Digital Geospatial Data (2023), Section 7.5

---

Key Takeaways#

• Accuracy (closeness to truth) and precision (repeatability) are distinct concepts; photogrammetric QC must address both
• Error sources span the entire workflow: camera calibration, ground control, atmosphere, image matching, and processing parameters
• Ground control points must be well-distributed, accurately surveyed (3--5x better than target accuracy), and clearly visible in imagery
• Checkpoints must be independent of the adjustment -- using GCPs to assess accuracy produces invalid (optimistically biased) results
• ASPRS Positional Accuracy Standards define accuracy classes based on RMSE of checkpoint residuals, with 95% confidence levels
• Horizontal accuracy is typically achievable at 1--2x GSD; vertical accuracy at 2--3x GSD under good conditions
• Quality assurance must be systematic across all project phases: pre-flight, in-flight, post-processing, and deliverable review
• The bowl effect (doming) is a common systematic error caused by weak geometry, poor calibration, or insufficient peripheral ground control
• GCP RMSE reported by software is a precision measure, not an accuracy measure; only independent checkpoint analysis provides true accuracy assessment
• A minimum of 20 checkpoints is recommended by ASPRS for statistically meaningful accuracy reporting

---

References#

1. Ghilani, C. D. & Wolf, P. R. Elementary Surveying: An Introduction to Geomatics (15th Ed.). Pearson, 2018.
2. Ghilani, C. D. & Wolf, P. R. Elementary Surveying: An Introduction to Geomatics (13th Ed.). Pearson, 2012.
3. Wolf, P. R., Dewitt, B. A. & Wilkinson, B. E. Elements of Photogrammetry with Applications in GIS (4th Ed.). McGraw-Hill, 2014.
4. American Society for Photogrammetry and Remote Sensing. Manual of Photogrammetry (6th Ed.). ASPRS, 2013.
5. American Society for Photogrammetry and Remote Sensing. Manual of Photogrammetry (5th Ed.). ASPRS, 2004.
6. ASPRS. ASPRS Positional Accuracy Standards for Digital Geospatial Data (Edition 2, Version 1.0). 2023.
7. Federal Aviation Administration. 14 CFR Part 107 -- Small Unmanned Aircraft Systems. 2016 (as amended).