Calibration Engineering - Ch.06: Camera Calibration [EN]

06.01. Zhang Checkerboard Method

Zhang's checkerboard method is a practical camera calibration technique for estimating intrinsic camera parameters from multiple images of a planar checkerboard observed at different orientations.

Because the target is flat and its square geometry is known, the method avoids the need for a precisely manufactured three-dimensional calibration fixture while still providing accurate geometric constraints for calibration.

The calibration target consists of alternating black and white squares arranged on a rigid planar surface. In computation, the useful reference points are normally the internal checkerboard corners rather than the outer edges of the board. Their physical coordinates can be defined on a plane with Z = 0, using the known square size to establish metric spacing between neighboring points.

Calibration begins by capturing several images while changing the position and orientation of the checkerboard relative to the camera. Images should contain meaningful variations in tilt, rotation, distance, and image location rather than nearly identical front-facing views. This diversity provides the geometric information required to separate camera intrinsic properties from the pose of the calibration target.

For every valid image, the checkerboard corners are detected and their image coordinates are measured with subpixel precision. These two-dimensional observations are associated with corresponding points in the checkerboard coordinate system. Detection quality is important because corner localization errors propagate through the optimization and can affect focal length, principal point, distortion, and pose estimates.

The mathematical foundation of the method is the planar projective transformation, or homography, between the checkerboard plane and the image plane. Since every reference point lies on Z = 0, each calibration image provides a homography relating known planar coordinates to observed pixel coordinates. Multiple homographies obtained from different board poses generate constraints on the camera intrinsic matrix.

The intrinsic matrix describes the internal geometric properties of the camera. Its principal parameters include focal lengths in the horizontal and vertical image directions and the coordinates of the principal point. A skew parameter may also be represented, although it is commonly assumed to be zero for modern cameras. These parameters remain shared across all calibration images acquired with unchanged camera settings.

Each checkerboard image also has its own extrinsic parameters because the target occupies a different pose relative to the camera. Rotation and translation describe the transformation between the checkerboard coordinate frame and the camera coordinate frame. Once an initial intrinsic matrix has been estimated, the homography associated with each image can be decomposed to obtain initial estimates of these pose parameters.

Real lenses deviate from the ideal pinhole camera model, so lens distortion must be incorporated into practical calibration. Radial distortion typically causes straight lines to bend increasingly toward the image boundary, while tangential distortion can arise from imperfect alignment of lens elements. Zhang calibration therefore combines the pinhole projection parameters with distortion coefficients during refinement.

Initial parameter estimates obtained from homography constraints are normally refined through nonlinear optimization. The optimization projects the known checkerboard points into every image using the current intrinsic, distortion, rotation, and translation parameters. It then adjusts these parameters to minimize the difference between predicted image locations and the corner positions actually detected in the calibration images.

Reprojection error is the primary numerical indicator used to evaluate calibration consistency. For each checkerboard corner, the projected location predicted by the calibrated camera model is compared with the measured image location. The resulting residuals can be summarized across individual images and the complete dataset, helping identify poor detections, unsuitable poses, or inconsistent observations.

A low average reprojection error alone does not guarantee a robust calibration. Calibration images should cover a broad portion of the camera field of view so that the model is constrained near image boundaries where lens distortion is often strongest. The checkerboard should also appear at several scales and orientations, because limited pose diversity can produce parameters that fit the collected images but generalize poorly.

Target quality directly influences calibration accuracy. The checkerboard should be flat, dimensionally stable, clearly printed, and large enough to provide reliable corner observations over the intended working distance. Warping of paper targets, incorrect square dimensions, motion blur, defocus, glare, and partial corner ambiguity introduce geometric errors that cannot be fully corrected by mathematical optimization.

Camera configuration must remain fixed throughout the calibration dataset. Changes in focus, zoom, resolution, digital cropping, image scaling, or optical configuration can alter the effective intrinsic parameters and invalidate the assumption that all images share one camera model. For robotics applications, calibration data should therefore be associated with the exact sensor mode used during normal perception operation.

Validation should preferably use observations that were not relied upon solely to obtain the parameter solution. Undistorted images can be inspected for straight-line consistency, while independent checkerboard views can be evaluated through reprojection error. Large localized residuals or systematic error patterns may indicate insufficient distortion modeling, inaccurate corner detection, target deformation, or an inappropriate calibration dataset.

In a robotics calibration workflow, Zhang's method provides the intrinsic geometric foundation required before many higher-level vision operations are performed. Reliable focal length, principal point, and distortion estimates support metric reconstruction, stereo processing, visual localization, object measurement, and camera-to-sensor alignment. Intrinsic calibration should therefore be treated as a controlled and traceable sensor configuration.

The method is especially useful because it combines simple calibration hardware with a mathematically well-constrained estimation procedure. Its practical accuracy depends less on sophisticated equipment than on careful target construction, diverse image acquisition, precise corner extraction, optimization, and independent validation. When these conditions are controlled, checkerboard calibration provides a repeatable baseline for robotic camera systems.

06.02. Lens Distortion Model

Lens distortion describes the systematic deviation between an ideal pinhole camera projection and the image formed by a real optical system. In an ideal model, a three-dimensional point is projected through a single optical center onto the image plane. Real lenses bend rays imperfectly, causing measured pixel positions to shift from their theoretically predicted locations.

Distortion becomes particularly important in robotics because geometric measurements depend on accurate relationships between image coordinates and physical directions. Even relatively small pixel displacement can affect visual localization, stereo reconstruction, object measurement, feature matching, and camera-to-sensor calibration. Wide-angle cameras usually exhibit stronger distortion, especially toward the boundaries of the field of view.

The distortion model is normally applied to normalized image coordinates derived from the pinhole projection. A three-dimensional point expressed in the camera coordinate frame is first converted into normalized coordinates by dividing its horizontal and vertical coordinates by depth. Lens distortion then modifies these normalized coordinates before the intrinsic camera matrix converts them into final pixel coordinates.

Radial distortion is one of the dominant components of lens distortion and is approximately symmetric around the optical axis. Its magnitude generally increases with radial distance from the principal region of the image. The distortion is commonly represented by polynomial coefficients such as k1, k2, and k3, which progressively model higher-order changes as points move farther from the image center.

Two characteristic forms of radial distortion are:

• barrel distortion
• pincushion distortion

Barrel distortion causes image magnification to decrease toward the periphery, making straight lines appear to curve outward relative to the ideal projection. Pincushion distortion produces the opposite tendency, with magnification increasing toward the edges and straight structures appearing to bend inward.

Tangential distortion represents asymmetric image displacement caused primarily by imperfect alignment between optical elements and the image sensor. If lens components are slightly decentered or tilted relative to the sensor plane, image points can shift both horizontally and vertically. Tangential effects are commonly parameterized using coefficients such as p1 and p2 together with the normalized image coordinates.

A conventional camera model therefore combines the ideal pinhole projection with radial and tangential distortion terms. The complete parameter set usually includes focal lengths, principal point coordinates, distortion coefficients, and the extrinsic transformation associated with each calibration observation. These quantities are estimated together so that the model reproduces the image formation behavior of the physical camera.

During checkerboard-based calibration, known target points are observed at many locations across the image. The calibration algorithm predicts where each point should appear according to the current camera and distortion parameters, then compares those predictions with detected corner locations. Nonlinear optimization adjusts the parameters to reduce the accumulated reprojection error across all calibration images.

Image coverage is especially important for reliable distortion estimation. Calibration points concentrated near the center provide relatively weak information about distortion because radial displacement is generally small in that region. Checkerboard observations should therefore extend toward the left, right, top, bottom, and corners of the usable image area while maintaining sufficiently accurate feature detection.

Different lenses may require different distortion model complexity. A standard radial-tangential model is often adequate for conventional perspective cameras with moderate fields of view, while fisheye or extremely wide-angle lenses can require specialized projection models. Adding unnecessary coefficients, however, can increase parameter correlation and overfitting rather than improving physically meaningful calibration accuracy.

Undistortion uses the estimated lens model to map distorted measurements toward their ideal geometric locations. In image processing, this can be implemented by generating a mapping between pixels in the corrected image and corresponding locations in the original image. Interpolation is then used to obtain corrected intensity values while avoiding gaps that would result from direct forward mapping.

Correcting an image may change the usable field of view and introduce blank regions near image boundaries. A tightly cropped output preserves only pixels supported by the original image, whereas a wider output can retain more peripheral information at the cost of invalid areas. Robotics software must therefore maintain consistency between the selected corrected image geometry and its corresponding camera matrix.

For many perception pipelines, explicit generation of a completely undistorted image is not always necessary. Individual feature coordinates, rays, or measurement points can instead be corrected mathematically using the calibrated distortion model. This approach may reduce unnecessary image resampling and preserve original sensor data while still providing geometrically corrected coordinates for localization or reconstruction algorithms.

Distortion parameters are valid only for the optical and imaging configuration under which they were calibrated. Changes in focus, zoom, lens mounting, sensor resolution, cropping, binning, or image scaling can modify the effective camera geometry. Calibration metadata should therefore identify the camera, lens, resolution, operating mode, and relevant configuration associated with each parameter set.

Validation should examine more than a single global reprojection-error value. Residual vectors distributed across the image can reveal systematic patterns indicating an insufficient distortion model, inaccurate principal point, target deformation, or poorly conditioned calibration data. Straight physical edges viewed across the image are also useful qualitative references for checking whether undistortion behaves consistently.

In robotic systems, a reliable lens distortion model forms part of the geometric foundation connecting camera measurements to the physical environment. Accurate distortion compensation improves the consistency of projection, reconstruction, localization, stereo correspondence, and multi-sensor alignment. It should therefore be calibrated, validated, version-controlled, and maintained together with the camera's intrinsic parameters throughout the sensor lifecycle.

06.03. Stereo Camera Calibration

Stereo camera calibration determines the geome