Every vital sign that a camera extracts from facial video begins with a photon hitting skin. What happens next — absorption, scattering, reflection — depends on the wavelength of that photon, the layers of tissue it encounters, and the chromophores present in each layer. Understanding this optical interaction isn't just academic. It explains why rPPG works, why the green channel dominates heart rate extraction, why skin tone affects signal quality, and what engineers can do about it.
The physics of light-tissue interaction underpins every rPPG algorithm ever published. Yet most discussions of camera-based vital signs skip directly to signal processing without examining the optical foundation. This gap matters because the opportunities and limitations of rPPG are fundamentally optical before they are computational.
"We have shown that the plethysmographic signal can be remotely measured from the human face using a simple consumer-level digital camera and ambient light as the illumination source." — Verkruysse, Svaasand, and Nelson, Optics Express (2008)
The Physics of Light-Tissue Interaction
When light strikes the skin surface, three things happen simultaneously: a portion reflects off the surface (specular reflection), a portion enters the tissue and gets absorbed, and a portion scatters through the tissue before eventually exiting (diffuse reflection). The ratio between these three outcomes determines what the camera sees — and what physiological information can be extracted.
Specular reflection bounces off the skin surface at a predictable angle, carrying no physiological information. It's essentially glare — the shiny highlight on a forehead under direct lighting. rPPG algorithms actively suppress specular reflection because it's a noise source.
Absorption is where the physiological signal lives. Light that penetrates the skin encounters chromophores — molecules that absorb specific wavelengths. The primary chromophores relevant to rPPG are hemoglobin (in blood), melanin (in the epidermis), and water (in all tissue layers). Each absorbs light at different wavelengths and to different degrees, creating a spectral fingerprint that cameras can decode.
Scattering occurs when photons bounce off cellular structures, collagen fibers, and other tissue components. Scattering is what allows light to penetrate deep enough to reach dermal blood vessels and then return to the surface where a camera can capture it. Without scattering, there would be no rPPG — light would either reflect off the surface or absorb completely.
The Beer-Lambert law provides the mathematical framework: light intensity decreases exponentially with the concentration of absorbers and the path length through tissue. In rPPG, the pulsatile component of blood volume — driven by the cardiac cycle — modulates the absorber concentration in real time, creating the tiny fluctuations in reflected light that cameras detect.
Skin Layers and Light Penetration
The skin has three primary layers, each with different optical properties:
- Epidermis (50-100 μm): The outermost layer contains melanin, the dominant absorber for visible light. Melanin concentration varies with skin type and determines how much light reaches deeper layers.
- Dermis (1-4 mm): Contains the capillary beds and arterial plexus where blood volume changes occur. This is where the rPPG signal originates. Collagen fibers in the dermis cause significant scattering.
- Hypodermis: Subcutaneous fat layer. Light that penetrates this deep rarely returns to the surface at useful intensities for rPPG.
Green light (500-570nm) penetrates to the upper dermis — deep enough to reach the superficial capillary plexus but not so deep that scattering losses eliminate the return signal. This is one reason green channel dominance in heart rate extraction isn't arbitrary — it's an optical property of tissue.
Chromophores That Matter
| Chromophore | Location | Absorption Peaks | Role in rPPG | Pulsatile? |
|---|---|---|---|---|
| Oxyhemoglobin (HbO₂) | Blood (dermis) | 418nm, 542nm, 577nm | Primary pulse signal carrier | Yes — varies with cardiac cycle |
| Deoxyhemoglobin (Hb) | Blood (dermis) | 430nm, 555nm | SpO2 estimation (ratio with HbO₂) | Yes — varies with cardiac cycle |
| Melanin | Epidermis | Broadband (stronger at shorter wavelengths) | Signal attenuator — reduces SNR | No — static |
| Water | All layers | >970nm (near-infrared) | Minimal visible-light impact | No — static |
| Bilirubin | Blood | ~460nm (blue) | Minor contributor, jaundice detection | Partially |
Hemoglobin is the star of rPPG. Oxygenated hemoglobin (HbO₂) and deoxygenated hemoglobin (Hb) have distinct absorption spectra, and their relative concentrations change with each heartbeat. The total hemoglobin concentration in the dermis rises and falls with arterial pulsation — this is the blood volume pulse that rPPG measures. At 540nm (green), hemoglobin absorption is particularly strong, which is why the green channel carries the clearest cardiac signal in standard RGB cameras.
The difference in absorption spectra between HbO₂ and Hb is also what makes SpO2 estimation possible. Pulse oximeters exploit this by comparing absorption at two wavelengths (typically 660nm red and 940nm infrared). rPPG researchers are exploring whether the red, green, and blue channels of a standard camera provide enough spectral separation to estimate the same ratio — an active and challenging area of research (Casalino et al., 2022).
Melanin is the primary complicating factor. As a broadband absorber concentrated in the epidermis, melanin attenuates all visible light before it reaches the dermal blood vessels. This reduces the amplitude of the blood volume pulse signal that returns to the camera, lowering the signal-to-noise ratio. The effect is wavelength-dependent — melanin absorbs shorter wavelengths (blue, green) more strongly than longer wavelengths (red, NIR) — which has implications for algorithm design.
Why Skin Tone Affects the Signal
The Fitzpatrick skin type scale classifies skin into six types based on melanin content and sun reactivity. From an rPPG perspective, the key variable is epidermal melanin concentration, which affects the optical path in two ways: it reduces the total amount of light reaching the dermis, and it preferentially attenuates the shorter wavelengths where hemoglobin absorption signals are strongest.
Nowara et al. (2020) documented performance differences across skin tones for several rPPG algorithms, finding that accuracy degraded for Fitzpatrick types V and VI compared to types I and II. This isn't a flaw in any single algorithm — it's a fundamental optical challenge that every light-based measurement faces. A finger pulse oximeter is affected by the same physics, though its controlled sensor geometry and calibrated light sources mitigate the impact.
Wang et al. (2017) addressed this directly with the POS algorithm, which projects the RGB signal onto a plane orthogonal to the skin tone vector. By explicitly accounting for the direction of skin-tone variation in color space, POS achieves more consistent performance across complexions than methods that don't model skin tone geometry.
Ba et al. (2023) showed that deep learning models trained on diverse datasets — with balanced representation across Fitzpatrick types — substantially narrowed the accuracy gap. The issue is partly algorithmic and partly a data problem: early rPPG datasets were heavily biased toward lighter skin tones, and models trained on them inherited that bias.
Engineering Around Optical Challenges
Researchers and engineers have developed several strategies to work within the physical constraints of light-tissue interaction:
Adaptive ROI selection targets the highest-perfusion skin regions dynamically, adjusting for head pose and selecting areas where the pulse signal is strongest relative to noise. Forehead-focused ROIs often outperform full-face averaging because the forehead has less subcutaneous fat and fewer facial expressions that cause non-rigid motion.
Illumination normalization techniques compensate for changes in ambient lighting by modeling the expected illumination component separately from the physiological signal. The CHROM algorithm (de Haan and Jeanne, 2013) approaches this through chrominance-based projection that cancels common-mode illumination variation.
Near-infrared augmentation leverages wavelengths (750-1000nm) where melanin absorption is significantly lower. NIR cameras or RGB+NIR systems penetrate deeper and provide more uniform performance across skin tones. While standard consumer cameras lack dedicated NIR channels, some researchers have explored modified camera systems for clinical rPPG applications.
Multi-site measurement combines signals from multiple facial regions, exploiting the fact that the pulse signal is correlated across sites while noise (motion, local lighting) is less correlated. Averaging or weighted combination of multi-site signals improves SNR without requiring hardware changes.
Frequently Asked Questions
Why does rPPG primarily use the green channel for heart rate?
Hemoglobin — the oxygen-carrying molecule in blood — has a strong absorption peak near 540nm, which falls squarely in the green portion of the visible spectrum. When blood volume increases with each heartbeat, green light absorption increases more than red or blue, making the green channel the most sensitive detector of the cardiac pulse signal.
How does skin tone affect rPPG accuracy?
Melanin acts as a broadband light absorber concentrated in the epidermis. Higher melanin concentration reduces the overall intensity of light reaching the dermal blood vessels and returning to the camera, which lowers the signal-to-noise ratio of the blood volume pulse. Modern algorithms address this through adaptive signal processing and diverse training datasets.
What is the Beer-Lambert law and how does it apply to rPPG?
The Beer-Lambert law describes how light intensity decreases exponentially as it passes through an absorbing medium. In rPPG, it models how light passing through skin is absorbed by chromophores like hemoglobin and melanin. The pulsatile component of absorption — caused by blood volume changes — is the signal rPPG algorithms extract.
Can rPPG work with infrared light?
Yes. Near-infrared (NIR) light penetrates deeper into tissue and is less affected by melanin absorption, which improves performance across skin tones and in low visible light conditions. Some researchers are exploring NIR-augmented rPPG systems that combine RGB and infrared channels for more robust measurement.
Related Articles
- What is rPPG Technology? — A comprehensive overview of rPPG covering the science, vital signs measured, and clinical applications.
- Contactless SpO2 Monitoring — SpO2 estimation relies directly on the differential absorption spectra of oxygenated and deoxygenated hemoglobin described here.
- rPPG Signal Processing — How algorithms transform the optical signals from light-skin interaction into vital sign measurements.