1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343
//! Irradiance volumes, also known as voxel global illumination.
//!
//! An *irradiance volume* is a cuboid voxel region consisting of
//! regularly-spaced precomputed samples of diffuse indirect light. They're
//! ideal if you have a dynamic object such as a character that can move about
//! static non-moving geometry such as a level in a game, and you want that
//! dynamic object to be affected by the light bouncing off that static
//! geometry.
//!
//! To use irradiance volumes, you need to precompute, or *bake*, the indirect
//! light in your scene. Bevy doesn't currently come with a way to do this.
//! Fortunately, [Blender] provides a [baking tool] as part of the Eevee
//! renderer, and its irradiance volumes are compatible with those used by Bevy.
//! The [`bevy-baked-gi`] project provides a tool, `export-blender-gi`, that can
//! extract the baked irradiance volumes from the Blender `.blend` file and
//! package them up into a `.ktx2` texture for use by the engine. See the
//! documentation in the `bevy-baked-gi` project for more details on this
//! workflow.
//!
//! Like all light probes in Bevy, irradiance volumes are 1×1×1 cubes that can
//! be arbitrarily scaled, rotated, and positioned in a scene with the
//! [`bevy_transform::components::Transform`] component. The 3D voxel grid will
//! be stretched to fill the interior of the cube, and the illumination from the
//! irradiance volume will apply to all fragments within that bounding region.
//!
//! Bevy's irradiance volumes are based on Valve's [*ambient cubes*] as used in
//! *Half-Life 2* ([Mitchell 2006, slide 27]). These encode a single color of
//! light from the six 3D cardinal directions and blend the sides together
//! according to the surface normal. For an explanation of why ambient cubes
//! were chosen over spherical harmonics, see [Why ambient cubes?] below.
//!
//! If you wish to use a tool other than `export-blender-gi` to produce the
//! irradiance volumes, you'll need to pack the irradiance volumes in the
//! following format. The irradiance volume of resolution *(Rx, Ry, Rz)* is
//! expected to be a 3D texture of dimensions *(Rx, 2Ry, 3Rz)*. The unnormalized
//! texture coordinate *(s, t, p)* of the voxel at coordinate *(x, y, z)* with
//! side *S* ∈ *{-X, +X, -Y, +Y, -Z, +Z}* is as follows:
//!
//! ```text
//! s = x
//!
//! t = y + ⎰ 0 if S ∈ {-X, -Y, -Z}
//! ⎱ Ry if S ∈ {+X, +Y, +Z}
//!
//! ⎧ 0 if S ∈ {-X, +X}
//! p = z + ⎨ Rz if S ∈ {-Y, +Y}
//! ⎩ 2Rz if S ∈ {-Z, +Z}
//! ```
//!
//! Visually, in a left-handed coordinate system with Y up, viewed from the
//! right, the 3D texture looks like a stacked series of voxel grids, one for
//! each cube side, in this order:
//!
//! | **+X** | **+Y** | **+Z** |
//! | ------ | ------ | ------ |
//! | **-X** | **-Y** | **-Z** |
//!
//! A terminology note: Other engines may refer to irradiance volumes as *voxel
//! global illumination*, *VXGI*, or simply as *light probes*. Sometimes *light
//! probe* refers to what Bevy calls a reflection probe. In Bevy, *light probe*
//! is a generic term that encompasses all cuboid bounding regions that capture
//! indirect illumination, whether based on voxels or not.
//!
//! Note that, if binding arrays aren't supported (e.g. on WebGPU or WebGL 2),
//! then only the closest irradiance volume to the view will be taken into
//! account during rendering. The required `wgpu` features are
//! [`bevy_render::settings::WgpuFeatures::TEXTURE_BINDING_ARRAY`] and
//! [`bevy_render::settings::WgpuFeatures::SAMPLED_TEXTURE_AND_STORAGE_BUFFER_ARRAY_NON_UNIFORM_INDEXING`].
//!
//! ## Why ambient cubes?
//!
//! This section describes the motivation behind the decision to use ambient
//! cubes in Bevy. It's not needed to use the feature; feel free to skip it
//! unless you're interested in its internal design.
//!
//! Bevy uses *Half-Life 2*-style ambient cubes (usually abbreviated as *HL2*)
//! as the representation of irradiance for light probes instead of the
//! more-popular spherical harmonics (*SH*). This might seem to be a surprising
//! choice, but it turns out to work well for the specific case of voxel
//! sampling on the GPU. Spherical harmonics have two problems that make them
//! less ideal for this use case:
//!
//! 1. The level 1 spherical harmonic coefficients can be negative. That
//! prevents the use of the efficient [RGB9E5 texture format], which only
//! encodes unsigned floating point numbers, and forces the use of the
//! less-efficient [RGBA16F format] if hardware interpolation is desired.
//!
//! 2. As an alternative to RGBA16F, level 1 spherical harmonics can be
//! normalized and scaled to the SH0 base color, as [Frostbite] does. This
//! allows them to be packed in standard LDR RGBA8 textures. However, this
//! prevents the use of hardware trilinear filtering, as the nonuniform scale
//! factor means that hardware interpolation no longer produces correct results.
//! The 8 texture fetches needed to interpolate between voxels can be upwards of
//! twice as slow as the hardware interpolation.
//!
//! The following chart summarizes the costs and benefits of ambient cubes,
//! level 1 spherical harmonics, and level 2 spherical harmonics:
//!
//! | Technique | HW-interpolated samples | Texel fetches | Bytes per voxel | Quality |
//! | ------------------------ | ----------------------- | ------------- | --------------- | ------- |
//! | Ambient cubes | 3 | 0 | 24 | Medium |
//! | Level 1 SH, compressed | 0 | 36 | 16 | Low |
//! | Level 1 SH, uncompressed | 4 | 0 | 24 | Low |
//! | Level 2 SH, compressed | 0 | 72 | 28 | High |
//! | Level 2 SH, uncompressed | 9 | 0 | 54 | High |
//!
//! (Note that the number of bytes per voxel can be reduced using various
//! texture compression methods, but the overall ratios remain similar.)
//!
//! From these data, we can see that ambient cubes balance fast lookups (from
//! leveraging hardware interpolation) with relatively-small storage
//! requirements and acceptable quality. Hence, they were chosen for irradiance
//! volumes in Bevy.
//!
//! [*ambient cubes*]: https://advances.realtimerendering.com/s2006/Mitchell-ShadingInValvesSourceEngine.pdf
//!
//! [spherical harmonics]: https://en.wikipedia.org/wiki/Spherical_harmonic_lighting
//!
//! [RGB9E5 texture format]: https://www.khronos.org/opengl/wiki/Small_Float_Formats#RGB9_E5
//!
//! [RGBA16F format]: https://www.khronos.org/opengl/wiki/Small_Float_Formats#Low-bitdepth_floats
//!
//! [Frostbite]: https://media.contentapi.ea.com/content/dam/eacom/frostbite/files/gdc2018-precomputedgiobalilluminationinfrostbite.pdf#page=53
//!
//! [Mitchell 2006, slide 27]: https://advances.realtimerendering.com/s2006/Mitchell-ShadingInValvesSourceEngine.pdf#page=27
//!
//! [Blender]: http://blender.org/
//!
//! [baking tool]: https://docs.blender.org/manual/en/latest/render/eevee/render_settings/indirect_lighting.html
//!
//! [`bevy-baked-gi`]: https://github.com/pcwalton/bevy-baked-gi
//!
//! [Why ambient cubes?]: #why-ambient-cubes
use bevy_ecs::component::Component;
use bevy_render::{
render_asset::RenderAssets,
render_resource::{
binding_types, BindGroupLayoutEntryBuilder, Sampler, SamplerBindingType, Shader,
TextureSampleType, TextureView,
},
renderer::RenderDevice,
texture::{FallbackImage, Image},
};
use std::{num::NonZeroU32, ops::Deref};
use bevy_asset::{AssetId, Handle};
use bevy_reflect::Reflect;
use crate::{
add_cubemap_texture_view, binding_arrays_are_usable, RenderViewLightProbes,
MAX_VIEW_LIGHT_PROBES,
};
use super::LightProbeComponent;
pub const IRRADIANCE_VOLUME_SHADER_HANDLE: Handle<Shader> =
Handle::weak_from_u128(160299515939076705258408299184317675488);
/// On WebGL and WebGPU, we must disable irradiance volumes, as otherwise we can
/// overflow the number of texture bindings when deferred rendering is in use
/// (see issue #11885).
pub(crate) const IRRADIANCE_VOLUMES_ARE_USABLE: bool = cfg!(not(target_arch = "wasm32"));
/// The component that defines an irradiance volume.
///
/// See [`crate::irradiance_volume`] for detailed information.
#[derive(Clone, Default, Reflect, Component, Debug)]
pub struct IrradianceVolume {
/// The 3D texture that represents the ambient cubes, encoded in the format
/// described in [`crate::irradiance_volume`].
pub voxels: Handle<Image>,
/// Scale factor applied to the diffuse and specular light generated by this component.
///
/// After applying this multiplier, the resulting values should
/// be in units of [cd/m^2](https://en.wikipedia.org/wiki/Candela_per_square_metre).
///
/// See also <https://google.github.io/filament/Filament.html#lighting/imagebasedlights/iblunit>.
pub intensity: f32,
}
/// All the bind group entries necessary for PBR shaders to access the
/// irradiance volumes exposed to a view.
pub(crate) enum RenderViewIrradianceVolumeBindGroupEntries<'a> {
/// The version used when binding arrays aren't available on the current platform.
Single {
/// The texture view of the closest light probe.
texture_view: &'a TextureView,
/// A sampler used to sample voxels of the irradiance volume.
sampler: &'a Sampler,
},
/// The version used when binding arrays are available on the current
/// platform.
Multiple {
/// A texture view of the voxels of each irradiance volume, in the same
/// order that they are supplied to the view (i.e. in the same order as
/// `binding_index_to_cubemap` in [`RenderViewLightProbes`]).
///
/// This is a vector of `wgpu::TextureView`s. But we don't want to import
/// `wgpu` in this crate, so we refer to it indirectly like this.
texture_views: Vec<&'a <TextureView as Deref>::Target>,
/// A sampler used to sample voxels of the irradiance volumes.
sampler: &'a Sampler,
},
}
impl<'a> RenderViewIrradianceVolumeBindGroupEntries<'a> {
/// Looks up and returns the bindings for any irradiance volumes visible in
/// the view, as well as the sampler.
pub(crate) fn get(
render_view_irradiance_volumes: Option<&RenderViewLightProbes<IrradianceVolume>>,
images: &'a RenderAssets<Image>,
fallback_image: &'a FallbackImage,
render_device: &RenderDevice,
) -> RenderViewIrradianceVolumeBindGroupEntries<'a> {
if binding_arrays_are_usable(render_device) {
RenderViewIrradianceVolumeBindGroupEntries::get_multiple(
render_view_irradiance_volumes,
images,
fallback_image,
)
} else {
RenderViewIrradianceVolumeBindGroupEntries::get_single(
render_view_irradiance_volumes,
images,
fallback_image,
)
}
}
/// Looks up and returns the bindings for any irradiance volumes visible in
/// the view, as well as the sampler. This is the version used when binding
/// arrays are available on the current platform.
fn get_multiple(
render_view_irradiance_volumes: Option<&RenderViewLightProbes<IrradianceVolume>>,
images: &'a RenderAssets<Image>,
fallback_image: &'a FallbackImage,
) -> RenderViewIrradianceVolumeBindGroupEntries<'a> {
let mut texture_views = vec![];
let mut sampler = None;
if let Some(irradiance_volumes) = render_view_irradiance_volumes {
for &cubemap_id in &irradiance_volumes.binding_index_to_textures {
add_cubemap_texture_view(
&mut texture_views,
&mut sampler,
cubemap_id,
images,
fallback_image,
);
}
}
// Pad out the bindings to the size of the binding array using fallback
// textures. This is necessary on D3D12 and Metal.
texture_views.resize(MAX_VIEW_LIGHT_PROBES, &*fallback_image.d3.texture_view);
RenderViewIrradianceVolumeBindGroupEntries::Multiple {
texture_views,
sampler: sampler.unwrap_or(&fallback_image.d3.sampler),
}
}
/// Looks up and returns the bindings for any irradiance volumes visible in
/// the view, as well as the sampler. This is the version used when binding
/// arrays aren't available on the current platform.
fn get_single(
render_view_irradiance_volumes: Option<&RenderViewLightProbes<IrradianceVolume>>,
images: &'a RenderAssets<Image>,
fallback_image: &'a FallbackImage,
) -> RenderViewIrradianceVolumeBindGroupEntries<'a> {
if let Some(irradiance_volumes) = render_view_irradiance_volumes {
if let Some(irradiance_volume) = irradiance_volumes.render_light_probes.first() {
if irradiance_volume.texture_index >= 0 {
if let Some(image_id) = irradiance_volumes
.binding_index_to_textures
.get(irradiance_volume.texture_index as usize)
{
if let Some(image) = images.get(*image_id) {
return RenderViewIrradianceVolumeBindGroupEntries::Single {
texture_view: &image.texture_view,
sampler: &image.sampler,
};
}
}
}
}
}
RenderViewIrradianceVolumeBindGroupEntries::Single {
texture_view: &fallback_image.d3.texture_view,
sampler: &fallback_image.d3.sampler,
}
}
}
/// Returns the bind group layout entries for the voxel texture and sampler
/// respectively.
pub(crate) fn get_bind_group_layout_entries(
render_device: &RenderDevice,
) -> [BindGroupLayoutEntryBuilder; 2] {
let mut texture_3d_binding =
binding_types::texture_3d(TextureSampleType::Float { filterable: true });
if binding_arrays_are_usable(render_device) {
texture_3d_binding =
texture_3d_binding.count(NonZeroU32::new(MAX_VIEW_LIGHT_PROBES as _).unwrap());
}
[
texture_3d_binding,
binding_types::sampler(SamplerBindingType::Filtering),
]
}
impl LightProbeComponent for IrradianceVolume {
type AssetId = AssetId<Image>;
// Irradiance volumes can't be attached to the view, so we store nothing
// here.
type ViewLightProbeInfo = ();
fn id(&self, image_assets: &RenderAssets<Image>) -> Option<Self::AssetId> {
if image_assets.get(&self.voxels).is_none() {
None
} else {
Some(self.voxels.id())
}
}
fn intensity(&self) -> f32 {
self.intensity
}
fn create_render_view_light_probes(
_: Option<&Self>,
_: &RenderAssets<Image>,
) -> RenderViewLightProbes<Self> {
RenderViewLightProbes::new()
}
}