1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
//! Irradiance volumes, also known as voxel global illumination.
//!
//! An *irradiance volume* is a cuboid voxel region consisting of
//! regularly-spaced precomputed samples of diffuse indirect light. They're
//! ideal if you have a dynamic object such as a character that can move about
//! static non-moving geometry such as a level in a game, and you want that
//! dynamic object to be affected by the light bouncing off that static
//! geometry.
//!
//! To use irradiance volumes, you need to precompute, or *bake*, the indirect
//! light in your scene. Bevy doesn't currently come with a way to do this.
//! Fortunately, [Blender] provides a [baking tool] as part of the Eevee
//! renderer, and its irradiance volumes are compatible with those used by Bevy.
//! The [`bevy-baked-gi`] project provides a tool, `export-blender-gi`, that can
//! extract the baked irradiance volumes from the Blender `.blend` file and
//! package them up into a `.ktx2` texture for use by the engine. See the
//! documentation in the `bevy-baked-gi` project for more details on this
//! workflow.
//!
//! Like all light probes in Bevy, irradiance volumes are 1×1×1 cubes that can
//! be arbitrarily scaled, rotated, and positioned in a scene with the
//! [`bevy_transform::components::Transform`] component. The 3D voxel grid will
//! be stretched to fill the interior of the cube, and the illumination from the
//! irradiance volume will apply to all fragments within that bounding region.
//!
//! Bevy's irradiance volumes are based on Valve's [*ambient cubes*] as used in
//! *Half-Life 2* ([Mitchell 2006, slide 27]). These encode a single color of
//! light from the six 3D cardinal directions and blend the sides together
//! according to the surface normal. For an explanation of why ambient cubes
//! were chosen over spherical harmonics, see [Why ambient cubes?] below.
//!
//! If you wish to use a tool other than `export-blender-gi` to produce the
//! irradiance volumes, you'll need to pack the irradiance volumes in the
//! following format. The irradiance volume of resolution *(Rx, Ry, Rz)* is
//! expected to be a 3D texture of dimensions *(Rx, 2Ry, 3Rz)*. The unnormalized
//! texture coordinate *(s, t, p)* of the voxel at coordinate *(x, y, z)* with
//! side *S* ∈ *{-X, +X, -Y, +Y, -Z, +Z}* is as follows:
//!
//! ```text
//! s = x
//!
//! t = y + ⎰  0 if S ∈ {-X, -Y, -Z}
//!         ⎱ Ry if S ∈ {+X, +Y, +Z}
//!
//!         ⎧   0 if S ∈ {-X, +X}
//! p = z + ⎨  Rz if S ∈ {-Y, +Y}
//!         ⎩ 2Rz if S ∈ {-Z, +Z}
//! ```
//!
//! Visually, in a left-handed coordinate system with Y up, viewed from the
//! right, the 3D texture looks like a stacked series of voxel grids, one for
//! each cube side, in this order:
//!
//! | **+X** | **+Y** | **+Z** |
//! | ------ | ------ | ------ |
//! | **-X** | **-Y** | **-Z** |
//!
//! A terminology note: Other engines may refer to irradiance volumes as *voxel
//! global illumination*, *VXGI*, or simply as *light probes*. Sometimes *light
//! probe* refers to what Bevy calls a reflection probe. In Bevy, *light probe*
//! is a generic term that encompasses all cuboid bounding regions that capture
//! indirect illumination, whether based on voxels or not.
//!
//! Note that, if binding arrays aren't supported (e.g. on WebGPU or WebGL 2),
//! then only the closest irradiance volume to the view will be taken into
//! account during rendering. The required `wgpu` features are
//! [`bevy_render::settings::WgpuFeatures::TEXTURE_BINDING_ARRAY`] and
//! [`bevy_render::settings::WgpuFeatures::SAMPLED_TEXTURE_AND_STORAGE_BUFFER_ARRAY_NON_UNIFORM_INDEXING`].
//!
//! ## Why ambient cubes?
//!
//! This section describes the motivation behind the decision to use ambient
//! cubes in Bevy. It's not needed to use the feature; feel free to skip it
//! unless you're interested in its internal design.
//!
//! Bevy uses *Half-Life 2*-style ambient cubes (usually abbreviated as *HL2*)
//! as the representation of irradiance for light probes instead of the
//! more-popular spherical harmonics (*SH*). This might seem to be a surprising
//! choice, but it turns out to work well for the specific case of voxel
//! sampling on the GPU. Spherical harmonics have two problems that make them
//! less ideal for this use case:
//!
//! 1. The level 1 spherical harmonic coefficients can be negative. That
//! prevents the use of the efficient [RGB9E5 texture format], which only
//! encodes unsigned floating point numbers, and forces the use of the
//! less-efficient [RGBA16F format] if hardware interpolation is desired.
//!
//! 2. As an alternative to RGBA16F, level 1 spherical harmonics can be
//! normalized and scaled to the SH0 base color, as [Frostbite] does. This
//! allows them to be packed in standard LDR RGBA8 textures. However, this
//! prevents the use of hardware trilinear filtering, as the nonuniform scale
//! factor means that hardware interpolation no longer produces correct results.
//! The 8 texture fetches needed to interpolate between voxels can be upwards of
//! twice as slow as the hardware interpolation.
//!
//! The following chart summarizes the costs and benefits of ambient cubes,
//! level 1 spherical harmonics, and level 2 spherical harmonics:
//!
//! | Technique                | HW-interpolated samples | Texel fetches | Bytes per voxel | Quality |
//! | ------------------------ | ----------------------- | ------------- | --------------- | ------- |
//! | Ambient cubes            |                       3 |             0 |              24 | Medium  |
//! | Level 1 SH, compressed   |                       0 |            36 |              16 | Low     |
//! | Level 1 SH, uncompressed |                       4 |             0 |              24 | Low     |
//! | Level 2 SH, compressed   |                       0 |            72 |              28 | High    |
//! | Level 2 SH, uncompressed |                       9 |             0 |              54 | High    |
//!
//! (Note that the number of bytes per voxel can be reduced using various
//! texture compression methods, but the overall ratios remain similar.)
//!
//! From these data, we can see that ambient cubes balance fast lookups (from
//! leveraging hardware interpolation) with relatively-small storage
//! requirements and acceptable quality. Hence, they were chosen for irradiance
//! volumes in Bevy.
//!
//! [*ambient cubes*]: https://advances.realtimerendering.com/s2006/Mitchell-ShadingInValvesSourceEngine.pdf
//!
//! [spherical harmonics]: https://en.wikipedia.org/wiki/Spherical_harmonic_lighting
//!
//! [RGB9E5 texture format]: https://www.khronos.org/opengl/wiki/Small_Float_Formats#RGB9_E5
//!
//! [RGBA16F format]: https://www.khronos.org/opengl/wiki/Small_Float_Formats#Low-bitdepth_floats
//!
//! [Frostbite]: https://media.contentapi.ea.com/content/dam/eacom/frostbite/files/gdc2018-precomputedgiobalilluminationinfrostbite.pdf#page=53
//!
//! [Mitchell 2006, slide 27]: https://advances.realtimerendering.com/s2006/Mitchell-ShadingInValvesSourceEngine.pdf#page=27
//!
//! [Blender]: http://blender.org/
//!
//! [baking tool]: https://docs.blender.org/manual/en/latest/render/eevee/render_settings/indirect_lighting.html
//!
//! [`bevy-baked-gi`]: https://github.com/pcwalton/bevy-baked-gi
//!
//! [Why ambient cubes?]: #why-ambient-cubes

use bevy_ecs::component::Component;
use bevy_render::{
    render_asset::RenderAssets,
    render_resource::{
        binding_types, BindGroupLayoutEntryBuilder, Sampler, SamplerBindingType, Shader,
        TextureSampleType, TextureView,
    },
    renderer::RenderDevice,
    texture::{FallbackImage, Image},
};
use std::{num::NonZeroU32, ops::Deref};

use bevy_asset::{AssetId, Handle};
use bevy_reflect::Reflect;

use crate::{
    add_cubemap_texture_view, binding_arrays_are_usable, RenderViewLightProbes,
    MAX_VIEW_LIGHT_PROBES,
};

use super::LightProbeComponent;

pub const IRRADIANCE_VOLUME_SHADER_HANDLE: Handle<Shader> =
    Handle::weak_from_u128(160299515939076705258408299184317675488);

/// On WebGL and WebGPU, we must disable irradiance volumes, as otherwise we can
/// overflow the number of texture bindings when deferred rendering is in use
/// (see issue #11885).
pub(crate) const IRRADIANCE_VOLUMES_ARE_USABLE: bool = cfg!(not(target_arch = "wasm32"));

/// The component that defines an irradiance volume.
///
/// See [`crate::irradiance_volume`] for detailed information.
#[derive(Clone, Default, Reflect, Component, Debug)]
pub struct IrradianceVolume {
    /// The 3D texture that represents the ambient cubes, encoded in the format
    /// described in [`crate::irradiance_volume`].
    pub voxels: Handle<Image>,

    /// Scale factor applied to the diffuse and specular light generated by this component.
    ///
    /// After applying this multiplier, the resulting values should
    /// be in units of [cd/m^2](https://en.wikipedia.org/wiki/Candela_per_square_metre).
    ///
    /// See also <https://google.github.io/filament/Filament.html#lighting/imagebasedlights/iblunit>.
    pub intensity: f32,
}

/// All the bind group entries necessary for PBR shaders to access the
/// irradiance volumes exposed to a view.
pub(crate) enum RenderViewIrradianceVolumeBindGroupEntries<'a> {
    /// The version used when binding arrays aren't available on the current platform.
    Single {
        /// The texture view of the closest light probe.
        texture_view: &'a TextureView,
        /// A sampler used to sample voxels of the irradiance volume.
        sampler: &'a Sampler,
    },

    /// The version used when binding arrays are available on the current
    /// platform.
    Multiple {
        /// A texture view of the voxels of each irradiance volume, in the same
        /// order that they are supplied to the view (i.e. in the same order as
        /// `binding_index_to_cubemap` in [`RenderViewLightProbes`]).
        ///
        /// This is a vector of `wgpu::TextureView`s. But we don't want to import
        /// `wgpu` in this crate, so we refer to it indirectly like this.
        texture_views: Vec<&'a <TextureView as Deref>::Target>,

        /// A sampler used to sample voxels of the irradiance volumes.
        sampler: &'a Sampler,
    },
}

impl<'a> RenderViewIrradianceVolumeBindGroupEntries<'a> {
    /// Looks up and returns the bindings for any irradiance volumes visible in
    /// the view, as well as the sampler.
    pub(crate) fn get(
        render_view_irradiance_volumes: Option<&RenderViewLightProbes<IrradianceVolume>>,
        images: &'a RenderAssets<Image>,
        fallback_image: &'a FallbackImage,
        render_device: &RenderDevice,
    ) -> RenderViewIrradianceVolumeBindGroupEntries<'a> {
        if binding_arrays_are_usable(render_device) {
            RenderViewIrradianceVolumeBindGroupEntries::get_multiple(
                render_view_irradiance_volumes,
                images,
                fallback_image,
            )
        } else {
            RenderViewIrradianceVolumeBindGroupEntries::get_single(
                render_view_irradiance_volumes,
                images,
                fallback_image,
            )
        }
    }

    /// Looks up and returns the bindings for any irradiance volumes visible in
    /// the view, as well as the sampler. This is the version used when binding
    /// arrays are available on the current platform.
    fn get_multiple(
        render_view_irradiance_volumes: Option<&RenderViewLightProbes<IrradianceVolume>>,
        images: &'a RenderAssets<Image>,
        fallback_image: &'a FallbackImage,
    ) -> RenderViewIrradianceVolumeBindGroupEntries<'a> {
        let mut texture_views = vec![];
        let mut sampler = None;

        if let Some(irradiance_volumes) = render_view_irradiance_volumes {
            for &cubemap_id in &irradiance_volumes.binding_index_to_textures {
                add_cubemap_texture_view(
                    &mut texture_views,
                    &mut sampler,
                    cubemap_id,
                    images,
                    fallback_image,
                );
            }
        }

        // Pad out the bindings to the size of the binding array using fallback
        // textures. This is necessary on D3D12 and Metal.
        texture_views.resize(MAX_VIEW_LIGHT_PROBES, &*fallback_image.d3.texture_view);

        RenderViewIrradianceVolumeBindGroupEntries::Multiple {
            texture_views,
            sampler: sampler.unwrap_or(&fallback_image.d3.sampler),
        }
    }

    /// Looks up and returns the bindings for any irradiance volumes visible in
    /// the view, as well as the sampler. This is the version used when binding
    /// arrays aren't available on the current platform.
    fn get_single(
        render_view_irradiance_volumes: Option<&RenderViewLightProbes<IrradianceVolume>>,
        images: &'a RenderAssets<Image>,
        fallback_image: &'a FallbackImage,
    ) -> RenderViewIrradianceVolumeBindGroupEntries<'a> {
        if let Some(irradiance_volumes) = render_view_irradiance_volumes {
            if let Some(irradiance_volume) = irradiance_volumes.render_light_probes.first() {
                if irradiance_volume.texture_index >= 0 {
                    if let Some(image_id) = irradiance_volumes
                        .binding_index_to_textures
                        .get(irradiance_volume.texture_index as usize)
                    {
                        if let Some(image) = images.get(*image_id) {
                            return RenderViewIrradianceVolumeBindGroupEntries::Single {
                                texture_view: &image.texture_view,
                                sampler: &image.sampler,
                            };
                        }
                    }
                }
            }
        }

        RenderViewIrradianceVolumeBindGroupEntries::Single {
            texture_view: &fallback_image.d3.texture_view,
            sampler: &fallback_image.d3.sampler,
        }
    }
}

/// Returns the bind group layout entries for the voxel texture and sampler
/// respectively.
pub(crate) fn get_bind_group_layout_entries(
    render_device: &RenderDevice,
) -> [BindGroupLayoutEntryBuilder; 2] {
    let mut texture_3d_binding =
        binding_types::texture_3d(TextureSampleType::Float { filterable: true });
    if binding_arrays_are_usable(render_device) {
        texture_3d_binding =
            texture_3d_binding.count(NonZeroU32::new(MAX_VIEW_LIGHT_PROBES as _).unwrap());
    }

    [
        texture_3d_binding,
        binding_types::sampler(SamplerBindingType::Filtering),
    ]
}

impl LightProbeComponent for IrradianceVolume {
    type AssetId = AssetId<Image>;

    // Irradiance volumes can't be attached to the view, so we store nothing
    // here.
    type ViewLightProbeInfo = ();

    fn id(&self, image_assets: &RenderAssets<Image>) -> Option<Self::AssetId> {
        if image_assets.get(&self.voxels).is_none() {
            None
        } else {
            Some(self.voxels.id())
        }
    }

    fn intensity(&self) -> f32 {
        self.intensity
    }

    fn create_render_view_light_probes(
        _: Option<&Self>,
        _: &RenderAssets<Image>,
    ) -> RenderViewLightProbes<Self> {
        RenderViewLightProbes::new()
    }
}