mirror of
https://github.com/bevyengine/bevy.git
synced 2026-07-01 08:12:51 -04:00
Chris Biscardi
e7b64b6b04
# Objective Currently Bevy doesn't support arbitrary glTF extensions. The ones it does support are hardcoded. We should support glTF extensions, as this is a primary mechanism for sharing behavior via data exported from applications like Blender. I personally have found usecases in exporting component data, lightmap textures/information, and processing other kinds of data (AnimationGraph, 3d meshes into 2d, etc). ## Solution This PR introduces a new `GltfExtensionHandler` trait that users can implement and add to the glTF loader processing via inserting into a Resource. There are two example processors currently added, with a third that I'd like to add after this PR. - `examples/gltf/gltf_extension_animation_graph.rs` duplicates the functionality of `animation_mesh`, constructing AnimationGraphs via extension processing and applying them to be played on the relevant nodes. - `examples/gltf/gltf_extension_mesh_2d.rs` duplicates the functionality of the `custom_gltf_vertex_attribute` example, showing how the extension processing could be used to convert 3d meshes to 2d meshes alongside custom materials. Both of these examples re-use existing assets and thus don't *actually use* extension data, but show how one could access the relevant data to say, only convert specifically labelled Mesh3ds to 2d, or process many animations into multiple graphs based on extension-data based labelling introduced in Blender. A third example I want to introduce after this PR is the same core functionality Skein requires: an example that uses reflected component data stored in glTF extensions and inserts that data onto the relevant entities, resulting in scenes that are "ready to go". ## Comparison to Extras In comparison to extensions: data placed in glTF extras is well supported through the `GltfExtras` category of components. Extras only support adding an additional `extras` field to any object. Data stored in extras is application-specific. It should be usable by Bevy developers to implement their own, application-specific, data transfer. This is supported by applications like Blender through the application of Custom Properties. Once data is used by more than one application, it belongs in a glTF extension. ## What is a glTF Extension? Extensions are named with a prefix like `KHR` or `EXT`. Bevy has already reserved the `BEVY` namespace for this, which is listed in the official [prefix list](https://github.com/KhronosGroup/glTF/blob/7bbd90978cad06389eee3a36882c5ef2f2039faf/extensions/Prefixes.md). For a glTF file, an extension must be listed in `extensionsUsed` and optionally `extensionsRequired`. ``` { "extensionsRequired": [ "KHR_texture_transform" ], "extensionsUsed": [ "KHR_texture_transform" ] } ``` Extension data is allowed in any place extras are also allowed, but also allow much more flexibility. Extensions are also allowed to define global data, add additional binary chunks, and more. For meshes, extensions can add additional attribute names, accessor types, and/or component types `KHR_lights_punctual` is a contained and understandable example of an extension: https://github.com/KhronosGroup/glTF/blob/7bbd90978cad06389eee3a36882c5ef2f2039faf/extensions/2.0/Khronos/KHR_lights_punctual/README.md . This one happens to be already hardcoded into Bevy's handling, so it doesn't benefit from arbitrary extension processing, but there are additional [ratified](https://github.com/KhronosGroup/glTF/tree/7bbd90978cad06389eee3a36882c5ef2f2039faf/extensions#ratified-khronos-extensions) and [in-progress](https://github.com/KhronosGroup/glTF/tree/7bbd90978cad06389eee3a36882c5ef2f2039faf/extensions#in-progress-khronos-and-multi-vendor-extensions-and-projects) extensions, as well as [vendor](https://github.com/KhronosGroup/glTF/tree/7bbd90978cad06389eee3a36882c5ef2f2039faf/extensions#vendor-extensions) and other arbitrary extensions that would benefit from userland support. ## Implementation This initial implementation is reasonably minimal: enabling extension processing for objects/etc as they're loaded which may also define extension data, including the scene world. This may leave out useful functionality; as detailed in the next section: "What's not implemented". Extension handlers are defined by implementing a trait which can optionally define hooks and data. Extension handler data is cloned to start with a fresh slate for each glTF load, which limits scope to "one glTF load". So while state can be maintained across hooks during a single load, users who want to combine or handle multiple glTF assets should do so in the main app, not in an extension handler. Following this, because the extensions are stored as `dyn GltfExtension` *and* we want to clone them to isolate state to a single load, `dyn_clone` must be included as a workaround to enable this cloning. An extension handler has to be added to the list of handler by accessing a `Resource` and pushing an instantiated handler into it. This Resource keeps the list of extension handlers so that a new glTF loader can bootstrap them. The design of the hooks is such that: - If no extensions handlers are registered, none are called for processing - If an extension handler is defined, it receives all "events" - handlers are defined by a trait, and default implementations are called if an override is not specified. - default implementations are no-ops It is important that extensions receive all events because certain information is not embedded in extension data. For example, processing animation data into an animation graph could require both processing animations with extension data, tracking the animation roots through hooks like `on_node`, *and* applying those graphs in the `on_scene_completed` hook. - Extension data is passed to hooks as `Option<&serde_json::Value>` which is only passing references around as the data has already been converted to `Value` by the `gltf` crate. - `LoadContext` is required for creating any new additional assets, like `AnimationGraph`s. - *scene* World access is provided in hooks like `on_scene_completed`, which allows calculating data over the course of a glTF load and applying it to a Scene. ### What's not implemented This PR chooses to *not* implement some features that it could. Instead the approach in this PR is to offer up the data that Bevy has already processed to extensions to do more with that data. - Overriding `load_image`/`process_loaded_texture` - This could allow projects like bevy_web_codecs, [which currently forks the entire gltf loader](https://github.com/jf908/bevy_web_codecs/tree/373bbf29be6555c7603fd6867a01159ab0f20fed/bevy_web_codecs_gltf). Associated [issue](https://github.com/bevyengine/bevy/issues/21185). However I believe this needs some design work dedicated to what exactly happens here to support that use case. - This PR doesn't include any refactoring of the glTF loader, which I feel is important for a first merge. - ~~There is some benefit to passing in the relevant `gltf::*` object to every hook. For example, I believe this is the only way to access extension data for `KHR_lights_punctual`, and [`KHR_materials_variants`](https://docs.rs/gltf/1.4.1/gltf/struct.Document.html#method.variants) or other extensions with "built-in" support. I haven't done this in all places.~~ (edit: after external implementation I decided this was a good idea and added it to more places) ## Testing ``` cargo run --example gltf_extension_animation_graph cargo run --example gltf_extension_mesh_2d ``` --- ## Showcase Both examples running: https://github.com/user-attachments/assets/f9e7c3c9-cdad-4d33-ace7-7c2ca5469d5e https://github.com/user-attachments/assets/baa9bc92-ca3b-46ad-a3f0-2f74bbc29b68 <details> <summary>An example that showcases converting Mesh3d to Mesh2d</summary> ```rust #[derive(Default, Clone)] struct GltfExtensionProcessorToMesh2d; impl GltfExtensionProcessor for GltfExtensionProcessorToMesh2d { fn extension_ids(&self) -> &'static [&'static str] { &[""] } fn dyn_clone(&self) -> Box<dyn GltfExtensionHandler> { Box::new((*self).clone()) } fn on_spawn_mesh_and_material( &mut self, load_context: &mut LoadContext<'_>, _gltf_node: &gltf::Node, entity: &mut EntityWorldMut, ) { if let Some(mesh3d) = entity.get::<Mesh3d>() && let Some(_) = entity.get::<MeshMaterial3d<StandardMaterial>>() { let material_handle = load_context.add_loaded_labeled_asset("AColorMaterial", (CustomMaterial {}).into()); let mesh_handle = mesh3d.0.clone(); entity .remove::<(Mesh3d, MeshMaterial3d<StandardMaterial>)>() .insert((Mesh2d(mesh_handle), MeshMaterial2d(material_handle.clone()))); } } } ``` </details> --------- Co-authored-by: Kristoffer Søholm <k.soeholm@gmail.com>