Unofficial Geometry Script & DynamicMeshComponent FAQ

Geometry Script(ing) is a Blueprint/Python (UFunction) library first released in Unreal Engine 5.0 that allows users to query and manipulate triangle meshes (and a few other geometric data types). I initially developed Geometry Script based on some previous public experiments I published on this website, specifically Mesh Generation and Editing at Runtime and Procedural Mesh Blueprints.

At time of writing, Geometry Script is an Experimental feature plugin in UE 5.1, which means it has pretty minimal documentation and learning materials. I have published a short series of tutorial videos on YouTube demonstrating how to use Geometry Script Blueprints for various tasks, see the playlist here. Geometry Script was also used heavily in the level design of the UE5 Lyra sample game, see documentation here.

As the main developer of Geometry Script, I get a lot of questions about how to use it, what it can do, etc. A lot of the same questions. So this page is (hopefully) a living document that I will update over time. Geometry Script is used primarily to modify UDynamicMesh objects, and the main way you access or render a UDynamicMesh is via DynamicMeshComponent / DynamicMeshActor. So, this FAQ will also cover some aspects of DynamicMeshComponent that frequently come up in the context of Geometry Scripting.

If you have questions this FAQ doesn’t answer, you might try posting on the Unreal Developer Community forums (https://dev.epicgames.com/community/), asking in the #geometry-scripting channel on the UnrealSlackers Discord ( https://unrealslackers.org ), or @ me on Mastodon (https://mastodon.gamedev.place/@rms80), or (still) Twitter (https://twitter.com/rms80). Note, however, that I strongly prefer to answer questions in public rather than in private/DM.

(Mandatory Disclaimer: your author, Ryan Schmidt, is an employee of Epic Games. However, gradientspace.com is his personal website and this article represents his personal thoughts and opinions. About triangles.)

Contents

(note: sorry, none of these are linked yet - soon!)

Basics

• Is there any documentation for Geometry Script at all?

• Does Geometry Script have a function for X?

• Is there a published Roadmap for Geometry Script?

• None of these Geometry Script functions show up for me in the Blueprint Editor

• Some functions are missing when I try to use them in my Actor Blueprint

• I can’t find the function “Event On Rebuild Generated Mesh” to override in my Actor Blueprint

• Does Geometry Script always run on the Game Thread?

• Can I run GeometryScript functions in a background thread or asynchronously?

• Is there any built-in support for running Geometry Script functions asynchronously?

• Can I run a Geometry Script Blueprint on the GPU?

• Does Geometry Script work with Skeletal Meshes?

• Does Geometry Script work with Landscape, Geometry Caches, Geometry Collections, Hair/Grooms, Cloth Meshes, or some other UE Geometry Representation?Is Geometry Script Deterministic? Are scripts likely to break in the future?

• Can I animate a mesh with Geometry Scripting? Can I implement my own skinning or vertex deformation?

Runtime Usage

• Can I use Geometry Script in my Game / At Runtime?

• Should I use DynamicMeshActors generated with Geometry Script in my Game?

• Will DynamicMeshComponent be as efficient as StaticMeshComponent in my Game?

• Why are all my GeneratedDynamicMeshActors disappearing in PIE or my built game ?!?

• Is GeometryScript function X fast enough to use in my game?

• How can I save/load Dynamic Meshes at Runtime?

• Can I use Geometry Script to modify Static Meshes in my Level at Runtime?

• The function “Copy Mesh From Static Mesh” is not working at Runtime

• The Mesh I get when using “Copy Mesh From Static Mesh” at Runtime is different than the mesh I get in the Editor

• The functions “Apply Displace from Texture Map” and/or “Sample Texture2D at UV Positions” are working in the Editor but not at Runtime

Rendering and UE Features

• Does DynamicMeshComponent support Nanite, Lumen, or Mesh Distance Fields?

• Does DynamicMeshComponent work with Runtime Virtual Texturing (RVT)?

• Does DynamicMeshComponent support Physics / Collision?

• DynamicMeshComponents don’t show up in Collision Debug Views!

• Does DynamicMeshComponent support LODs?

• Does DynamicMeshComponent support Instanced Rendering?

Lyra Sample Game

• How does the Non-Destructive Level Design with GeometryScript-based mesh generator “Tool” objects work in Lyra?

• How can I migrate the Lyra Tool system to my own project?

Basics

Is there any documentation for Geometry Script at all?

Yes! Here is a direct link into the UE5 documentation: https://docs.unrealengine.com/5.1/en-US/geometry-script-users-guide .

Several livestream and tutorial sessions have also been recorded. At UnrealFest 2022, the Introduction to Geometry Scripting session demonstrated how to create various Editor Utilities with Geometry Script, and during the Modeling and Geometry Scripting in UE: Past, Present, and Future session I gave a brief demo and some high-level context around Geometry Script. Earlier in 2022, I participated in an Inside Unreal livestream where I did some Geometry Scripting demos.

Does Geometry Script have a function for X?

This is often a difficult question to answer without more information. However, a relatively complete reference for all the current Geometry Script library functions is available in the UE5 documentation here: https://docs.unrealengine.com/5.1/en-US/geometry-script-reference-in-unreal-engine

Is there a published Roadmap for Geometry Script?

Currently there is not. Geometry Script is being actively developed and the most effective way to see what is coming in the next UE5 Release is to look at what commits have been made in the UE5 Main branch in the Unreal Engine Github. Here is a direct link to the Geometry Script plugin history: https://github.com/EpicGames/UnrealEngine/commits/ue5-main/Engine/Plugins/Experimental/GeometryScripting. Note that this link will not work unless you are logged into GitHub with an account that has access to Unreal Engine, which requires you to sign up for an Epic Games account (more information here).

None of these Geometry Script functions show up for me in the Blueprint Editor

You probably don’t have the Geometry Script plugin enabled. It is not enabled by default. The first video in my Geometry Script Tutorial Playlist on Youtube shows how to turn on the Plugin.

Some functions are missing when I try to use them in my Actor Blueprint

You are likely trying to use a function that is Editor-Only. Some functions like creating new Volumes or StaticMesh/SkeletalMesh Assets, and the Catmull Clark SubD functions, are Editor-Only and can only be used in Editor Utility Actors/Actions/Widgets, or GeneratedDynamicMeshActor BP subclasses.

I can’t find the function “Event On Rebuild Generated Mesh” to override in my Actor Blueprint

This event only exists in Actor Blueprints that derive from the GeneratedDynamicMeshActor class. It’s likely you are trying to find it in a generic Actor Blueprint, or in a DynamicMeshActor Blueprint.

Does Geometry Script always run on the Game Thread?

Currently Yes. Actor Blueprints and Editor Utility Blueprints are always executed on the Game Thread, and so the Geometry Script functions that are called also run on the Game Thread. Some Geometry Script functions will internally execute portions of their work on task threads, eg via C++ calls to ParallelFor, Async, or UE::Tasks::Launch(). However this will only occur in the context of a single function, and the function will not return until all that parallel work is completed.

Can I run GeometryScript functions in a background thread or asynchronously?

It is generally considered to be not safe to modify any UObject in a background thread. Geometry Script functions modify a UDynamicMesh, which is a UObject, and technically it is possible for a UObject to become unreferenced and garbage-collected at any time.

However, if in your specific use case you know that the UObject will not become unreferenced, then most(***) Geometry Script functions can safely be run in a background thread, as long as you don’t try to edit the same mesh from multiple threads. In my article on Modeling Mode Extension Plugins, I demonstrated taking advantage of this to build interactive mesh editing tools using Geometry Script that compute the mesh edit asynchronously.

The (***) above is because any Geometry Script function that touches an Asset, Component, or Actor (eg has any of those as input) cannot safely be run asynchronously.

Is there any built-in support for running Geometry Script functions asynchronously?

No, as of 5.1 there is not.

Can I run a Geometry Script Blueprint on the GPU?

No, this is not possible and won’t ever be. Geometry Script is a thin wrapper around a large C++ library of mesh processing algorithms and data structures. General-purpose C++ code cannot be run directly on a GPU. In addition, many of the mesh processing operations exposed in Geometry Script, like Mesh Booleans, involve complex queries and modifications over unstructured graphs where dynamic memory allocations are generally involved, which is the kind of computation problem that CPUs are much better at than GPUs.

Does Geometry Script work with Skeletal Meshes?

In 5.1 there is limited support for from a SkeletalMesh to a DynamicMesh and back, similar to the StaticMesh approach. However, how to automatically generate or update skin weights after complex mesh edits basically remains an unsolved problem in animation, and so procedural mesh edits done this way likely will not result in desirable skin weights.

Does Geometry Script work with Landscape, Geometry Caches, Geometry Collections, Hair/Grooms, Cloth Meshes, or some other UE Geometry Representation?

Not in UE 5.1. Nearly all Geometry Script functions only work with UDynamicMesh objects. There are functions to convert the internal mesh representations from Static and Skeletal Meshes, and Volume Actors, into a UDynamicMesh, and then functions to convert back. No such functions currently exist for these other geometry types.

Is Geometry Script Deterministic? Are scripts likely to break in the future?

Most functions in Geometry Script are deterministic. Several are not, however - in particular mesh simplification and remeshing functions currently may not produce the same mesh every time. In general, it is difficult to provide hard determinism and compatibility guarantees in procedural mesh generation systems, as things that are clear bugs or performance issues can change the result mesh when they are fixed/resolved. Deterministic versions of operations may also be slower, as in some cases the most efficient parallel-processing implementation produces non-determinism. Operations like a Mesh Boolean can have a huge dependency tree of geometric operations, and any change to one of them might affect the result. So the only way to ensure deterministic compatibility is to keep the “old” version of the code around, bloating the binary size (this is what CAD software generally does to ensure compatibility between versions).

Can I animate a mesh with Geometry Scripting? Can I implement my own skinning or vertex deformation?

This is technically possible, either by fully regenerating the mesh every frame, or by (for example) using Geometry Script to update the vertex positions of a DynamicMesh every frame. However, this is not a very efficient way to implement animation, and except for very simple examples (eg say a simple primitive shapes, basic mesh booleans, etc) is unlikely to provide acceptable runtime performance. Each time the DynamicMesh is regenerated or updated, a series of relatively expensive operations occur, including generating new vertex and index buffers and uploading them to the GPU (this GPU upload is often the most expensive part). Skinning in something like a SkeletalMesh is computed directly on the GPU and so is much, much faster.

However if you don’t need to update the deformation every frame, or don’t need realtime performance (ie for experimental or research purposes), Geometry Scripting may work for you. It is possible to (eg) iterate over all mesh vertices and update their positions, even for meshes with millions of vertices.

Runtime Usage

Can I use Geometry Script in my Game / At Runtime?

Mostly Yes. Some Geometry Script functions are Editor-Only, but the majority work at Runtime. Notable things that do not work at Runtime include creating new Volume actors, creating/updating Static or Skeletal meshes, and Catmull Clark SubD.

Should I use DynamicMeshActors generated with Geometry Script in my Game?

If your meshes are static in the Game, IE you just want the convenience of procedural mesh generation for in-Editor Authoring, then the answer is probably no. DynamicMeshComponent (the component underlying DynamicMeshActor) is relatively expensive compared to StaticMeshComponent (see below). You should almost certainly “bake” any generated DynamicMeshActors into StaticMesh/Component/Actors, that’s what we did in the Lyra sample game. I have a short tutorial video on doing so here.

If your meshes are dynamic, ie they need to be dynamically generated at Runtime, or modified in-Game, then the answer is probably yes. There are various other options like ProceduralMeshComponent, the third-party RuntimeMeshComponent which is quite good, and runtime-generated StaticMesh Assets. However none of these options has an internal UDynamicMesh that can be directly edited with Geometry Script.

Will DynamicMeshComponent be as efficient as StaticMeshComponent in my Game?

No. DynamicMeshComponent uses the Dynamic Draw Path instead of the Static Draw Path, which has more per-frame rendering overhead (there is a GDC talk on YouTube about the Unreal Engine Rendering Pipeline by Marcus Wassmer which explains the Static Draw Path optimizations for Static Meshes). DynamicMeshComponent does not support instanced rendering, so mesh memory usage is generally higher. And the GPU index/vertex buffers created by a DynamicMeshComponent are not as optimized as those in a cooked StaticMesh asset (really, really not as optimized).

In addition, the DynamicMeshComponent always keeps the UDynamicMesh memory available on the CPU - a cooked StaticMesh Asset usually does not. The FDynamicMesh3 class underlying UDynamicMesh also has a minimum size and grows in fixed “chunks” of memory, so (eg) a one-triangle DynamicMesh will consume quite a bit more memory than a comparable one-triangle StaticMesh’s FMeshDescription would.

Why are all my GeneratedDynamicMeshActors disappearing in PIE or my built game ?!?

GeneratedDynamicMeshActor is an Editor-Only subclass of DynamicMeshActor, meant for in-Editor procedural mesh generation. GeneratedDynamicMeshActor’s convenient “rebuild” system is usually not appropriate in a game context, where you likely need to more precisely manage when meshes are generated/etc. I have a short tutorial video here on how to set up a DynamicMeshActor that can be generated/edited at Runtime.

Is GeometryScript function X fast enough to use in my game?

Since Geometry Script works on Meshes, which have a variable number of triangles and vertices, the only answer anyone can ever give to this question is “you will have to try it and profile”. Any non-trivial function in Geometry Script is at least linear in the number of vertices/triangles, and many are more complex. For example the Mesh Boolean node must build an AABBTree for each of the two input meshes, then do relatively expensive pairwise-triangle intersection tests (based on the AABBTree traversal, which efficiently skips most non-intersections). If you have two basic cubes, this is quite fast. If you try to subtract a cube every frame, the mesh will accumulate more and more triangles over time, and the Boolean will become increasingly expensive.

How can I save/load Dynamic Meshes at Runtime?

Unreal Engine doesn’t provide any built-in mesh load/save system at Runtime. You would have to implement this yourself in C++. There is a system called Datasmith Runtime which can load various mesh formats at Runtime but this is not part of Geometry Scripting.

Can I use Geometry Script to modify Static Meshes in my Level at Runtime?

No, this is not possible. Static Mesh Assets you created in the Editor and placed in a level are “cooked” when you create your game executable. It is not possible to update a cooked asset at Runtime, the mesh data has been converted to optimized GPU index and vertex buffers.

The function “Copy Mesh From Static Mesh” is not working at Runtime

Try checking the Output Log, you will likely see that there are warning messages about the “Allow CPU Access” flag on the Static Mesh Asset. You must enable this flag in the Editor to be able to access the StaticMesh index and vertex buffers on the CPU at Runtime. Note that this will increase memory usage for the Asset.

The Mesh I get when using “Copy Mesh From Static Mesh” at Runtime is different than the mesh I get in the Editor

In the Editor, by default CopyMeshFromStaticMesh will access the “Source” mesh, however at Runtime the Source mesh is not available. Even with the Allow CPU Access flag enabled, at Runtime only the “cooked” index and vertex buffers that will be passed to the GPU are available on the CPU. This representation of the mesh does not support “shared” or “split” UVs or normals, the mesh will be split along any UV and hard-normal seams. So what would be a closed solid cube in the Editor will become 6 disconnected rectangles in the index/vertex buffer representation. This is problematic for many mesh modeling operations. You can in many cases use the Weld Mesh Edges function to merge the mesh back together at the added seams, however this may introduce other problems, and very small triangles may have been fully discarded.

The functions “Apply Displace from Texture Map” and/or “Sample Texture2D at UV Positions” are working in the Editor but not at Runtime

Try checking the Output Log, you will likely see warning messages about Texture Compression. In their cooked representation, Texture2D Assets are generally compressed in formats that can only be efficiently decompressed on the GPU, and so are not readable in Geometry Script. The VectorDisplacementMap texture compression mode in the Texture2D Asset is effectively uncompressed RGBA8, so you must configure a Texture asset with this compression mode in the Editor for it to be readable at Runtime.

Rendering and UE Features

Does DynamicMeshComponent support Nanite, Lumen, or Mesh Distance Fields?

As of 5.1, no.

Does DynamicMeshComponent work with Runtime Virtual Texturing (RVT)?

As of 5.1, no. RVT requires usage of the Static Draw Path, and DynamicMeshComponent uses the Dynamic Draw Path

Does DynamicMeshComponent support Physics / Collision?

Yes. DynamicMeshComponent supports both Complex and Simple collision, similar to StaticMesh. The Collision settings and geometry are stored on the DynamicMeshComponent, not the DynamicMesh directly (unlike how they are stored on a StaticMesh Asset). So, to set collision geometry or change settings, you must call functions on the Component, not on the DynamicMesh.

DynamicMeshComponents don’t show up in Collision Debug Views!

This is currently not supported in UE 5.1

Does DynamicMeshComponent support LODs?

No.

Does DynamicMeshComponent support Instanced Rendering?

As of 5.1, no.

Lyra Sample Game

How does the Non-Destructive Level Design with GeometryScript-based mesh generator “Tool” objects work in Lyra?

I wrote extensive documentation for the Lyra Tool system here: https://docs.unrealengine.com/5.0/en-US/lyra-geometry-tools-in-unreal-engine/. The basic principle is that StaticMeshComponents are linked with a GeneratedDynamicMeshActor “Tool” mesh generator which still exists in the level (they are “stored” under the map). A helper Actor called the “Cold Storage” is used to maintain links between the Tool instance and it’s Components. Each Tool must be associated with a single StaticMesh Asset.

How can I migrate the Lyra Tool system to my own project?

This is somewhat complex, and the simplest route would be to migrate your content in a copy of the Lyra game. However several users have figured out how to migrate from Lyra to your own project. This YouTube tutorial by BuildGamesWithJohn is one that I believe will work, and other users have reported that this tutorial by JohnnyTriesVR will also work.

The Interactive Tools Framework in UE5

During the last few major versions of UE4, a stack of libraries was built up in the service of the new and expanding Modeling Editor Mode. This included a low-level mesh/geometry processing plugin (cleverly named GeometryProcessing), the InteractiveToolsFramework, and the MeshModelingToolset plugin. In UE5 these libraries have been significantly expanded, but also undergone some major reorganization, and and some portions have been taken out of Experimental status.

In previous Tutorials, I have covered using these libraries in-Editor to build custom Modeling Tools (https://www.gradientspace.com/tutorials/2020/1/2/libigl-in-unreal-engine), doing command-line Geometry Processing (https://www.gradientspace.com/tutorials/2020/9/21/command-line-geometry-processing-with-unreal-engine), doing Runtime Procedural Mesh Generation (https://www.gradientspace.com/tutorials/2020/10/23/runtime-mesh-generation-in-ue426 and https://www.gradientspace.com/tutorials/2020/11/11/procedural-mesh-blueprints-in-ue426), and most recently, using the Interactive Tools Framework (ITF) to build a small Runtime 3D Modeling App (https://www.gradientspace.com/tutorials/2021/01/19/the-interactive-tools-framework-in-ue426). With the changes in 5.0, all these posts and sample projects need to be updated (not a small effort!).

In this article I will describe the high-level changes to the ITF / GeometryProcessing / MeshModelingToolset stack. This will serve as a rough “porting guide” for UE4 usage of these libraries. I have also updated the Runtime Tools Framework Demo to work with UE5, the updated code project is available on Github (https://github.com/gradientspace/UE5RuntimeToolsFrameworkDemo), and I will discuss some details later in the post.

GeometryProcessing

Several major structural changes were made to the GeometryProcessing Plugin. The first and foremost is that portions of it were moved into the Engine core, to a module named GeometryCore. This was necessary to allow the core Engine and Editor to use the various Geometry algorithms, as they cannot easily have plugin dependencies. Specifically the contents of the GeometricObjects module of GeometryProcessing were moved, and that module no longer exists. So, to update your Build.cs files, generally you can just replace the “GeometricObjects” references with “GeometryCore”. Over time more GeometryProcessing functionality may migrate to GeometryCore as it becomes needed for core Engine features.

The core FDynamicMesh3 class and various associated types were also moved from the DynamicMesh module (although that module still exists and contains many processing algorithms). The paths to these files has changed, so for example where you could previously #include “DynamicMesh3.h”, you will now have to #include “DynamicMesh/DynamicMesh3.h”. A few other frequently-used utility headers like MeshNormals.h, MeshTangents.h, and MeshTransforms.h were also moved to this new DynamicMesh subfolder of GeometryCore.

Another major change is that nearly all code in GeometryCore and GeometryProcessing was moved to a namespace, UE::Geometry::. Moving code into this namespace resolved many naming conflicts with Engine types, and reduces the need to use highly verbose naming to avoid conflicts in the global namespace. This does, however, tend to mean any code written against GeometryProcessing will need some updates. Most code in the engine simply does a using namespace UE::Geometry; in any affected .cpp files, however this should never be done in a header. Generally the Engine uses explicit full names for UE::Geometry types in headers, eg in class definitions that are not also in the UE::Geometry namespace. In some cases you will also find class-scoped using declarations like using FDynamicMesh3 = UE::Geometry::FDynamicMesh3;

The GeometryProcessing plugin in UE4 was a largely self-contained set of libraries. The core GeometricObjects library even defined it’s own templated math types for Vectors/etc. This was because the core FVector type in UE4 used 32-bit float precision, and GeometryProcessing primarily uses doubles. In UE5, the rest of the Engine has caught up with GeometryProcessing, and the core FVector is now double-precision, a specialization of the UE::Math::TVector<T> template.

A major complication of this conversion was that the “short names” for the new explicit float and double core types were chosen to be FVector3f and FVector3d, exactly the global-scoped type names that had been used in GeometryProcessing for it’s templated Vector type. So, to resolve this conflict, and simplify usage of GeometryProcessing across the Engine, the GeometryProcessing FVector3<T> template was fully replaced by the new UE::Math::TVector<T>. Similar changes were made for TVector2 and a few other types. This may sound straightforward, but GeometryProcessing had used some vector-math naming and idioms common to external libaries like Eigen, that were in conflict with some of the “Unrealisms” of FVector. So, some former functions of GeometryProcessing’s FVector2/3/f/d were moved to standalone functions, to avoid duplication in the core Vector type. For example FVector3d.Normalized() no longer exists as a member function, and a free-function UE::Geometry::Normalized() must now be used. These changes required extensive (but very rote) refactoring of the entire GeometryProcessing library. As a result, most UE4 GeometryProcessing-based vector-math code is likely to not compile in UE5 without similar modifications.

Some other type conflicts were resolved by renaming the GeometryProcessing type, rather than by switching to the Engine types. In particular, the UE::Geometry variant of FTransform remains, and was renamed to FTransformSRT3f/d to resolve the name conflict and more clearly indicate it’s functionality (“SRT” indicates the Scale/Rotate/Translate transform order applied to points). In future there may be variants that could (for example) support composition of transforms with non-uniform scaling, which is not possible with the core Engine FTransform3f/d. In general, where UE::Geometry has “it’s own” version of a type, the goal is to provide a variant that is more compatible with standard math libraries and “textbook” equations, which in turns simplifies integration of those libraries by licensees, porting algorithms, etc. A prime example is the UE::Geometry::TMatrix3, which uses textbook post-multiplication ordering for matrix/matrix multiplies, vs the Engine TMatrix which uses a somewhat unusual pre-multiplication-of-transpose, that can trip up attempts to (eg) implement a formula one might find online or in a research paper.

Finally, GeometryCore and GeometryProcessing were taken out of Experimental status. What this means is that in future breaking changes will generally not be made without going through standard UE deprecation processes, ie APIs that need to be modified will be deprecated for an Engine release before being removed.

GeometryFramework

A central character in several of my previous tutorials was the USimpleDynamicMeshComponent class, which provided a renderable Component based on FDynamicMesh3. In UE4 this was primarily used by Modeling Mode, to support fast live previews of mesh editing, and could also be created and used at Runtime. In UE5, this Component has been become a fully-functional type, and was renamed to UDynamicMeshComponent. It was also moved from the ModelingComponents module of the MeshModelingToolset plugin, to a core Engine module named GeometryFramework, which now also includes an associated Actor type, ADynamicMeshActor, as well as UDynamicMesh, a UObject wrapper for a FDynamicMesh3.

UDynamicMeshComponent was significantly “cleaned up”, and some areas that were previously a bit ad-hoc, like support for Tangents, are now much cleaner. Support for both Simple and Complex collision was added, and a full UBodySetup API is included, as well as various helper functions. A key thing to note about Physics support, though, is that Async physics build is not supported, ie changes to collision geometry require a relatively slow game-thread recomputation. Async physics build has been added in the UE5 Main development stream and will land in 5.1.

FDynamicMesh3 is now serializable, and the UDynamicMesh wrapper can be added as a UProperty of any UObject. UDynamicMeshComponent now uses a UDynamicMesh to store it’s mesh, rather than directly storing a FDynamicMesh3. This means UDynamicMeshComponent is serializable, ie you can add an instance to any Actor, initialize/edit the UDynaimcMesh, and it will save/load as expected, and be included in the cooked game build.

Note, however, that UDynamicMesh is not currently an “asset type”, ie you cannot make one in the Content Browser like a UStaticMesh. Technically nothing prevents you from writing your own code to do that, as an Asset is simply a serialized UObject. However by default the UDynamicMeshComponent will create it’s own UDynamicMesh instance, and it will be serialized “with the Component”, which means it is stored “in the level”. I will cover this in depth in future Tutorials.

To avoid breaking code, the direct FDynamicMesh3 access functions of UDynamicMeshComponent still exist, such as FDynamicMesh3* GetMesh(). However, it is strongly recommended that the ProcessMesh() and EditMesh() functions be used instead. These give UDynamicMesh/Component some control over when the mesh update actually occurs, which (in future) will allow for safe access from multiple threads, ie mesh updates will not need to all be done on the game thread. These two functions also exist on UDynamicMesh, as well as other “mesh containers” like UPreviewMesh that are used heavily in MeshModelingToolset.

ADynamicMeshActor is a standard Actor type for a UDynamicMeshComponent, similar to AStaticMeshActor and UStaticMeshComponent. It is Blueprintable, and the new Geometry Script plugin can be used to do mesh generation and editing for UDynamicMeshActor/Component in Blueprints. I won’t discuss that further here, but I do have an extensive series of Youtube videos on the topic. Similarly, DynamicMeshActors can now be emitted by the mesh creation Tools in Modeling Mode, and similarly the mesh editing Tools generally all work on DynamicMeshComponents.

InteractiveToolsFramework

The Interactive Tools Framework (ITF) has become more deeply integrated into the UE Editor, while still remaining fully functional for Runtime use. Usage is no longer limited to Modeling and Pain modes, many UE Editor Modes now use the ITF to some extent. However some major refactoring has occurred to support this broader usage base. In particular some aspects of the ITF were quite specific to Modeling, and an attempt has been made to remove these aspects, or at least make them optional.

UToolTargets

Perhaps the most significant change is that the previous way that Modeling Tools interacted with “Mesh Objects” like a StaticMesh or Volume, via FPrimitiveComponentTarget, has been deprecated and replaced. FPrimitiveComponentTarget was a relatively simple wrapper around something that could provide a FMeshDescription, which was used to bootstrap the Modeling Mode, however it had major problems. In particular, it relied on a global registry, which meant that if an Engine module registered a FComponentTargetFactory, a plugin could not easily override that Factory (even at Runtime). Similarly, since the Engine does not support RTTI, it was quite cumbersome for a plugin to extend the core FPrimitiveComponentTarget API with additional functionality without making Engine changes, and then build Tools that used that functionality.

The replacement is the UToolTarget system, where a base UToolTarget class defines no functionality itself, and UInterfaces are used to add sets of API functions. The UObject system supports run-time checked type querying/casting, which allows the Tool system to then determine if a given UToolTarget supports a particular UInterface. For example the IPrimitiveComponentBackedTarget interface provides functions for accessing the Actor, Component, Transform, etc of a PrimitiveComponent, and the IMeshDescriptionProvider interface provides APIs for accessing a MeshDescription for a given ToolTarget.

To avoid the global-registry problem, UToolTargetFactory implementations for particular Component/Object types are registered with the UToolTargetManager, which lives in the UInteractiveToolsContext, adjacent to the ToolManager, GizmoManager, and so on. A given UToolTarget implementation like UStaticMeshComponentToolTarget will implement various of the APIs above, and an Editor Mode will register it’s Factory with the ToolTargetManager on setup (see UModelingToolsEditorMode::Enter() as an example).

To support “capability queries”, ie to ask the ToolTargetManager if it can build a ToolTarget for a given target Object (Actor, Component, Asset, etc) and set of required ToolTarget APIs, there is a FToolTargetTypeRequirements type. The common usage is that a ToolBuilder will have a static FToolTargetTypeRequirements enumerating the APIs it requires, which will be passed to the ToolTargetManager. An example such function is shown below for UBaseMeshProcessingTool/Builder.

const FToolTargetTypeRequirements& UBaseMeshProcessingToolBuilder::GetTargetRequirements() const
{
	static FToolTargetTypeRequirements TypeRequirements({
		UMaterialProvider::StaticClass(),
		UMeshDescriptionCommitter::StaticClass(),
		UMeshDescriptionProvider::StaticClass(),
		UPrimitiveComponentBackedTarget::StaticClass()
		});
	return TypeRequirements;
}

The UToolTarget system is highly flexible, as it does not explicitly define any base interfaces or require specific object types. A ToolTarget Interface can be defined for any UObject type, which then allows Tools to manipulate instances of that UObject - or even specific UObjects - via the published UInterfaces.

UContextObjectStore

In addition to UToolTargetManager, a very generic mechanism for passing objects down into Tools from higher levels has been added to the InteractiveToolsContext, the UContextObjectStore. This is, to be blunt, basically just a list of UObject pointers, that can be searched by class type. The basic usage is to add an object to the Store, eg ToolsContext->ContextObjectStore->AddContextObject(NewObject<SomeUType>()), and then later that UObject instance can be found by querying ToolsContex->ContextObjectStore->FindContext<SomeUType>(). Other Manager types like the ToolManager have helper functions to access the ContextObjectStore.

The purpose of the ContextObjectStore was to replace the proliferation of ToolsContext APIs that ITF implementors were required to provide. For example, the previous strategy to expose some UE Editor functionality would have been to abstract it in an interface like the IToolsContextQueriesAPI. However expanded usage of the ITF in the Editor means that more information needs to be passed from the Editor to Tools, and abstracting all those channels via APIs at the ITF level would be very complex. So, the ContextObjectStore was intended to be used to pass Editor-level API abstractions (or simply Editor-level objects and data structures directly) in a generic way, customizable for specific “Editor”/”Client” situations.

A mechanism like the ContextObjectStore can be easily abused, however. It is nothing more than a shared list of UObject pointers, effectively a global list from the PoV of Tools living inside a given ToolsContext. So, for example, any UObject instance can be added to the store, and the store will prevent that object from being garbage collected, as long as it exists. Similarly multiple objects of the same type can be added, and only the first will ever be found. Or by the same token, nothing prevents “someone else” from removing a context object you added.

If you go spelunking, you will find some places in the UE codebase where the ContextObjectStore has been used for purposes other than “Editor-level providing abstract/generic APIs to Tool-level”, and is instead used as a convenient way to pass data members around. I strongly encourage you to not treat those usages as a pattern that should be followed. Ask yourself, “would I consider passing this data by temporarily sticking it in a global void-pointer array”? If the answer is a clear no, then using the ContextObjectStore is probably not the right approach.

UModelingObjectsCreationAPI

The IToolsContextAssetAPI in the UE4 ITF is a prime example of the type API that is better done via the ContextObjectStore. That API was used by Tools to emit new Assets, which required some abstraction to permit Runtime usage of the ITF. However the ToolsContext was required to provide an IToolsContextAssetAPI implementation, even if it did not have the concept of Assets (ie, at Runtime!). And then because that API existed, many Modeling Tools emitted “Assets” even though they were actually just trying to emit “Meshes”, which limited how easily they could be adapted to different use cases.

To resolve this situation, IToolsContextAssetAPI has been removed from the ITF in UE5, and the core ITF has no concept of “emitting Assets”. Instead, a UModelingObjectsCreationAPI type has been defined in the ModelingComponents module of MeshModelingToolset. This type contains a function CreateMeshObject() which Tools can use to create new ‘Mesh Objects’, which could be a StaticMesh Asset, but also could be a Volume, DynamicMeshActor, or any other Mesh Type (eg as we will do in our Runtime demo below). UEditorModelingObjectsCreationAPI is the implementation used in Modeling Mode in the UE Editor.

The ContextObjectStore is used to provide an implementation of UModelingObjectsCreationAPI to the Modeling Tools. Primarily the Tools use a static utility function UE::Modeling::CreateMeshObject(), which finds the UModelingObjectsCreationAPI implementation in the ContextObjectStore, and uses it to create the Mesh Object. An extensive FCreateMeshObjectParams struct is used to provide mesh creation information, ie names, materials, the source MeshDescription or DynamicMesh, and so on. Similarly the function returns a FCreateMeshObjectResult that provides pointers to the new Actor, Component, and Asset, where applicable.

A similar set of functions and types is available for Texture objects, and more are likely to be added in the future. Note, however, that support for this API is completely optional - a Runtime ITF implementation would only need to provide a UModelingObjectsCreationAPI implementation if it was to use MeshModelingToolset Tools that emit new Mesh Objects.

UCombinedTransformGizmo

The UTransformGizmo developed for Modeling Mode in the UE Editor is designed to also work at Runtime, however this means it’s behavior in the UE Editor is not ideal (particularly for rendering), and it has a significantly different UX. To make way for a future Gizmo implementation, UTransformGizmo was renamed to UCombinedTransformGizmo. In addition, the concept of “default Gizmos” is in the process of being removed from the GizmoManager, and so usage of UCombinedTransformGizmo should now be done via a set of utility functions in /BaseGizmos/TransformGizmoUtil.h.

To create new 3D Gizmos from Modeling Tools and Runtime ITF code, a helper object can be automatically registered in the ContextObjectStore using the function UE::TransformGizmoUtil::RegisterTransformGizmoContextObject(), and similarly unregistered using DeregisterTransformGizmoContextObject(). Once registered, the utility functions UE::TransformGizmoUtil::Create3AxisTransformGizmo() and ::CreateCustomTransformGizmo() are available and should replace previous calls to UInteractiveGizmoManager::CreateTransformGizmo(). However Gizmos can be discarded the same way as before, using the various UInteractiveGizmoManager::DestroyGizmo() variants.

Finally, if you previously tried to use UTransformGizmo in your own projects, you will likely have run across the need to call GizmoRenderingUtil::SetGlobalFocusedSceneViewTrackingEnabled(). This was, to be blunt, a very gross hack needed to work around limitations of communicating between the game thread and render thread, because the sub-Components used in the Gizmo figure out some aspects of their rendering on the render thread. This caused no end of problems in the Editor, and so it was removed and replaced with a more structured system based on an object called the UGizmoViewContext. This object is created and added to the ContextStore (again…) by the TransformGizmoUtil registration function above. It is then necessary to update this GizmoViewContext with the active FSceneView every frame. This is generally straightforward and you can see how it is used in the sample project below, in the function URuntimeToolsFrameworkSubsystem::Tick(). But just note that UCombinedTransformGizmo instances will not function without this SceneView being set correctly.

UInteractiveToolsContext Customization

The core ITF classes - UInputRouter, UInteractiveToolManager, UInteractiveGizmoManager, UToolTargetManager, and UContextObjectStore - are fully functional on their own, however for many users of the ITF it may be desirable to customize or extend the behavior of the base classes. This was previously somewhat difficult unless you also subclassed UInteractiveToolsContext and replaced the Initialize() function, which from a maintenance perspective is not ideal, as the base implementation may be extended in the future (as it was in UE5 to add the ToolTargetManager and ContextObjectStore). To simplify customization of these base Managers, the default UInteractiveToolsContext now allows you to provide custom functions to create and destroy each of the sub-objects.

This capability is used in the Editor, to support a “hierarchy” of InteractiveToolsContexts. This is useful information if you are building in-Editor Tooling using the ITF - in addition to each Editor Mode (UEdMode) having a local InteractiveToolsContext (ITC), there is also a ToolsContext that lives at the “ModeManager” level. This ITC is considered a “parent” ITC, and (for example) the InputRouter is shared between the Parent and any active Mode ITCs, to enforce “one capture at a time” behavior. You may find having a hierarchy of ToolsContexts helpful if you are building a complex at-Runtime DCC Tool. If you want to browse the Editor code for this, see the base class UEditorInteractiveToolsContext and subclasses UModeManagerInteractiveToolsContext and UEdModeInteractiveToolsContext.

UAssetEditors now also support UEdModes, and hence have a UInteractiveToolsContext in each Mode by default. This is not heavily used in existing Asset Editors, however the new UV Editor (a companion to Modeling Mode) is an example of an Asset Editor built primarily using the ITF. A variant of Modeling Mode has also been integrated into the Static Mesh Editor, although this is not enabled by default, and only exposes a few Tools there. The plugin is called Static Mesh Editor Modeling Mode, and can be found in the source tree in \Plugins\Experimental\StaticMeshEditorModeling\. This plugin is fully self-contained, ie no Engine changes are needed for a plugin to “add itself” to the StaticMeshEditor.

MeshModelingToolset / Exp

The MeshModelingToolset plugin has been moved out of Experimental status, however portions of the plugin and modules that needed to remain Experimental were then moved to a “MeshModelingToolsetExp” plugin. The only real effect of this in terms of porting projects is that your .uplugin and .build.cs files may need to be updated to add “MeshModelingToolsExp” and/or “MeshModelingToolsEditorOnlyExp”.

Runtime Tools Framework Sample Project

A port of the UE4 Sample Project to UE5 is available on github here: https://github.com/gradientspace/UE5RuntimeToolsFrameworkDemo. I made this project by forking the UE4 project, and then submitted the port in a single commit. So, if you are interested in what specifically had to be updated, or you need to perform a similar upgrade on your own project based on my sample, you can browse the diff here. I will give a high-level overview of the changes below

RuntimeGeometryUtils Plugin

This plugin was used in several of my other sample projects, it’s not really critical to this sample, but it provides the OBJ import and the ADynamicSDMCActor. Most of the changes are simply updating paths, dealing with the new UE::Geometry namespace, and some minor API and function changes. The URuntimeDynamicMeshComponent class was removed as it is no longer needed - it just added collision support to USimpleDynamicMeshComponent, but the replacement UDynamicMeshComponent in UE5 now includes full simple and complex collision support. To minimize changes involved in the port I left UGeneratedMesh intact, however the new engine UDynamicMesh class is a superior replacement, I hope to do that change in the future (and similarly for ADynamicSDMCActor, replacing with the Engine’s ADynamicMeshActor).

RuntimeToolsSystem Module

The RuntimeToolsSystem Module is the main code of the sample, and implements the “Tools Framework Back-End” for Runtime. This is where most of the changes have taken place.

As discussed above, the FPrimitiveComponentTarget system was removed in favor of the new UToolTarget system. So, URuntimeDynamicMeshComponentToolTarget has been deleted, and a new URuntimeDynamicMeshComponentToolTarget and URuntimeDynamicMeshComponentToolTargetFactory added in it’s place. This new ToolTarget factory is registered in URuntimeToolsFrameworkSubsystem::InitializeToolsContext().

To support creation of new meshes by the various Tools, a new URuntimeModelingObjectsCreationAPI implementation of UModelingObjectsCreationAPI was added. The ::CreateMeshObject() function spawns a new URuntimeMeshSceneObject (which is a wrapper for a mesh Actor/Component) via the URuntimeMeshSceneSubsystem. An instance of this API implementation is similarly registered in URuntimeToolsFrameworkSubsystem::InitializeToolsContext(). As it was no longer needed, the FRuntimeToolsContextAssetImpl implementation of IToolsContextAssetAPI was also removed (and that API no longer exists).

The above changes allow some of the Modeling Tool subclasses in the /Tools/ subfolder to be simplified. In particular, several Tools had to override base-class functions to handle creating new objects, because the previous AssetAPI-based versions could not be hacked to function correctly. With the new UModelingObjectsCreationAPI, the Tool-creates-new-Mesh-object flow is much cleaner and no longer required any customization at the Tool level.

Finally several small changes were needed to update support for the 3D Transform (TRS) Gizmo. First, The function UE::TransformGizmoUtil::RegisterTransformGizmoContextObject() must be called to register an instance of the UCombinedTransformGizmoContextObject in the ContextObjectStore, this is done in URuntimeToolsFrameworkSubsystem::InitializeToolsContext(). This object registers the various sub-gizmos with the GizmoManager, and provides a wrapper that can spawn instances of the new UCombinedTransformGizmo. In future this will not be directly possible via the GizmoManager (it still is in 5.0, for legacy reasons, but this is due to change). So, the next step is to update USceneObjectTransformInteraction to call UE::TransformGizmoUtil::CreateCustomTransformGizmo() instead of talking to the GizmoManager directly.

Finally, there had previously been calls in the AToolsContextActor to call GizmoRenderingUtil::SetGlobalFocusedSceneViewTrackingEnabled(). This was, frankly, a gross hack, that set global pointers which allowed the Gizmo to communicate information based on the FSceneView between the Game and Render threads. In 5.0 this is no longer necessary. Instead, in URuntimeToolsFrameworkSubsystem::Tick(), an instance of UGizmoViewContext is fetched from the ContextObjectStore, and passed the current FSceneView. This is all that is necessary to provide the Gizmo with correct camera information. The UGizmoViewContext is automatically created and configured by the GizmoRenderingUtil registration function that was called above.

And that’s it! Those are the major changes that were necessary.

If you run into problems, or have questions, please don’t hesitate to find me on twitter ( https://twitter.com/rms80 ) or on the Epic Dev Community ( https://forums.unrealengine.com/u/rmseg ).

Modeling Mode Extension Plugins in UE5

In Unreal Engine 5.0, Modeling Mode includes a large set of Tools, built using the Interactive Tools Framework, which allow users to create and edit meshes in the Level Viewport. However, something else that came with UE5.0 is the ability for third-party plugins to add additional Tools into Modeling Mode. In this tutorial I will explain how that works, and provide sample code that adds a small set of custom Tools.

I will not go deeply into the mechanics of implementing new Interactive Tools in this article. I have discussed this quite a bit in some previous posts, such as The Interactive Tools Framework and Interactive Mesh Processing with libigl. At time of writing, those Tutorials have not been updated for UE5, and some changes to the Interactive Tools Framework (ITF) APIs may have broken that older Tool code (updates to those posts are in progress and I will update this article when they are done). But the high-level concepts/etc are still applicable, and I have included a few basic UE5.0-style Tools with this article’s sample code.

The Tools will appear in their own section of the Modeling Mode tool palette, as shown in the image to the right - I called the extension “Extension Demo”, and there are 4 Tools - Noise, Cut, ClickBP, and MeshEdBP. Note that I set the color of this section using the Modeling Mode palette customization features, which I demonstrated in this YouTube Video. The Palette customization is completely separate from the Extension mechanism, but is fully functional with Extensions (so, for example, I could add an Extension Tool to my Favorites section).

(Mandatory Disclaimer: your author, Ryan Schmidt, is an employee of Epic Games. However, gradientspace.com is his personal website and this article represents his personal thoughts and opinions. This document and sample code are not supported by Epic Games!)

IModularFeature and the IModelingModeToolExtension API

Modular Features are a mechanism for core Unreal Engine and Editor code to support features and capabilities defined in plugins. Essentially, the way it works is that UE publishes standard interfaces, and then plugins can implement that interface and register themselves as a provider of an implementation. Various standard Editor/Engine features are implemented this way, but it also allows for third-party developers to provide their own implementation(s).

The core of the system is the IModularFeature interface, which doesn’t actually contain any functions, it’s just the magic base-class that makes the system work. If you search the codebase for “public IModularFeature” you will find many subclasses of this base class, all of these are different Modular Feature APIs that the engine supports in some way (I found 96 in UE5.0). In our case, we will be using IModelingModeToolExtension, which is defined in Engine\Plugins\Editor\ModelingToolsEditorMode\Source\ModelingToolsEditorMode\Public\ModelingModeToolExtensions.h. This is an API that Modeling Editor Mode uses to interact with the available Tool Extensions. Essentially, Modeling Mode will ask each registered extension “What Tools do you have for me?” and then it will add sections and buttons to the Modeling Mode Tool Palette to expose those Tools to the user. It is really that simple.

The IModelingModeToolExtension API has 3 functions, listed below (minus some boilerplate)

class IModelingModeToolExtension
{
  virtual FText GetExtensionName();
  virtual FText GetToolSectionName();
  virtual void GetExtensionTools(const FExtensionToolQueryInfo& QueryInfo, TArray<FExtensionToolDescription>& ToolsOut)
};

GetExtensionName() returns an identifier string that needs to be unique for your Extension. GetToolSectionName() returns the string that will be used in the Tool Palette. And GetExtensionTools() returns a list of FExtensionToolDescription objects, one per Tool, that Modeling Mode will use to create the Tool Palette button and launch the Tool on-click. This struct is shown below, again basically we have an identifier name, Command info for the Button (which includes the Icon), and then the ToolBuilder that Modeling Mode will use to create an instance of the Tool.

struct FExtensionToolDescription
{
	FText ToolName;                         // long name of the Tool
	TSharedPtr<FUICommandInfo> ToolCommand; // Command that is launched by the Palette
	UInteractiveToolBuilder* ToolBuilder;   // Tool Builder that will be called to create the Tool
};

The FExtensionToolQueryInfo struct passed to GetExtensionTools() by the Mode is for optional/advanced usage, we will skip it for now, and I’ll cover it below in the Extra Bits section.

So, basically, the way you use IModelingModeToolExtension is as follows:

1. Write some UInteractiveTool implementations, these are your Tools

2. Create a new Plugin, and subclass/implement IModelingModeToolExtension. Normally this is done in the same class as the IModuleInterface implementation, ie the class that contains the StartupModule()/ShutdownModule() functions (Note that your Tools do not have to be in this plugin/module, but they can be)

3. In the GetExtensionTools() function, return a list of your available Tools with a ToolBuilder for each

4. In the StartupModule() function, call IModularFeatures::Get().RegisterModularFeature(IModelingModeToolExtension::GetModularFeatureName(), this); (and similarly UnregisterModularFeature in ShutdownModule)

Step 4 is the critical one, basically this is how your Plugin/Module tells the Modular Feature system “Hey I implement this named Interface!”, where the name is provided by IModelingModeToolExtension::GetModularFeatureName(). Modeling Mode will (later) ask the Modular Feature system “Hey does anyone implement this named interface?”, and the Modular Feature system will say “Yes! This thing does!” and provide your implementation.

From the Point of View of the Extension Writer, this is all there is to it. There is a bunch of boilerplate in setting up the TCommands and Style objects for your plugin, which you can copy-paste from my sample, but the actual Modeling Mode Extension part only takes these few steps listed above.

The UE5ToolPluginDemo Sample Project

You can grab the UE5ModelingModeExtensionDemo project from the Gradientspace Github at https://github.com/gradientspace/UE5ModelingModeExtensionDemo. I have included the entire project, however this is just a generated standard UE5 Blank project template. All the relevant code for the sample is in \Plugins\SampleModelingModeExtension, which you can easily copy-paste to any other Project, as it has no dependencies on, or references to, the base Project.

There are 3 boilerplate/setup classes in the SampleModelingModeExtension plugin, lets briefly go through them:

FSampleModelingModeExtensionModule

This class is the main class for the Module, which implements IModuleInterface (ie StartupModule() / ShutdownModule() ) and our IModelingModeToolExtension Modular Feature API, discussed above. The cpp contains the bare minimum lines to register the Style, Commands, and Modular Feature, and provide the list of Tool description structs.

FSampleModelingModeExtensionStyle

The Style class is used to provide icons for the Tool Palette. If you had (for example) registered details customizations for property sets of the Tools, you might put additional things in this Style, but otherwise, it’s 100% copy-pastable boilerplate except the lines that look like this:

StyleSet->Set("SampleModelingModeExtensionCommands.BeginMeshNoiseTool", new IMAGE_BRUSH_SVG("Icons/NoiseTool", DefaultIconSize));

Note that the strings here are important. The SampleModelingModeExtensionCommands string is not the class name, but rather the string passed to the TCommands constructor in the FSampleModelingModeExtensionCommands class. I just tend to use the same string everywhere. The BeginMeshNoiseTool string is the name of the FUICommandInfo member variable of the FSampleModelingModeExtensionCommands, passed to the UI_COMMAND macro, and not (eg) the name of the Tool Icon or identifier string.

FSampleModelingModeExtensionCommands

This class is a subclass of TCommands, which uses the Curiously Recurring Template Pattern (CRTP - Wikipedia). Basically we just use this to provide UI Command objects that can be assigned to Tool Palette buttons. The UI Command system will provide the icon (via the Style), and the FExtensionToolDescription struct will let Modeling Mode link up the Command to the ToolBuilder you specify. So, again, this is really just lines of boilerplate that look like the following

UI_COMMAND(BeginMeshNoiseTool, "Noise", "Add Noise to selected Mesh", EUserInterfaceActionType::ToggleButton, FInputChord());

Where you will have one line for each Tool your plugin provides.

The Sample Tools

I have included 4 basic Tools in the sample project. UMeshNoiseTool is a basic tessellate-and-displace Tool, and UMeshPlaneCutTool shows how to create a 3D gizmo inside a Tool, and then use it to do a plane cut of the active mesh. In both cases the Tool subclasses the UBaseMeshProcessingTool base tool. This base tool provides very useful functionality for editing single meshes - in particular, it automatically computes the mesh operation asynchronously in a background thread, with support for cancelling. This is done via a FDynamicMeshOperator, you will find the simple Noise and PlaneCut operators in the .cpp files.

The Noise and PlaneCut tools are basic versions of the Displace and PlaneCut Tools in Modeling Mode. Many Tools in Modeling Mode are also derived from UBaseMeshProcessingTool (although for example the Modeling Mode Plane Cut Tool support operating on multiple objects, which adds a lot of complexity). However the other two Tools in this sample are unique in that they use a Blueprint to define their behavior (!)

UActorClickedBPTool

View fullsize

Users of modeling tools each have their own unique workflows, and this often results in feature requests for Tools that are extremely specific to a particular task or workflow. However, adding a new Tool tends to increase the cognitive load for all users, and at some point, a Tool is just “too niche” to add to the UI by default. In UE5.0, I added Geometry Script to allow end-users to create their own Tools in UE Blueprints. If you aren’t familiar with Geometry Script, I suggest you check out this playlist of short videos I have posted on YouTube. Using Geometry Script and Editor Utility Widgets, it is possible to make something that is similar to a Modeling Mode Tool, like I did in this video where I made a “convex hull wrapper” mesh generation Tool. However, this approach is a bit clunky compared to a Tool integrated into Modeling Mode.

So, UActorClickedBPTool is a Modeling Mode Tool that doesn’t have any built in behavior, and instead just executes an arbitrary function - defined in a Blueprint - on whatever Actor is clicked while using the Tool. You provide the BP function by subclassing the UActorClickedBPToolOperation object type, which has a function OnApplyActionToActor that can be overridden in the BP. When configured in the Tool, this function will be called on Actor click.

As a simple example, I implemented a BP Operation that flips the Normals of clicked StaticMeshActors. You will find the BP_FlipNormals blueprint operation in the /ActorClickBP Content folder. The entire Blueprint is shown below (click to enlarge the image). The gif above-right is a brief demo of using the Tool, assigning the FlipNormals BP, and then clicking on two meshes in the level a few times.

View fullsize

UMeshProcessingBPTool

After my success with the ActorClickedBPTool, I decided to try to make a variant of UBaseMeshProcessingTool where the mesh processing operation could be implemented in BP using Geometry Script. This is in theory no more complex than UActorClickedBPTool, however to avoid writing a lot of boilerplate code, I decided to directly subclass UBaseMeshProcessingTool, and just wrap the BP execution in a FDynamicMeshOperator like in the Noise and Plane Cut Tools. This basically works, and UMeshProcessingBPTool has the same structure as the ActorClickedBPTool, but in this case UMeshProcessingBPToolOperation is the base UObject class for the mesh processing operation, and BP subclasses implement the OnRecomputeMesh function which takes a UDynamicMesh as input/output.

I have included two MeshProcessingBPToolOperation examples in the /MeshEditBP content subfolder. BP_Noise is shown in the video below, it implements a simple PN-Tessellation and Perlin-Noise displacement operation. This is basically the exact same Geometry Script setup as I have demonstrated in other videos, just running in the context of an Interactive Tool. BP_Boxes is a simple blueprint that appends random cubes to the input mesh, not particularly useful, I just wanted a second example.

Now, there is a subtle complication with this setup. You may recall that earlier I mentioned that UBaseMeshProcessingTool automatically evaluates it’s FDynamicMeshOperators in background threads. However, it is generally not safe to access UObjects in background threads, and so (in general) Blueprints also cannot be executed asynchronously. A conundrum! To work around this, in the MeshProcessingBPTool I have added a bit of a hack, where by default the Operator will (via a helper class) force the BP execution to occur on the Game thread (generally making the background thread computation useless)

However, I have found that in fact it is not catastrophic to evaluate the Mesh Processing BP operation from the background thread, as long as it does not touch any Actors/Assets/Components/Subsystems except the input UDynamicMesh. This is, as far as I am aware, 100% off-book, unsupported usage of Unreal Engine, and may not continue to work in the future. But it does seem to work now, in 5.0, and allows the Mesh Processing BP to be executed asynchronously. To enable this, the UMeshProcessingBPToolOperation BP base class has a second function, GetEnableBackgroundExecution, that you can override in the BP to return true (instead of default false).

UMeshProcessingBPTool Parameters

One caveat with the MeshProcessingBPTool (and the UActorClickedBPTool) is that there is currently no way for the Blueprint to expose parameters in the Modeling Mode Tool Settings panel without making some C++ modifications. The way I have done it in the sample is to make a small UStruct, FMeshProcessingBPToolParameters, which contains 2 float parameters and 2 integer parameters. An instance of this struct is included in the UMeshProcessingBPToolProperties, which results in it being included in the Tool Settings details panel. A copy of this struct is passed to the UMeshProcessingBPToolOperation::OnRecomputeMesh() function, allowing the UI to be used to control settings in the mesh processing BP. However, of course if you need more parameters or need to do any customization/etc, you will have to edit the C++ struct. This is not difficult, but it’s also not convenient.

The challenge is that only UInteractiveToolPropertySet subclasses can be passed by the Tool to the UI level. This type is currently not extendable in Blueprint, although this could be added in a C++ subclass. One way to possibly approach this problem would be to have the BP Operation subclass provide a set of such Blueprint-defined-Property-Sets. However the Tool would have to query the BP and manage the resulting Property Set instances, and…something…would have to copy values back-and-forth. It seems a bit tricky but perhaps a motivated reader will figure out a solution!

Adding More Tools

To add a new Tool, there are just a few steps

1. Create your Tool implementation (ie MyNewTool.h and .cpp)

2. Add a new FUICommandInfo to FSampleModelingModeExtensionCommands, in the header and cpp. This will be used to launch your Tool. The UI_COMMAND macro defines the icon label and tooltip

3. Add a new icon in the StyleSet in FSampleModelingModeExtensionStyle::Initialize(), as described above, with the label matching the Command from step 2

4. Add a FExtensionToolDescription struct for your Tool in FSampleModelingModeExtensionModule::GetExtensionTools(), using the right Command and ToolBuilder for your new Tool. You will likely also have to include your MyNewTool.h header (to access the ToolBuilder)

That’s it! When you build and run, your new Tool should be included in the Modeling Mode tool palette.

Miscellany

One last thing to mention is the FExtensionToolQueryInfo struct passed to the IModelingModeToolExtension::GetExtensionTools() function. This data structure is used to provide Modeling Mode’s UInteractiveToolsContext to the Extension, as some Extensions may need access to the Context. In addition, when Extensions are initially being queried for the Tool Palette setup, the Tool Builders are not actually required yet, and creating them can be skipped (in some cases this might be necessary or at least more performant). The FExtensionToolQueryInfo.bIsInfoQueryOnly flag will tell your Extension about this info. But this is optional/advanced usage and probably not something to worry about!

I also just wanted to mention that this Modeling Mode Extension system is not just for third parties, it is also used to add optional toolsets in the UE codebase. In particular the HairModelingToolset plugin adds a small set of tools for working with Groom Assets. These Tools were formerly included directly in Modeling Mode, however by moving them to a separate, optional plugin, they can be left disabled by default, and only enabled for users who need Groom Asset Editing. This is precisely the kind of niche toolset that Modeling Mode Extensions are intended to support.

Conclusions

Modeling Mode Extensions allow third parties to greatly increase the power of Modeling Mode in UE5. Even if you don’t want to write your own Tools from scratch, most existing Modeling Tools can be subclassed and customized to make them work better for specific use cases, and those custom versions exposed via an Extension. I am interested in hearing from readers about what kinds of “Base Tools” they might find useful to build on (particular in the case of Tools that can be customized with Blueprints). For example, I think a “Base BP Tool” that supports a generic brush-style interaction, calling a BP for each brush stamp, could be very interesting. Maybe something to tackle in a future post!

Similarly, if you find yourself trying to write a Tool Extension and hitting a wall, please don’t hesitate to post in the comments or get in touch on twitter @rms80.

The Interactive Tools Framework in UE4.26 (at Runtime!)

In this article, I am going to cover a lot of ground. I apologize in advance for the length. However, the topic of this article is essentially “How to Build 3D Tools using Unreal Engine”, which is a big one. By the end of this article, I will have introduced the Interactive Tools Framework, a system in Unreal Engine 4.26 that makes it relatively straightforward to build many types of interactive 3D Tools. I’m going to focus on usage of this Framework “at Runtime”, ie in a built Game. However, this exact same Framework is what we use to build the 3D Modeling Tools suite in the Unreal Editor. And, many of those Tools, are directly usable at Runtime! Sculpting in your Game! It’s pretty cool.

There is a short video of the ToolsFramworkDemo app to the right, and are a few screenshots below - this is a built executable, not running in the UE Editor (although that works, too). The demo allows you to create a set of meshes, which can be selected by clicking (multiselect supported with shift-click/ctrl-click), and a 3D transform gizmo is shown for the active selection. A small set of UI buttons on the left are used to do various things. The Add Bunny button will import and append a bunny mesh, and Undo and Redo do what you might expect. The World button toggles the Gizmo between World and Local coordinate systems.

The rest of the buttons launch various Modeling Tools, which are the exact same tool implementations as are used in Modeling Mode in the UE 4.26 Editor. PolyExtrude is the Draw Polygon Tool, in which you draw a closed polygon on a 3D workplane (which can be repositioned by ctrl-clicking) and then interactively set the extrusion height. PolyRevolve allows you to draw an open or closed path on a 3D workplane - double-click or close the path to end - and then edit the resulting surface of revolution. Edit Polygons is the PolyEdit tool from the Editor, here you can select faces/edges/vertices and move them with a 3D gizmo (note that the various PolyEdit sub-operations, like Extrude and Inset, are not exposed in the UI, but would work if they were). Plane Cut cuts the mesh with a workplane and Boolean does a mesh boolean (requires two selected objects). Remesh retriangulates the mesh (unfortunately I couldn’t easily display the mesh wireframe). Vertex Sculpt allows you to do basic 3D sculpting of vertex positions, and DynaSculpt does adaptive-topology sculpting, this is what I’ve shown being applied to the Bunny in the screenshot. Finally the Accept and Cancel buttons either Apply or Discard the current Tool result (which is just a preview) - I’ll explain this further below.

19/06/22 - This article is now somewhat out-of-date, and the sample project is broken in UE5. I have published a working port of the sample project to UE5 here: https://github.com/gradientspace/UE5RuntimeToolsFrameworkDemo, and an article about what has changed here: https://www.gradientspace.com/tutorials/2022/6/1/the-interactive-tools-framework-in-ue5 . If you are just interested in what changed in the code, the port was done in a single commit so you can browse the diffs.

All This geometry was created in the demo. Window is selected and being rotated with gizmo.

oh no bunny is growing some new parts

This is not a fully functional 3D Modeling tool, it’s just a basic demo. For one, there is no saving or export of any kind (wouldn’t be hard to add a quick OBJ export, though!). Support for assigning Materials is non-existent, the Materials you see are hardcoded or automatically used by the Tools (eg flat shading in the Dynamic Mesh Sculpting). Again, a motivated C++ developer could add things like that relatively easily. The 2D user interface is an extremely basic UMG user interface. I’m assuming that’s throw-away, and you would build your own UI. Then again, if you wanted to do a very simple domain-specific modeling tool, like say a 3D sculpting tool for cleaning up medical scans, you might be able to get away with this UI after a bit of spit-and-polish.

(Mandatory Disclaimer: your author, Ryan Schmidt, is an employee of Epic Games. However, gradientspace.com is his personal website and this article represents his personal thoughts and opinions. About triangles.)

Getting and Running The Sample Project

Before we begin, this tutorial is for UE 4.26, which you can install from the Epic Games Launcher. The project for this tutorial is on Github in the gradientspace UnrealRuntimeToolsFrameworkDemo repository (MIT License). Currently this project will only work on Windows as it depends on the MeshModelingToolset engine plugin, which is currently Windows-only. Getting that plugin to work on OSX/Linux would mainly be a matter of selective deleting, but it would require an Engine source build, and that’s beyond the scope of this tutorial.

Once you are in the top-level folder, right-click on ToolsFrameworkDemo.uproject in Windows Explorer and select Generate Visual Studio project files from the context menu. This will generate ToolsFrameworkDemo.sln, which you can use to open Visual Studio. You can also open the .uproject directly in the Editor (it will ask to compile), but you may want to refer to the C++ code to really understand what is going on in this project.

Build the solution and start (press F5) and the Editor should open into the sample map. You can test the project in PIE using the large Play button in the main toolbar, or click the Launch button to build a cooked executable. This will take a few minutes, after which the built game will pop up in a separate window. You can hit escape to exit full-screen, if it starts up that way (I think it’s the default). In full-screen, you’ll have to press Alt+F4 to exit as there is no menu/UI.

Overview

This article is so long it needs a table of contents. Here is what I am going to cover:

First, I am going to explain some background on the Interactive Tools Framework (ITF) as a concept. Where it came from, and what problem it is trying to solve. Feel free to skip this author-on-his-soapbox section, as the rest of the article does not depend on it in any way.

Next I will explain the major pieces of the UE4 Interactive Tools Framework. We will begin with Tools, ToolBuilders, and the ToolManager, and talk about Tool Life Cycles, the Accept/Cancel Model, and Base Tools. Input handling will be covered in The Input Behavior System, Tool settings stored via Tool Property Sets, and Tool Actions.

Next I will explain the Gizmos system, for implementing in-viewport 3D widgets, focusing on the Standard UTransformGizmo which is shown in the clips/images above.

At the highest level of the ITF, we have the Tools Context and ToolContext APIs, I’ll go into some detail on the 4 different APIs that a client of the ITF needs to implement - IToolsContextQueriesAPI, IToolsContextTransactionsAPI, IToolsContextRenderAPI, and IToolsContextAssetAPI. Then we’ll cover a few details specific to mesh editing Tools, in particular Actor/Component Selections, FPrimitiveComponentTargets, and FComponentTargetFactory.

Everything up to this point will be about the ITF modules that ship with UE4.26. To use the ITF at Runtime, we will create our own Runtime Tools Framework Back-End, which includes a rudimentary 3D scene of selectable mesh “scene objects”, a pretty standard 3D-app transform gizmo system, and implementations of the ToolsContext APIs I mentioned above that are compatible with this runtime scene system. This section is basically explaining the extra bits we have to add to the ITF to use it at Runtime, so you’ll need to read the previous sections to really understand it.

Next I’ll cover some material specific to the demo, including ToolsFrameworkDemo Project Setup that was necessary to get the demo to work, RuntimeGeometryUtils Updates, in particular collision support for USimpleDynamicMeshComponent, and then some notes on Using Modeling Mode Tools at Runtime, because this generally requires a bit of glue code to make the existing mesh editing Tools be functional in a game context.

And that’s it! Let’s begin…

Interactive Tools Framework - The Why

I don’t love the idea of starting an article about something by justifying it’s existence. But, I think I need to. I have spent many years - basically my entire career - building 3D Creation/Editing Tools. My first system was ShapeShop (which hasn’t been updated since 2008 but still works - a testament to Windows backwards compatibility!). I also built Meshmixer, which became an Autodesk product downloaded millions of times, and is widely used to this day. I am continually amazed to discover, via twitter search, what people are doing with Meshmixer (a lot of digital dentistry!!). I’ve also built other fully-functional systems that never saw the light of day, like this 3D Perspective Sketching interface we called Hand Drawn Worlds I built at Autodesk Research. After that, I helped to build some medical 3D design tools like the Archform dental aligner planning app and the NiaFit lower-leg prosthetic socket design tool (in VR!). Oh and Cotangent, which sadly I abandoned before it had any hope of catching on.

Self-congratulation aside, what I have learned over the last 15-odd years of making these 3D tools is that it is incredibly easy to make a giant mess. I started working on what became Meshmixer because Shapeshop had reached a point where it was just impossible to add anything to it. However, there were parts of Shapeshop that formed a very early “Tool Framework”, which I extracted and used as the basis for various other projects, and even bits of Meshmixer (which also ultimately became very brittle!). The code is still on my website. When I left Autodesk, I returned to this problem, of How To Build Tools, and created the frame3Sharp library which made it (relatively) easy to build at-Runtime 3D tools in a C# Game Engine. This framework grew around the Archform, NiaFit, and Cotangent apps mentioned above, and powers them to this day. But, then I joined Epic, and started over in C++!

So, that’s the origin story of the UE4 Interactive Tools Framework. Using this Framework, a small team (6-or-fewer people, depending on the month) has built Modeling Mode in UE4, which has over 50 “Tools”. Some are quite simple, like a Tool to Duplicate a thing with options, and some are extremely complex, like an entire 3D Sculpting Tool. But the critical point is, the Tools code is relatively clean and largely independent - nearly all of the Tools are a single self-contained cpp/h pair. Not independent by cutting-and-pasting, but independent in that, as much as possible, we have moved “standard” Tool functionality that would otherwise have to be duplicated, into the Framework.

Lets Talk About Frameworks

One challenge I have in explaining the Interactive Tools Framework is that I don’t have a point of reference to compare it to. Most 3D Content Creation tools have some level of “Tool Framework” in their codebase, but unless you have tried to add a feature to Blender, you probably have never interacted with these things. So, I can’t try to explain by analogy. And those tools don’t really try very hard to provide their analogous proto-frameworks as capital-F Frameworks. So it’s hard to get a handle on. (PS: If you think you know of a similar Framework, please get in touch and tell me!)

Frameworks are very common, though, in other kinds of Application Development. For example, if you want to build a Web App, or Mobile App, you are almost certainly going to be using a well-defined Framework like Angular or React or whatever is popular this month (there are literally hundreds). These Frameworks tend to mix low-level aspects like ‘Widgets’ with higher-level concepts like Views. I’m focusing on the Views here, because the vast majority of these Frameworks are based around the notion of Views. Generally the premise is that you have Data, and you want to put that data in Views, with some amount of UI that allows the user to explore and manipulate that Data. There’s even a standard term for it, “Model-View-Controller” architecture. The XCode Interface Builder is the best example I know of this, where you literally are storyboarding the Views that the user will see, and defining the App Behavior via transitions between these Views. Every phone app I use on a regular basis works this way.

Stepping up a level in complexity, we have Applications like, say, Microsoft Word or Keynote, which are quite different from a View-based Application. In these apps the user spends the majority of their time in a single View, and is directly manipulating Content rather than abstractly interacting with Data. But the majority of the manipulation is in the form of Commands, like deleting text, or editing Properties. For example in Word when I’m not typing my letters, I’m usually either moving my mouse to a command button so I can click on it - a discrete action - or opening dialog boxes and changing properties. What I don’t do is spend a lot of time using continuous mouse input (drag-and-drop and selection are notable exceptions).

Now consider a Content Creation Application like Photoshop or Blender. Again, as a user you spend the majority of your time in a standardized View, and you are directly manipulating Content rather than Data. There are still vast numbers of Commands and Dialogs with Properties. But many users of these apps - particularly in Creative contexts - also spend a huge amount of time very carefully moving the mouse while they hold down one of the buttons. Further, while they are doing this, the Application is usually in a particular Mode where the mouse-movement (often combined with modifier hotkeys) is being captured and interpreted in a Mode-specific way. The Mode allows the Application to disambiguate between the vast number ways that that the mouse-movement-with-button-held-down action could be interpreted, essentially to direct the captured mouse input to the right place. This is fundamentally different than a Command, which is generally Modeless, as well as Stateless in terms of in the Input Device.

In addition to Modes, a hallmark of Content Creation Applications are what I will refer to as Gizmos, which are additional transient interactive visual elements that are not part of the Content, but provide a (semi-Modeless) way to manipulate the Content. For example, small boxes or chevrons at the corners of a rectangle that can be click-dragged to resize the rectangle would be a standard example of a Gizmo. These are often called Widgets, but I think it’s confusing to use this term because of the overlap with button-and-menu Widgets, so I’ll use Gizmos.

So, now I can start to hint at what the Interactive Tool Framework is for. At the most basic level, it provides a systematic way to implement Modal States that Capture and Respond to User Input, which I’m going to call Interactive Tools or Tools for brevity, as well as for implementing Gizmos (which I will posit are essentially spatially-localized context-sensitive Modes, but we can save that discussion for Twitter).

Why Do I Need a Framework For This?

This is a question I have been asked many times, mainly by people who have not tried to build a complex Tool-based Application. The short answer is, to reduce (but sadly not eliminate) the chance that you will create an unholy disaster. But I’ll do a long one, too.

An important thing to understand about Tool-based applications is that as soon as you give users the option to use the Tools in any order, they will, and this will make everything much more complicated. In a View-based Application, the user is generally “On Rails”, in that the Application allows for doing X after Y but not before. When I start up the Twitter app, I can’t just jump directly to everything - I have to go through sequences of Views. This allows the developers of the Application to make vast assumptions about Application State. In particular, although Views might manipulate the same underlying DataModel (nearly always some form of database), I never have to worry about disambiguating a tap in one View from a tap in another. In some sense the Views are the Modes, and in the context of a particular View, there are generally only Commands, and not Tools.

As a result, in a View-based Application it is very easy to talk about Workflows. People creating View-based Applications tend to draw lots of diagrams that look like this:

These diagrams might be the Views themselves, but more often they are the steps a User would take through the Application - a User Story if you will. They are not always strictly linear, there can be branches and loops (a Google Image Search for Workflow has lots of more complex examples). But there are always well-defined entry and exit points. The User starts with a Task, and finishes with that Task completed, by way of the Workflow. It is then very natural to design an Application that provides the Workflow where the User can complete the Task. We can talk about Progress through the Workflow in a meaningful way, and the associated Data and Application State also make a kind of Progress. As additional Tasks are added, the job of the development team is to come up with a design that allows these necessary Workflows to be efficiently accomplished.

The fundamental complication in Content Creation/Editing Applications is that this methodology doesn’t apply to them at all. Ultimately the difference, I think, is that there is no inherent notion of Progress in a Content Creation/Editing Tool. For example, as a Powerpoint user, I can (and do!) spend hours re-organizing my slides, tweaking the image size and alignment, slightly adjusting text. In my mind I might have some nebulous notion of Progress, but this is not encoded in the Application. My Task is outside the Application. And without a clear Task or measure of Progress, there is no Workflow!

I think a more useful mental model for Content Creation/Editing Applications is like the image on the right. The green central hub the default state in these Applications, where generally you are just viewing your Content. For example Panning and Zooming your Image in Photoshop, or navigating around your 3D Scene in Blender. This is where the user spends a significant percentage of their time. The Blue spokes are the Tools. I go to a Tool for a while, but I always return to the Hub.

So if I were to track my state over time, it would be a winding path in and out of the default Hub, through untold numbers of Tools. There is no well-defined Order, as a user I am generally free to use the Tools in any Order I see fit. In a microcosm, we might be able to find small well-defined Workflows to analyze and optimize, but at the Application level, the Workflows are effectively infinite.

It might seem relatively obvious that the architectural approaches you need to take here are going to be different then in the Views approach. By squinting at it just the right way, one could argue that each Tool is basically a View, and so what is really different here? The difference, in my experience, is what I think of as Tool Sprawl.

If you have well-defined Workflows, then it is easy to make judgements about what is and isn’t necessary. Features that are extraneous to the required Workflows don’t just waste design and engineering time, they ultimately make the Workflows more complex than necessary - and that makes the User Experience worse! Modern software development orthodoxy is laser-focused on this premise - build the minimally viable product, and iterate, iterate, iterate to remove friction for the user.

Tool-based Applications are fundamentally different in that every additional Tool increases the value of the Application. If I have no use for a particular Tool, then except for the small UI overhead from the additional toolbar button necessary to launch the Tool, it’s addition hardly affects me at all. Of course, learning a new Tool will take some effort. But, the pay-off for that effort is this new Tool can now be combined with all the others! This leads to a sort of Application-level Network Effect, where each new Tool is a force-multiplier for all the existing Tools. This is immediately apparent if one observes virtually all major Content Creation/Editing Tools, where there are untold numbers of toolbars and menus of toolbars and nested tabs of toolbars, hidden behind other toolbars. To an outsider this looks like madness, but to the users, it’s the whole point.

Many people who come from the Workflow-oriented software world look upon these Applications in horror. I have observed many new projects where the team starts out trying to build something “simple”, that focuses on “core workflows”, perhaps for “novice users”, and lots of nice linear Workflow diagrams get drawn. But the reality is that Novice Users are only Novices until they have mastered your Application, and then they will immediately ask for more features. And so you will add a Tool here and there. And several years later you will have a sprawling set of Tools, and if you don’t have a systematic way to organize it all, you will have a mess on your hands.

Containing The Damage

Where does the mess come from? From what I have seen, there are a few very common ways to get in trouble. The first is just under-estimating the complexity of the task at hand. Many Content Creation Apps start out as “Viewers”, where all the app logic for things like 3D camera controls are done directly within the mouse and UI button handlers. Then over time new Editing functionality is incorporated by just adding more if/else branches or switch cases. This approach can carry on for quite a long time, and many 3D apps I have worked on still have these vestigial code-limbs at their core. But you’re just digging a deeper code-hole and filling it with code-spaghetti. Eventually, some actual software architecture will be needed, and painful refactoring efforts will be required (followed by years of fixing regressions, as users discover that all their favorite features are broken or work slightly differently now).

Even with some amount of “Tool Architecture”, how to handle device input is tricky, and often ends up leading to messy architectural lock-in. Given that “Tools” are often driven by device input, a seemingly-obvious approach is to directly give Tools input event handlers, like OnMouseUp/OnMouseMove/OnMouseDown functions. This becomes a natural place to put the code that “does things”, for example on a mouse event you might directly apply a brush stamp in a painting tool. Seems harmless until users ask for support for other input devices, like touch, or pen, or VR controllers. Now what? Do you just forward calls to your mouse handlers? What about pressure, or 3D position? And then comes automation, when users start asking for the ability to script what your Tool does. I have been in situations myself where “inject fake mouse event to force OnMouseX to run” started to seem like a viable solution (It is not. Absolutely not. Really, don’t).

Putting important code in input event handlers also leads to things like rampant copy-paste of standard event-handling patterns, which can be tedious to unwind if changes need to be made. And, expensive mouse event handlers will actually make your app feel less responsive than it ought to, due to something called mouse event priority. So, you really want to handle this part of your Tool Architecture carefully, because seemingly-standard design patterns can encourage a whole range of problems.

At the same time, if the Tools Architecture is too tightly defined, it can become a barrier to expanding the toolset, as new requirements come in that don’t “fit” the assumptions underlying the initial design. If many tools have been built on top of that initial architecture, it becomes intractable to change, and then clever Engineers are forced to come up with workarounds, and now you have two (or more) Tool Architectures. One of the biggest challenges is precisely how to divide up responsibilities between the Tool implementations and the Framework.

I can’t claim that the Interactive Tools Framework (ITF) will solve these problems for you. Ultimately, any successful software will end up being trapped by early design decisions, on top of which mountains have been built, and changing course can only happen at great expense. I could tell you stories all day, about how I have done this to myself. What I can say is, the ITF as realized in UE4 hopefully benefits from my past mistakes. Our experience with people using the ITF to build new Tools in the UE4 Editor over the past 2 years has (so far) been relatively painless, and we are continually looking for ways to smooth out any points of friction that do come up.

Tools, ToolBuilders, and the ToolManager

As I laid out above, an Interactive Tool is a Modal State of an Application, during which Device Input can be captured and interpreted in a specific way. In the Interactive Tools Framework (ITF), the UInteractiveTool base class represents the Modal State, and has a very small set of API functions that you are likely to need to implement. Below I have summarized the core UInteractiveTool API in psuedo-C++ (I have omitted things like virtual, const, optional arguments, etc, for brevity). There are other sets of API functions that we will cover to some extent later, but these are the critical ones. You initialize your Tool in ::Setup(), and do any finalization and cleanup in ::Shutdown(), which is also where you would do things like an ‘Apply’ operation. EToolShutdownType is related to the HasAccept() and CanAccept() functions, I will explain more below. Finally a Tool will be given a chance to Render() and Tick each frame. Note that there is also a ::Tick() function, but you should override ::OnTick() as the base class ::Tick() has critical functionality that must always run.

UCLASS()
class UInteractiveTool : public UObject, public IInputBehaviorSource
{
    void Setup();
    void Shutdown(EToolShutdownType ShutdownType);
    void Render(IToolsContextRenderAPI* RenderAPI);
    void OnTick(float DeltaTime);

    bool HasAccept();
    bool CanAccept();
};

A UInteractiveTool is not a standalone object, you cannot simply spawn one yourself. For it to function, something must call Setup/Render/Tick/Shutdown, and pass appropriate implementations of things like the IToolsContextRenderAPI, which allow the Tool to draw lines/etc. I will explain further below. But for now what you need to know is, to create a Tool instance, you will need to request one from a UInteractiveToolManager. To allow the ToolManager to build arbitrary types, you register a <String, UInteractiveToolBuilder> pair with the ToolManager. The UInteractiveToolBuilder is a very simple factory-pattern base class that must be implemented for each Tool type:

UCLASS()
class UInteractiveToolBuilder : public UObject
{
    bool CanBuildTool(const FToolBuilderState& SceneState);
    UInteractiveTool* BuildTool(const FToolBuilderState& SceneState);
};

The main API for UInteractiveToolManager is summarized below. Generally you will not need to implement your own ToolManager, the base implementation is fully functional and should do everything required to use Tools. But you are free to extend the various functions in a subclass, if necessary.

The functions below are listed in roughly the order you would call them. RegisterToolType() associates the string identifier with a ToolBuilder implementation. The Application then sets an active Builder using SelectActiveToolType(), and then ActivateTool() to create a new UInteractiveTool instance. There are getters to access the active Tool, but there is rarely call to do this frequently, in practice. The Render() and Tick() functions must be called each frame by the Application, which then call the associated functions for the active Tool. Finally DeactiveTool() is used to terminate the active Tool.

UCLASS()
class UInteractiveToolManager : public UObject, public IToolContextTransactionProvider
{
    void RegisterToolType(const FString& Identifier, UInteractiveToolBuilder* Builder);
    bool SelectActiveToolType(const FString& Identifier);
    bool ActivateTool();

    void Tick(float DeltaTime);
    void Render(IToolsContextRenderAPI* RenderAPI);

    void DeactivateTool(EToolShutdownType ShutdownType);
};

Tool Life Cycle

At the high level, the Life Cycle of a Tool is as follows

1. ToolBuilder is registered with ToolManager

2. Some time later, User indicates they wish to start Tool (eg via button)

3. UI code sets Active ToolBuilder, Requests Tool Activation

4. ToolManager checks that ToolBuilder.CanBuildTool() = true, if so, calls BuildTool() to create new instance

5. ToolManager calls Tool Setup()

6. Until Tool is deactivated, it is Tick()’d and Render()’d each frame

7. User indicates they wish to exit Tool (eg via button, hotkey, etc)

8. ToolManager calls Tool Shutdown() with appropriate shutdown type

9. Some time later, Tool instance is garbage collected

Note the last step. Tools are UObjects, so you cannot rely on the C++ destructor for cleanup. You should do any cleanup, such as destroying temporary actors, in your Shutdown() implementation.

EToolShutdownType and the Accept/Cancel Model

A Tool can support termination in two different ways, depending on what type of interactions the Tool supports. The more complex alternative is a Tool which can be Accepted (EToolShutdownType::Accept) or Cancelled (EToolShutdownType::Cancel). This is generally used when the Tool’s interaction supports some kind of live preview of an operation, that the user may wish to discard. For example, a Tool that applies a mesh simplification algorithm to a selected Mesh likely has some parameters the user may wish to explore, but if the exploration is unsatisfactory, the user may prefer to not apply the simplification at all. In this case, the UI can provide buttons to Accept or Cancel the active Tool, which result in calls to ToolManager::DeactiveTool() with the appropriate EToolShutdownType value.

The second termination alternative - EToolShutdownType::Completed - is simpler in that it simply indicates that the Tool should “exit”. This type of termination can be used to handle cases where there is no clear ‘Accept’ or ‘Cancel’ action, for example in Tools that simply visualize data, Tools where editing operations are applied incrementally (eg spawning objects based on click points), and so on.

To be clear, you do not need to use or support Accept/Cancel-style Tools in your usage of the ITF. Doing so generally results in a more complex UI. And if you support Undo in your application, then even Tools that could have Accept and Cancel options, can equivalently be done as Complete-style Tools, and the user can Undo if they are unhappy. However, if the Tool completion can involve lengthy computations or is destructive in some way, supporting Accept/Cancel tends to result in a better user experience. In the UE Editor’s Modeling Mode, we generally use Accept/Cancel when editing Static Mesh Assets for precisely this reason.

Another decision you will have to make is how to handle the modal nature of Tools. Generally it is useful to think of the user as being “in” a Tool, ie in the particular Modal state. So how do they get “out”? You can require the user to explicitly click Accept/Cancel/Complete buttons to exit the active Tool, this is the simplest and most explicit, but does mean clicks are necessary, and the user has to mentally be aware of and manage this state. Alternately you could automatically Accept/Cancel/Complete when the user selects another Tool in the Tool toolbar/menu/etc (for example). However this raises a thorny issue of whether one should auto-Accept or auto-Cancel. There is no right answer to this question, you must decide what is best for your particular context (although in my experience, auto-Cancelling is can be quite frustrating when one accidentally mis-clicks!)

Base Tools

One of the main goals of the ITF is to reduce the amount of boilerpate code necessary to write Tools, and improve consistency. Several “tool patterns” come up so frequently that we have included standard implementations of them in the ITF, in the /BaseTools/ subfolder. Base Tools generally include one or more InputBehaviors (see below), whose actions are mapped to virtual functions you can override and implement. I will briefly describe each of these Base Tools as they are both a useful way to build your own Tools, and a good source of sample code for how to do things:

USingleClickTool captures mouse-click input and, if the IsHitByClick() function returns a valid hit, calls OnClicked() function. You provide implementations of both of these. Note that the FInputDeviceRay structure here includes both a 2D mouse position, and 3D ray.

class INTERACTIVETOOLSFRAMEWORK_API USingleClickTool : public UInteractiveTool
{
    FInputRayHit IsHitByClick(const FInputDeviceRay& ClickPos);
    void OnClicked(const FInputDeviceRay& ClickPos);
};

UClickDragTool captures and forwards continuous mouse, input instead of a single click. If CanBeginClickDragSequence() returns true (generally you would do a hit-test here, similar to USingleClickTool), then OnClickPress() / OnClickDrag() / OnClickRelease() will be called, similar to standard OnMouseDown/Move/Up event patterns. Note, however, that you must handle the case where the sequence aborts without a Release, in OnTerminateDragSequence().

class INTERACTIVETOOLSFRAMEWORK_API UClickDragTool : public UInteractiveTool
{
    FInputRayHit CanBeginClickDragSequence(const FInputDeviceRay& PressPos);
    void OnClickPress(const FInputDeviceRay& PressPos);
    void OnClickDrag(const FInputDeviceRay& DragPos);
    void OnClickRelease(const FInputDeviceRay& ReleasePos);
    void OnTerminateDragSequence();
};

UMeshSurfacePointTool is similar to UClickDragTool in that it provides a click-drag-release input handling pattern. However, UMeshSurfacePointTool assumes that it is acting on a target UPrimitiveComponent (how it gets this Component will be explained below). The default implementation of the HitTest() function below will use standard LineTraces (so you don’t have to override this function if that is sufficient). UMeshSurfacePointTool also supports Hover, and tracks the state of Shift and Ctrl modifier keys. This is a good starting point for simple “draw-on-surface” type tools, and many of the Modeling Mode Tools derive from UMeshSurfacePointTool. (One small note: this class also supports reading stylus pressure, however in UE4.26 stylus input is Editor-Only) ((Extra Note: Although it is named UMeshSurfacePointTool, it does not actually require a Mesh, just a UPrimitiveComponent that supports a LineTrace))

class INTERACTIVETOOLSFRAMEWORK_API UMeshSurfacePointTool : public UInteractiveTool
{
    bool HitTest(const FRay& Ray, FHitResult& OutHit);
    void OnBeginDrag(const FRay& Ray);
    void OnUpdateDrag(const FRay& Ray);
    void OnEndDrag(const FRay& Ray);

    void OnBeginHover(const FInputDeviceRay& DevicePos);
    bool OnUpdateHover(const FInputDeviceRay& DevicePos);
    void OnEndHover();
};

There is a fourth Base Tool, UBaseBrushTool, that extends UMeshSurfacePointTool with various functionality specific to Brush-based 3D Tool, ie a surface painting brush, 3D sculpting tool, and so on. This includes a set of standard brush properties, a 3D brush position/size/falloff indicator, tracking of “brush stamps”, and various other useful bits. If you are building brush-style Tools, you may find this useful.

FToolBuilderState

The UInteractiveToolBuilder API functions both take a FToolBuilderState argument. The main purpose of this struct is to provide Selection information - it indicates what the Tool would or should act on. Key fields of the struct are shown below. The ToolManager will construct a FToolBuilderState and pass it to the ToolBuilders, which will then use it to determine if they can operate on the Selection. Both Actors and Components can be passed, but also only Actors and Components, in the UE4.26 ITF implementation. Note that if a Component appears in SelectedComponents, then it’s Actor will be in SelectedActors. The UWorld containing these Actors is also included.

struct FToolBuilderState
{
    UWorld* World;
    TArray<AActor*> SelectedActors;
    TArray<UActorComponent*> SelectedComponents;
};

In the Modeling Mode Tools, we do not directly operate on Components, we wrap them in an standard container, so that we can, for example, 3D sculpt “any” mesh Component that has a container implementation. This is largely why I can write this tutorial, because I can make those Tools edit other types of meshes, like Runtime meshes. But when building your own Tools, you are free to ignore FToolBuilderState. Your ToolBuilders can use any other way to query scene state, and your Tools are not limited to acting on Actors or Components.

On ToolBuilders

A frequent question that comes up among users of the ITF is whether the UInteractiveToolBuilder is necessary. In the simplest cases, which are the most common, your ToolBuilder will be straightforward boilerplate code (unfortunately since it is a UObject, this boilerplate cannot be directly converted to a C++ template). The utility of ToolBuilders arises when one starts to re-purpose existing UInteractiveTool implementations to solve different problems.

For example, in the UE Editor we have a Tool for editing mesh polygroups (effectively polygons), called PolyEdit. We also have a very similar tool for editing mesh triangles, called TriEdit. Under the hood, these are the same UInteractiveTool class. In TriEdit mode, the Setup() function configures various aspects of the Tool to be appropriate for triangles. To expose these two modes in the UI, we use two separate ToolBuilders, which set a “bIsTriangleMode” flag on the created Tool instance after it is allocated, but before Setup() runs.

I certainly won’t claim this is an elegant solution. But, it was expedient. In my experience, this situation arises all the time as your set of Tools evolves to handle new situations. Frequently an existing Tool can be shimmed in to solve a new problem with a bit of custom initialization, a few additional options/properties, and so on. In an ideal world one would refactor the Tool to make this possible via subclassing or composition, but we rarely live in the ideal world. So, the bit of unsightly code necessary to hack a Tool to do a second job, can be placed in a custom ToolBuilder, where it is (relatively) encapsulated.

The string-based system for registering ToolBuilders with the ToolManager can allow your UI level (ie button handlers and so on) to launch Tools without having to actually know about the Tool class types. This can often allow for a cleaner separation of concerns when building the UI. For example, in the ToolsFrameworkDemo I will describe below, the Tools are launched by UMG Blueprint Widgets that simply pass string constants to a BP Function - they have no knowledge of the Tool system at all. However, the need to set an ‘Active’ builder before spawning a Tool is somewhat of a vestigial limb, and these operations will likely be combined in the future.

The Input Behavior System

Above I stated that “An Interactive Tool is a Modal State of an Application, during which Device Input can be captured and interpreted in a specific way”. But the UInteractiveTool API does not have any mouse input handler functions. This is because Input Handling is (mostly) decoupled from the Tools. Input is captured and interpreted by UInputBehavior objects that the Tool creates and registers with the UInputRouter, which “owns” the input devices and routes input events to the appropriate Behavior.

The reason for this separation is that the vast majority of input handling code is cut-and-pasted, with slight variations in how particular interactions are implemented. For example consider a simple button-click interaction. In a common event API you would have something like OnMouseDown(), OnMouseMove(), and OnMouseUp() functions that can be implemented, and lets say you want to map from those events to a single OnClickEvent() handler, for a button press-release action. A surprising number of applications (particularly web apps) will fire the click in OnMouseDown - which is wrong! But, blindly firing OnClickEvent in OnMouseUp is also wrong! The correct behavior here is actually quite complex. In OnMouseDown(), you must hit-test the button, and begin capturing mouse input. In OnMouseUp, you have to hit-test the button again, and if the cursor is still hitting the button, only then is OnClickEvent fired. This allows for cancelling a click and is how all serious UI toolkits have it implemented (try it!).

If you have even tens of Tools, implementing all this handling code, particularly for multiple devices, becomes very error-prone. So for this reason, the ITF moves these little input-event-handling state machines into UInputBehavior implementations which can be shared across many tools. In fact a few simple behaviors like USingleClickInputBehavior, UClickDragBehavior, and UHoverBehavior handle the majority of cases for mouse-driven interaction. The Behaviors then forward their distilled events to target objects via simple interfaces that something like a Tool or Gizmo can implement. For example USingleClickInputBehavior can act on anything that implemments IClickBehaviorTarget, which just has two functions - IsHitByClick() and OnClicked(). Note that because the InputBehavior doesn’t know what it is acting on - the “button” could be a 2D rectangle or an arbitrary 3D shape - the Target interface has to provide the hit-testing functionality.

Another aspect of the InputBehavior system is that Tools do not directly talk to the UInputRouter. They only provide a list of UInputBehavior’s that they wish to have active. The additions to the UInteractiveTool API to support this are shown below. Generally, in a Tool’s ::Setup() implementation, one or more Input Behaviors are created and configured, and passed to AddInputBehavior. The ITF then calls GetInputBehaviors when necessary, to register those behaviors with the UInputRouter. Note: currently the InputBehavior set cannot change dynamically during the Tool, however you can configure your Behaviors to ignore events based on whatever criteria you wish.

class UInteractiveTool : public UObject, public IInputBehaviorSource
{
    // ...previous functions...

    void AddInputBehavior(UInputBehavior* Behavior);
    const UInputBehaviorSet* GetInputBehaviors();
};

The UInputRouter is similar to the UInteractiveToolManager in that the default implementation is sufficient for most usage. The only job of the InputRouter is to keep track of all the active InputBehaviors and mediate capture of the input device. Capture is central to input handling in Tools. When a MouseDown event comes into the InputRouter, it checks with all the registered Behaviors to ask if they want to start capturing the mouse event stream. For example if you press down over a button, that button’s registered USingleClickInputBehavior would indicate that yes, it wants to start capturing. Only a single Behavior is allowed to capture input at a time, and multiple Behaviors (which don’t know about eachother) might want to capture - for example, 3D objects that are overlapping from the current view. So, each Behavior returns a FInputCaptureRequest that indicates “yes” or “no” along with depth-test and priority information. The UInputRouter then looks at all the capture requests and, based on depth-sorting and priority, selects one Behavior and tells it that capture will begin. Then MouseMove and MouseRelease events are only passed to that Behavior until the Capture terminates (usually on MouseRelease).

In practice, you will rarely have to interact with UInputRouter when using the ITF. Once the connection between application-level mouse events and the InputRouter is established, you shouldn’t ever need to touch it again. This system largely deals away with common errors like mouse handling “getting stuck” due to a capture gone wrong, because the UInputRouter is ultimately in control of mouse capture, not individual Behaviors or Tools. In the accompanying ToolsFrameworkDemo project, I have implemented everything necessary for the UInputRouter to function.

The basic UInputBehavior API is shown below. The FInputDeviceState is a large structure that contains all input device state for a given event/time, including status of common modifier keys, mouse button state, mouse position, and so on. One main difference from many input events is that the 3D World-Space Ray associated with the input device position is also included.

UCLASS()
class UInputBehavior : public UObject
{
    FInputCapturePriority GetPriority();
    EInputDevices GetSupportedDevices();

    FInputCaptureRequest WantsCapture(const FInputDeviceState& InputState);
    FInputCaptureUpdate BeginCapture(const FInputDeviceState& InputState);
    FInputCaptureUpdate UpdateCapture(const FInputDeviceState& InputState);
    void ForceEndCapture(const FInputCaptureData& CaptureData);

    // ... hover support...
}

I have omitted some extra parameters in the above API, to simplify things. In particular if you implement your own Behaviors, you will discover there is an EInputCaptureSide enum passed around nearly everywhere, largely as a default EInputCaptureSide::Any. This is for future use, to support the situation where a Behavior might be specific to a VR controller in either hand.

However, for most apps you will likely find that you never actually have to implement your own Behavior. A set of standard behaviors, such as those mentioned above, is included in the /BaseBehaviors/ folder of the InteractiveToolFramework module. Most of the standard Behaviors are derived from a base class UAnyButtonInputBehavior, which allows them to work with any mouse button, including “custom” buttons defined by a TFunction (which could be a keyboard key)! Similarly the standard BehaviorTarget implementations all derive from IModifierToggleBehaviorTarget, which allows for arbitrary modifier keys to be configured on a Behavior and forwarded to the Target without having to subclass or modify the Behavior code.

Direct Usage of UInputBehaviors

In the discussion above, I focused on the case where a UInteractiveTool provides a UInputBehaviorSet. Gizmos will work similarly. However, the UInputRouter itself is not aware of Tools at all, and it is entirely possible to use the InputBehavior system separately from either. In the ToolsFrameworkDemo, I implemented the click-to-select-meshes interaction this way, in the USceneObjectSelectionInteraction class. This class implements IInputBehaviorSource and IClickBehaviorTarget itself, and is just owned by the framework back-end subsystem. Even this is not strictly necessary - you can directly register a UInputBehavior you create yourself with the UInputRouter (note, however, that due to an API oversight on my part, in UE4.26 you cannot explicitly unregister a single Behavior, you can only unregister by source).

Non-Mouse Input Devices

Additional device types are currently not handled in the UE4.26 ITF implementation, however the previous iteration of this behavior system in frame3Sharp supported touch and VR controller input, and these should (eventually) work similarly in the ITF design. The general idea is that only the InputRouter and Behaviors need to explicitly know about different input modalities. An IClickBehaviorTarget implementation should work similarly with a mouse button, finger tap, or VR controller click, but also nothing rules out additional Behavior Targets tailored for device-specific interactions (eg from a two-finger pinch, spatial controller gesture, and so on). Tools can register different Behaviors for different device types, the InputRouter would take care of handling which devices are active and capturable.

Currently, some level of handling of other device types can be accomplished by mapping to mouse events. Since the InputRouter does not directly listen to the input event stream, but rather the ITF back-end creates and forwards events, this is a natural place to do such mappings, some more detail will be explained below.

A Limitation - Capture Interruption

One limitation of this system which is important to be aware of when designing your interactions is that “interruption” of an active capture is not yet supported by the framework. This most frequently arises when one wishes to have an interaction that would either be a click, or a drag, depending on if the mouse is immediately released in the same location, or moved some threshold distance. In simple cases this can be handled via UClickDragBehavior, with your IClickDragBehaviorTarget implementation making the determination. However, if the click and drag actions need to go to very different places that are not aware of eachother, this may be painful. A cleaner way to support this kind of interaction is to allow one UInputBehavior to “interrupt” another - in this case, the drag to “interrupt” the click’s active capture when it’s preconditions (ie sufficient mouse movement) are met. This is an area of the ITF that may be improved in the future.

Tool Property Sets

UInteractiveTool has one other set of API functions that I haven’t covered, which is for managing a set of attached UInteractiveToolPropertySet objects. This is a completely optional system that is somewhat tailored for usage in the UE Editor. For Runtime usage it is less effective. Essentially UInteractiveToolPropertySet’s are for storing your Tool Settings and Options. They are UObjects with UProperties, and in the Editor, these UObjects can be added to a Slate DetailsView to automatically expose those properties in the Editor UI.

The additional UInteractiveTool APIs are summarized below. Generally in the Tool ::Setup() function, various UInteractiveToolPropertySet subclasses will be created and passed to AddToolPropertySource(). The ITF back-end will use the GetToolProperties() function to initialize the DetailsView panel, and then the Tool can show and hide property sets dynamically using SetToolPropertySourceEnabled()

class UInteractiveTool : public UObject, public IInputBehaviorSource
{
    // ...previous functions...
public:
    TArray<UObject*> GetToolProperties();
protected:
    void AddToolPropertySource(UObject* PropertyObject);
    void AddToolPropertySource(UInteractiveToolPropertySet* PropertySet);
    bool SetToolPropertySourceEnabled(UInteractiveToolPropertySet* PropertySet, bool bEnabled);
};

In the UE Editor, UProperties can be marked up with meta tags to control the generated UI widgets - things like slider ranges, valid integer values, and enabling/disabling widgets based on the value of other properties. Much of the UI in the Modeling Mode works this way.

Unfortunately, UProperty meta tags are not available at Runtime, and the DetailsView panels are not supported in UMG Widgets. As a result, the ToolPropertySet system becomes much less compelling. It does still provide some useful functionality though. For one, the Property Sets support saving and restoring their Settings across Tool invocations, using the SaveProperties() and RestoreProperties() functions of the property set. You simply call SaveProperties() on each property set in your Tool Shutdown(), and RestoreProperties() in ::Setup().

A second useful ability is the WatchProperty() function, which allows for responding to changes in PropertySet values without any kind of change notification. This is necessary with UObjects because C++ code can change a UProperty on a UObject directly, and this will not cause any kind of change notification to be sent. So, the only way to reliably detect such changes is via polling. Yes, polling. It’s not ideal, but do consider that (1) a Tool necessarily has a limited number of properties that a user can possibly handle and (2) only one Tool is active at a time. To save you from having to implement a stored-value-comparison for each property in your ::OnTick(), you can add watchers using this pattern:

MyPropertySet->WatchProperty( MyPropertySet->bBooleanProp,  [this](bool bNewValue) { // handle change! } );

In UE4.26 there are some additional caveats (read: bugs) that must be worked around, see below for more details.

Tool Actions

Finally, the last major part of the UInteractiveTool API is support for Tool Actions. These are not widely used in the Modeling Mode toolset, except to implement hotkey functionality. However, the Tool Actions are not specifically related to hotkeys. What they allow is for a Tool to expose “Actions” (ie parameterless functions) that can be called via integer identifiers. The Tool constructs and returns a FInteractiveToolActionSet, and then higher-level client code can enumerate these actions, and execute them using the ExecuteAction function defined below.

class UInteractiveTool : public UObject, public IInputBehaviorSource
{
    // ...previous functions...
public:
    FInteractiveToolActionSet* GetActionSet();
    void ExecuteAction(int32 ActionID);
protected:
    void RegisterActions(FInteractiveToolActionSet& ActionSet);
};

The sample code below shows two Tool Actions being registered. Note that although the FInteractiveToolAction contains a hotkey and modifier, these are only suggestions to the higher-level client. The UE Editor queries Tools for Actions, and then registers the suggested hotkeys as Editor hotkeys, which allows the user to remap them. UE does not have any kind of similar hotkey system at Runtime, you would need to manually map these hotkeys yourself

void UDynamicMeshSculptTool::RegisterActions(FInteractiveToolActionSet& ActionSet)
{
    ActionSet.RegisterAction(this, (int32)EStandardToolActions::BaseClientDefinedActionID + 61,
        TEXT("SculptDecreaseSpeed"),
        LOCTEXT("SculptDecreaseSpeed", "Decrease Speed"),
        LOCTEXT("SculptDecreaseSpeedTooltip", "Decrease Brush Speed"),
        EModifierKey::None, EKeys::W,
        [this]() { DecreaseBrushSpeedAction(); });

    ActionSet.RegisterAction(this, (int32)EStandardToolActions::ToggleWireframe,
        TEXT("ToggleWireframe"),
        LOCTEXT("ToggleWireframe", "Toggle Wireframe"),
        LOCTEXT("ToggleWireframeTooltip", "Toggle visibility of wireframe overlay"),
        EModifierKey::Alt, EKeys::W,
        [this]() { ViewProperties->bShowWireframe = !ViewProperties->bShowWireframe; });
}

Ultimately each ToolAction payload is stored as a TFunction<void()>. If you are just forwarding to another Tool function, like the DecreaseBrushSpeedAction() call above, you don’t necessarily benefit from the ToolAction system, and there is no need to use it at all. However due to current limitations with Tool exposure to Blueprints, ToolActions (because they can be called via a simple integer) may be an effective way to expose Tool functionality to BP without having to write many wrapper functions.

Gizmos

As I have mentioned, “Gizmo” refers to those little in-viewport clicky-things we use in 2D and 3D Content Creation/Editing Apps to let you efficiently manipulate parameters of visual elements or objects. If you’ve used any 3D tool, you have almost certainly used a standard Translate/Rotate/Scale Gizmo, for example. Like Tools, Gizmos capture user input, but instead of being a full Modal state, a Gizmo is generally transient, ie Gizmos can come and go, and you can have multiple Gizmos active at the same time, and they only capture input if you click “on” them (what “on” means can be a bit fuzzy). Because of this, Gizmos generally require some specific visual representation that allows the user to indicate when they want to “use” the Gizmo, but conceptually you can also have a Gizmo that does this based on a hotkey or application state (eg checkbox).

In the Interactive Tools Framework, Gizmos are implemented as subclasses of UInteractiveGizmo, which is very similar to UInteractiveTool:

UCLASS()
class UInteractiveGizmo : public UObject, public IInputBehaviorSource
{
    void Setup();
    void Shutdown();
    void Render(IToolsContextRenderAPI* RenderAPI);
    void Tick(float DeltaTime);

    void AddInputBehavior(UInputBehavior* Behavior);
    const UInputBehaviorSet* GetInputBehaviors();
}

And similarly Gizmo instances are managed by a UInteractiveGizmoManager, using UInteractiveGizmoBuilder factories registered via strings. Gizmos use the same UInputBehavior setup, and are similarly Rendered and be Ticked every frame by the ITF.

At this high level, the UInteractiveGizmo is just a skeleton, and to implement a custom Gizmo you will have to do quite a bit of work yourself. Unlike Tools, it’s more challenging to provide “base” Gizmos because of the visual-representation aspect. In particular, the standard InputBehaviors will require that you are able to do raycast hit-testing against your Gizmo, and so you can’t just draw arbitrary geometry in the Render() function. That said, the ITF does provide a very flexible standard Translate-Rotate-Scale Gizmo implementation, which can be repurposed to solve many problems.

Standard UTransformGizmo

It would be very questionable to call the ITF a framework for building 3D tools if it didn’t include standard Translate-Rotate-Scale (TRS) Gizmos. What is currently available in UE4.26 is a combined TRS gizmo (screenshot to the right) called UTransformGizmo that supports Axis and Plane Translation (axis lines and central chevrons), Axis rotation (circles), Uniform Scale (central box), Axis Scale (outer axis brackets), and Plane Scale (outer chevrons). These sub-gizmos are separately configurable, so you can (for example) create a UTransformGizmo instance that only has XY-plane translation and Z rotation just by passing certain enum values to the Gizmo builder.

This TRS Gizmo is not a single monolithic Gizmo, it is built up out of a set of parts that can be repurposed for many other uses. This subsystem is complex enough that it warrants a separate article, but to summarize, each element of the UTransformGizmo that I mentioned above is actually a separate UInteractiveGizmo (so, yes, you can have nested/hierarchical Gizmos, and you could subclass UTransformGizmo to add additional custom controls). For example, the axis-translation sub-gizmos (drawn as the red/green/blue line segments) are instances of UAxisPositionGizmo, and the rotation circles are UAxisAngleGizmo.

The sub-gizmos like UAxisPositionGizmo do not explicitly draw the lines in the image above. They are instead connected to an arbitrary UPrimitiveComponent which provides the visual representation and hit-testing. So, you could use any UStaticMesh, if you wished. By default, UTransformGizmo spawns custom gizmo-specific UPrimitiveComponents, in the case of the lines, it is a UGizmoArrowComponent. These GizmoComponents provide some niceties like constant screen-space dimensions, hover support, and so on. But you absolutely do not have to use them, and the Gizmo look could be completely customized for your purposes (a topic for a future Gizmo-focused article!).

So, the UAxisPositionGizmo is really just an implementation of the abstract concept of “specifying position along a line based on mouse input”. The 3D line, mapping from line position to abstract parameter (in the default case, 3D world position), and state-change information are all implemented via UInterfaces and so can be customized if necessary. The visual representation is only to inform the user, and to provide a hit-target for the InputBehavior that captures the mouse. This allows functionality like arbitrary Snapping or parameter constraints to be integrated with minimal difficultly.

But, this is all an aside. In practice, to use a UTransformGizmo, you just request one from the GizmoManager using one of the following calls:

class UInteractiveGizmoManager 
{
    // ... 
    UTransformGizmo* Create3AxisTransformGizmo(void* Owner);
    UTransformGizmo* CreateCustomTransformGizmo(ETransformGizmoSubElements Elements, void* Owner);
}

Then you create a UTransformProxy instance and set it as the Target of the Gizmo. The Gizmo will now be fully functional, you can move it around the 3D scene, and respond to transform changes via the UTransformProxy::OnTransformChanged delegate. Various other delegates are available, eg for begin/end a transform interaction. Based on these delegates, you could transform objects in your scene, update parameters of an object, and so on.

A slightly more complex usage is if you want the UTransformProxy to directly move one or more UPrimitiveComponents, ie to implement the normal “select objects and move them with gizmo” type of interface that nearly every 3D design app has. In this case the Components can be added as targets of the Proxy. The Gizmo still acts on the UTransformProxy, and the Proxy re-maps that single transform to relative transforms on the object set.

The UTransformGizmo does not have to be owned by a Tool. In the ToolsFrameworkDemo, the USceneObjectTransformInteraction class watches for selection changes in the runtime objects Scene, and if there is an active selection, spawns a suitable new UTransformGizmo. The code is only a handful of lines:

TransformProxy = NewObject<UTransformProxy>(this);
for (URuntimeMeshSceneObject* SceneObject : SelectedObjects)
{
    TransformProxy->AddComponent(SO->GetMeshComponent());
}

TransformGizmo = GizmoManager->CreateCustomTransformGizmo(ETransformGizmoSubElements::TranslateRotateUniformScale, this);
TransformGizmo->SetActiveTarget(TransformProxy);

In this case I am passing ETransformGizmoSubElements::TranslateRotateUniformScale to create TRS gizmos that do not have the non-uniform scaling sub-elements. To destroy the gizmo, the code simply calls DestroyAllGizmosByOwner, passing the same void* pointer used during creation:

GizmoManager->DestroyAllGizmosByOwner(this);

The UTransformGizmo automatically emits the necessary undo/redo information, which will be discussed further below. So as long as the ITF back-end in use supports undo/redo, so will the gizmo transformations.

Local vs Global Coordinate Systems

The UTransformGizmo supports both local and global coordinate systems. By default, it requests the current Local/Global setting from the ITF back-end. In the UE Editor, this is controlled in the same way as the default UE Editor gizmos, by using the same world/local toggle at the top of the main viewport. You can also override this behavior, see the comments in the UTransformGizmoBuilder header.

One caveat, though. UE4 only supports non-uniform scaling transformations in the local coordinate-system of a Component. This is because two separate FTransform’s with non-uniform scaling cannot be combined into a single FTransform, in most cases. So, when in Global mode, the TRS Gizmo will not show the non-uniform scaling handles (the axis-brackets and outer-corner chevrons). The default UE Editor Gizmos have the same limitation, but handle it by only allowing usage of the Local coordinate system in the scaling Gizmo (which is not combined with the translate and rotate Gizmos).

The Tools Context and ToolContext APIs

At this point we have Tools and a ToolManager, and Gizmos and a GizmoManager, but who manages the Managers? Why, the Context of course. UInteractiveToolsContext is the topmost level of the Interactive Tools Framework. It is essentially the “universe” in which Tools and Gizmos live, and also owns the InputRouter. By default, you can simply use this class, and that’s what I’ve done in the ToolsFrameworkDemo. In the UE Editor usage of the ITF, there are subclasses that mediate the communication between the ITF and higher-level Editor constructs like an FEdMode (for example see UEdModeInteractiveToolsContext).

The ToolsContext also provides the Managers and InputRouter with implementations of various APIs that provide “Editor-like” functionality. The purpose of these APIs is to essentially provide an abstraction of an “Editor”, which is what has allowed us to prevent the ITF from having explicit Unreal Editor dependencies. In the text above I have mentioned the “ITF back-end” multiple times - this is what I was referring to.

If it’s still not clear what I mean by an “abstraction of an Editor”, perhaps an example. I have not mentioned anything about object Selections yet. This is because the concept of selected objects is largely outside the scope of the ITF. When the ToolManager goes to construct a new Tool, it does pass a list of selected Actors and Components. But it gets this list by asking the Tools Context. And the Tools Context doesn’t know, either. The Tools Context needs to ask the Application that created it, via the IToolsContextQueriesAPI. This surrounding Application must create an implementation of IToolsContextQueriesAPI and pass it to the ToolsContext on construction.

The ITF cannot solve “how object selection works” in a generic way because this is highly dependent on your Application. In the ToolsFrameworkDemo I have implemented a basic mesh-objects-and-selection-list mechanism, that behaves similarly to most DCC tools. The Unreal Editor has a similar system in the main viewport. However, in Asset Editors, there is only ever a single object, and there is no selection at all. So the IToolsContextQueriesAPI used inside Asset Editors is different. And if you were using the ITF in a game context, you likely will have a very different notion of what “selection” is, or even what “objects” are.

So, our goal with the ToolContext APIs is to require the minimal set of functions that allow Tools to work within “an Editor-like container”. These APIs have grown over time as new situations arise where the Editor-container needs to be queried. They are defined in the file ToolContextInterfaces.h and summarized below

IToolsContextQueriesAPI

This API provides functions to query state information from the Editor container. The most critical is GetCurrentSelectionState(), which will be used by the ToolManager to determine which selected actors and Components to pass to the ToolBuilders. You will likely need to have a custom implementation of this in your usage of the ITF. GetCurrentViewState() is also required for many Tools to work correctly, and for the TRS Gizmos, as it provides the 3D camera/view information. However the sample implementation in the ToolsFrameworkDemo is likely sufficient for any Runtime use that is a standard fullscreen single 3D view. The other functions here can have trivial implementations that just return a default value.

class IToolsContextQueriesAPI
{
    void GetCurrentSelectionState(FToolBuilderState& StateOut);
    void GetCurrentViewState(FViewCameraState& StateOut);
    EToolContextCoordinateSystem GetCurrentCoordinateSystem();
    bool ExecuteSceneSnapQuery(const FSceneSnapQueryRequest& Request, TArray<FSceneSnapQueryResult>& Results );
    UMaterialInterface* GetStandardMaterial(EStandardToolContextMaterials MaterialType);
}

IToolsContextTransactionsAPI

The IToolsContextTransactionsAPI is mainly used to send data back to the Editor container. DisplayMessage() is called by Tools with various user-informative messages, error and status messages, and so on. These can be ignored if preferred. PostInvalidation() is used to indicate that a repaint is necessary, which is generally can be ignored in a Runtime context where the engine is continually redrawing at maximum/fixed framerate. RequestSelectionChange() is a hint certain Tools provide, generally when they create a new object, and can be ignored.

class IToolsContextTransactionsAPI
{
    void DisplayMessage(const FText& Message, EToolMessageLevel Level);
    void PostInvalidation();
    bool RequestSelectionChange(const FSelectedOjectsChangeList& SelectionChange);

    void BeginUndoTransaction(const FText& Description);
    void AppendChange(UObject* TargetObject, TUniquePtr<FToolCommandChange> Change, const FText& Description);
    void EndUndoTransaction();
}

AppendChange() is called by Tools that want to emit a FCommandChange record (actually a FToolCommandChange subclass), which is the core component of the ITF approach to Undo/Redo. To understand why this design is the way it is, I have to explain about about how Undo/Redo works in the UE Editor. The Editor does not use a Command-Objects/Pattern approach to Undo/Redo, which is generally the way that most 3D Content Creation/Editing Tools do it. Instead the Editor uses a Transaction system. After opening a Transaction, UObject::Modify() is called on any object that is about to be modified, and this saves a copy of all the UObject’s current UProperty values. When the Transaction is closed, the UProperties of modified objects are compared, and any changes are serialized. This system is really the only way to do it for something like UObjects, that can have arbitrary user-defined data via UProperties. However, Transaction systems are known to not perform well when working with large complex data structures like meshes. For example, storing arbitrary partial changes to a huge mesh as a Transaction would involve making a full copy up front, and then searching for and encoding changes to the complex mesh data structures (essentially unstructured graphs). This is very difficult (read: slow) computational problem. Similarly a simple 3D translation will modify every vertex, requiring a full copy of all the position data in a Transaction, but in a Change can be stored as just the translation vector and a bit if information about what operation to apply.

So, when building the ITF, we added support for embedding FCommandChange objects inside UE Editor Transactions. This is a bit of a kludge, but generally works, and a useful side-effect is that these FCommandChanges can also be used at Runtime, where the UE Editor Transaction system does not exist. Most of our Modeling Mode Tools are continually calling AppendChange() as the user interacts with the Tool, and the Gizmos do this as well. So, we can build a basic Undo/Redo History system simply by storing these Changes in the order they come in, and then stepping back/forward in the list on Undo/Redo, calling Revert()/Apply() on each FToolCommandChange object.

BeginUndoTransaction() and EndUndoTransaction() are related functions that mark the start and end of a set of Change records that should be grouped - generally AppendChange() will be called one or more times in-between. To provide the correct UX - ie that a single Undo/Redo hotkey/command processes all the Changes at once - the ToolsFrameworkDemo has a very rudimentary system that stores a set of FCommandChanges.

IToolsContextRenderAPI

This API is passed to UInteractiveTool::Render() and UInteractiveGizmo::Render() to provide information necessary for common rendering tasks. GetPrimitiveDrawInterface() returns an implementation of the abstract FPrimitiveDrawInterface API, which is a standard UE interface that provides line and point drawing functions (commonly abbreviated as PDI). Various Tools use the PDI to draw basic line feedback, for example the edges of the currently Polygon being drawn in the Draw Polygon Tool. Note, however, that PDI line drawing at Runtime is not the same as PDI line drawing in the Editor - it has lower quality and cannot draw the stipped-when-hidden lines that the Editor can.

GetCameraState(), GetSceneView(), and GetViewInteractionState() return information about the current View. These are important in the Editor because the user may have multiple 3D viewports visible (eg in 4-up view), and the Tool must draw correctly in each. At Runtime, there is generally a single camera/view and you should be fine with the basic implementations in the ToolsFramworkDemo. However if you wanted to implement multiple views, you would need to provide them correctly in this API.

class IToolsContextRenderAPI
{
    FPrimitiveDrawInterface* GetPrimitiveDrawInterface();
    FViewCameraState GetCameraState();
    const FSceneView* GetSceneView();
    EViewInteractionState GetViewInteractionState();
}

IToolsContextAssetAPI

The ITooslContextAssetAPI can be used to emit new objects. This is an optional API, and I have only listed the top-level function below, there are other functions that the API includes that are somewhat specific to the UE Editor. This is the hardest part to abstract as it requires some inherent assumptions about what “Objects” are. However, it is also not something that you are required to use in your own Tools. The GenerateStaticMeshActor() function is used by the Editor Modeling Tools to spawn new Static Mesh Assets/Components/Actors, for example in the Draw Polygon Tool, this function is called with the extruded polygon (part of the AssetConfig argument) to create the Asset. This creation process involves things like finding a location (which possibly spawns dialog boxes/etc), creating a new package, and so on.

class IToolsContextAssetAPI
{
    AActor* GenerateStaticMeshActor(
        UWorld* TargetWorld,
        FTransform Transform,
        FString ObjectBaseName,
        FGeneratedStaticMeshAssetConfig&& AssetConfig);
 }

At Runtime, you cannot create Assets, so this function has to do “something else”. In the ToolsFrameworkDemo, I have implemented GenerateStaticMeshActor(), so that some Modeling Mode Tools like the Draw Polygon Tool are able to function. However, it emits a different Actor type entirely.

Actor/Component Selections and PrimitiveComponentTargets

FPrimitiveComponentTarget was removed in UE5, and replaced with a new approach/system. See the section entitled UToolTargets in my article about UE5 changes to the Interactive Tools Framework: https://www.gradientspace.com/tutorials/2022/6/1/the-interactive-tools-framework-in-ue5

In the Tools and ToolBuilders Section above, I described FToolBuilderState, and how the ToolManager constructs a list of selected Actors and Components to pass to the ToolBuilder. If your Tool should act on Actors or Components, you can pass that selection on to the new Tool instance. However if you browse the Modeling Mode Tools code, you will see that most tools act on something called a FPrimitiveComponentTarget, which is created in the ToolBuilders based on the selected UPrimitiveComponents. And we have base classes USingleSelectionTool and UMultiSelectionTool, which most Modeling Mode tools derive from, that hold these selections.

This is not something you need to do if you are building your own Tools from scratch. But, if you want to leverage Modeling Mode Tools, you will need to understand it, so I will explain. The purpose of FPrimitiveComponentTarget is to provide an abstraction of “a mesh that can be edited” to the Tools. This is useful because we have many different Mesh types in Unreal (and you may have your own). There is FMeshDescription (used by UStaticMesh), USkeletalMesh, FRawMesh, Cloth Meshes, Geometry Collections (which are meshes), and so on. Mesh Editing Tools that have to manipulate low-level mesh data structures would essentially require many parallel code paths to support each of these. In addition, updating a mesh in Unreal is expensive. As I have explained in previous tutorials, when you modify the FMeshDescription inside a UStaticMesh, a “build” step is necessary to regenerate rendering data, which can take several seconds on large meshes. This would not be acceptable in, for example, a 3D sculpting Tool where the user expects instantaneous feedback.

So, generally the Modeling Mode Tools cannot directly edit any of the UE Component mesh formats listed above. Instead, the ToolBuilder wraps the target Component in a FPrimitiveComponentTarget implementation, which must provide an API to Read and Write it’s internal mesh (whatever the format) as a FMeshDescription. This allows Tools that want to edit meshes to support a single standard input/output format, at the (potential) cost of mesh conversions. In most Modeling Mode Tools, we then convert that FMeshDescription to a FDynamicMesh3 for actual editing, and create a new USimpleDynamicMeshComponent for fast previews, and only write back the updated FMeshDescription on Tool Accept. But this is encapsulated inside the Tool, and not really related to the FPrimtiveComponentTarget.

FComponentTargetFactory

We need to allow the Interactive Tools Framework to create an FPrimitiveComponentTarget-subclass wrapper for a Component it does not know about (as many Components are part of plugins not visible to the ITF). For example, UProceduralMeshComponent or USimpleDynamicMeshComponent. To do this we provide a FComponentTargetFactory implementation, which has two functions:

class INTERACTIVETOOLSFRAMEWORK_API FComponentTargetFactory
{
public:
    virtual bool CanBuild( UActorComponent* Candidate ) = 0;
    virtual TUniquePtr<FPrimitiveComponentTarget> Build( UPrimitiveComponent* PrimitiveComponent ) = 0;
};

These are generally very simple, for an example, see FStaticMeshComponentTargetFactory in EditorComponentSourceFactory.cpp, which builds FStaticMeshComponentTarget instances for UStaticMeshComponents. The FStaticMeshComponentTarget is also straightforward in this case. We will take advantage of this API to work around some issues with Runtime usage below.

Finally once the FComponentTargetFactory is available, the global function AddComponentTargetFactory() is used to register it. Unfortunately, in UE4.26 this function stores the Factory in a global static TArray that is private to ComponentSourceInterfaces.cpp, and as a result cannot be modified or manipulated in any way. On Startup, the Editor will register the default FStaticMeshComponentTargetFactory and also FProceduralMeshComponentTargetFactory, which handles PMCs. Both of these factories have issues that prevent them from being used at Runtime for mesh editing Tools, and as a result, until this system is improved, we cannot use SMCs or PMCs for Runtime mesh editing. We will instead create a new ComponentTarget for USimpleDynamicMeshComponent (see previous tutorials for details on this mesh Component type).

ToolBuilderUtil.h

If you look at the ToolBuilders for most tools, you will see that the CanBuildTool() and BuildTool() implementations are generally calling static functions in the ToolBuilderUtil namespace, as well as the functions CanMakeComponentTarget() and MakeComponentTarget(). These latter two functions enumerate through the list of registered ComponentTargetFactory instances to determine if a particular UPrimitiveComponent type can be handled by any Factory. The ToolBuilderUtil functions are largely just iterating through selected Components in the FToolBuilderState (described above) and calling a lambda predicate (usually one of the above functions).

I will re-iterate here that you are not required to use the FPrimitiveComponentTarget system in your own Tools, or even the FToolBuilderState. You could just as easily query some other (global) Selection system in your ToolBuilders, check for casts to your target Component type(s), and pass UPrimitiveComponent* or subclasses to your Tools. However, as I mentioned, the Modeling Mode tools work this way, and it will be a significant driver of the design of the Runtime mesh editing Tools Framework I will now describe.

Runtime Tools Framework Back-End

Creating a Runtime back-end for the Interactive Tools Framework is not really that complicated. The main things we have to figure out are:

1. How to collect mouse input events (ie mouse down/move/up) and send this data to the UInputRouter

2. How to implement the IToolsContextQueriesAPI and IToolsContextRenderAPI

3. (Optionally) how to implement IToolsContextTransactionsAPI and IToolsContextAssetAPI

4. How/when to Render() and Tick() the UInteractiveToolManager and UInteractiveGizmoManager

That’s it. Once these things are done (even skipping step 3) then basic Tools and Gizmos (and even the UTransformGizmo) will be functional.

In this sample project, all the relevant code to accomplish the above is in the RuntimeToolsSystem module, split into four subdirectories:

• RuntimeToolsFramework\ - contains the core ToolsFramework implementation

• MeshScene\ - a simple “Scene Graph” of Mesh Objects, which is what our mesh editing Tools will edit, and a basic History (ie undo/redo) system

• Interaction\ - basic user-interface interactions for object selection and transforming with a UTransformGizmo, built on top of the ITF

• Tools\ - subclasses of several MeshModelingToolset UInteractiveTools and/or Builders, necessary to allow them to function properly at Runtime

At a high level, here is how everything is connected, in plain english (hopefully this will make it easier to follow the descriptions below). A custom Game Mode, AToolsFrameworkDemoGameModeBase, is initialized on Play, and this in turn initializes the URuntimeToolsFrameworkSubsystem, which manages the Tools Framework, and the URuntimeMeshSceneSubsystem. The latter manages a set of URuntimeMeshSceneObject’s, which are wrappers around a mesh Actor and Component that can be selected via clicking and transformed with a UTransformGizmo. The URuntimeToolsFrameworkSubsystem initializes and owns the UInteractiveToolsContext, as well as various helper classes like the USceneObjectSelectionInteraction (which implements clicking selection), the USceneObjectTransformInteraction (which manages the transform Gizmo state), and the USceneHistoryManager (which provides the undo/redo system). The URuntimeToolsFrameworkSubsystem also creates a UToolsContextRenderComponent, which is used to allow PDI rendering in the Tools and Gizmos. Internally, the URuntimeToolsFrameworkSubsystem also defines the various API implementations, this is all fully contained in the cpp file. The final piece is the default Pawn for the Game Mode, which is a AToolsContextActor that is spawned by the GameMode on Play. This Actor listens for various input events and forwards them to the URuntimeToolsFrameworkSubsystem. A FSimpleDynamicMeshComponentTargetFactory is also registered on Play, which allows for the Mesh Component used in the URuntimeMeshSceneObject to be edited by existing Modeling Mode tools.

Whew! Since it’s relatively independent from the Tools Framework aspects, lets start with the Mesh Scene aspects.

URuntimeMeshSceneSubsystem and MeshSceneObjects

The purpose of this demo is to show selection and editing of meshes at Runtime, via the ITF. Conceivably this could be done such that any StaticMeshActor/Component could be edited, similar to how Modeling Mode works in the UE Editor. However, as I have recommended in previous tutorials, if you are building some kind of Modeling Tool app, or game Level Editor, I don’t think you want to build everything directly out of Actors and Components. At minimum, you will likely want a way to serialize your “Scene”. And you might want to have visible meshes in your environment that are not editable (if even just as 3D UI elements). I think it’s useful to have an independent datamodel that represents the editable world - a “Scene” of “Objects” that is not tied to particular Actors or Components. Instead, the Actors/Components are a way to implement desired functionality of these SceneObjects, that works in Unreal Engine.

So, that is what I’ve done in this demo. URuntimeMeshSceneObject is a SceneObject that is represented in the UE level by a ADynamicSDMCActor, which I described in previous tutorials. This Actor is part of the RuntimeGeometryUtils plugin. It spawns/manages a child mesh USimpleDynamicMeshComponent that can be updated when needed. In this project we will not be using any of the Blueprint editing functionality I previously developed, instead we will use the Tools to do the editing, and only use the SDMC as a way to display our source mesh.

URuntimeMeshSceneSubsystem manages the set of existing URuntimeMeshSceneObjects, which I will abbreviate here (and in the code) as an “SO”. Functions are provided to spawn a new SO, find one by Actor, delete one or many SOs, and also manage a set of selected SOs. In addition, the FindNearestHitObject() can be used to cast rays into the Scene, similar to a LineTrace (but will only hit the SOs).

The URuntimeMeshSceneSubsystem also owns the Materials assigned to the SO when selected, and the default Material. There is only baseline support for Materials in this demo, all created SOs are assigned the DefaultMaterial (white), and when selected are swapped to the SelectedMaterial (orange). However the SOs do track an assigned material and so you could relatively easily extend what is there now.

USceneHistoryManager

Changes to the Scene - SceneObject creation, deletion, and editing, Selection changes, Transform changes, and so on - are stored by the USceneHistoryManager. This class stores a list of FChangeHistoryTransaction structs, which store sequences of FChangeHistoryRecord, which is a tuple (UObject*, FCommandChange, Text). This system roughly approximates the UE Editor transaction system, however only explicit FCommandChange objects are supported, while in the Editor, changes to UObjects can be automatically stored in a transaction. I described FCommandChange in more detail above, in the IToolsContextTransactionsAPI section. Essentially these are objects that have Apply() and Revert() functions, which must “redo” or “undo” their effect on any modified global state.

The usage pattern here is to call BeginTransaction(), then AppendChange() one or more times, then EndTransaction(). The IToolsContextTransactionsAPI implementation will do this for ITF components, and things like the scene selection change will do it directly. The Undo() function rolls back to the previous history state/transaction, and the Redo() function rolls forward. Generally the idea is that all changes are grouped into a single transaction for a single high-level user “action”, so that one does not have to Undo/Redo multiple times to get “through” a complex state change. To simplify this, BeginTransaction()/EndTransaction() calls can be nested, this occurs frequently when multiple separate functions need to be called and each needs to emit it’s own transactions. Like any app that supports Undo/Redo, the History sequence is truncated if the user does Undo one or more times, and then does an action that pushes a new transaction/change.

AToolsContextActor

View fullsize

In an Unreal Engine game, the player controls a Pawn Actor, and in a first-person-view game the scene is rendered from the Pawn’s viewpoint. In the ToolsFrameworkDemo we will implement a custom ADefaultPawn subclass called AToolsContextActor to collect and forward user input to the ITF. In addition, this Actor will handle various hotkey input events defined in the Project Settings. And finally, the AToolsContextActor is where I have implemented standard right-mouse-fly (which is ADefaultPawn’s standard behavior, I am just forwarding calls to it) and the initial steps of Maya-style alt-mouse camera control (however orbit around a target point is not implemented, yet).

All the event connection setup is done in AToolsContextActor::SetupPlayerInputComponent(). This is a mix of hotkey events defined in the Input section of the Project Settings, and hardcoded button Action and mouse Axis mappings. Most of the hardcoded mappings - identifiable as calls to UPlayerInput::AddEngineDefinedActionMapping() - could be replaced with configurable mappings in the Project Settings.

This Actor is automatically created by the Game Mode on startup. I will describe this further below.

I will just mention here that another option, rather than having the Pawn forward input to the ITF’s InputRouter, would be to use a custom ViewportClient. The ViewportClient is “above” the level of Actors and Pawns, and to some degree is responsible for turning raw device input into the Action and Axis Mappings. Since our main goal as far as the ITF is concerned is simply to collect device input and forward it to the ITF, a custom ViewportClient might be a more natural place to do that. However, that’s just not how I did it in this demo.

URuntimeToolsFrameworkSubsystem

The central piece of the Runtime ITF back-end is the URuntimeToolsFrameworkSubsystem. This UGameInstanceSubsystem (essentially a Singleton) creates and initializes the UInteractiveToolsContext, all the necessary IToolsContextAPI implementations, the USceneHistoryManager, and the Selection and Transform Interactions, as well as several other helper objects that will be described below. This all occurs in the ::InitializeToolsContext() function.

The Subsystem also has various Blueprint functions for launching Tools and managing the active Tool. These are necessary because the ITF is not currently exposed to Blueprints. And finally it does a bit of mouse state tracking, and in the ::Tick() function, constructs a world-space ray for the cursor position (which is a bit of relatively obscure code) and then forwards this information to the UInputRouter, as well as Tick’ing and Render’ing the ToolManager and GizmoManager.

If this feels like a bit of a grab-bag of functionality, well, it is. The URuntimeToolsFrameworkSubsystem is basically the “glue” between the ITF and our “Editor”, which in this case is extremely minimal. The only other code of note are the various API implementations, which are all defined in the .cpp file as they are not public classes.

FRuntimeToolsContextQueriesImpl is the implementation of the IToolsContextQueriesAPI. This API provides the SelectionState to ToolBuilders, as well as supporting a query for the current View State and Coordinate System state (details below). The ExecuteSceneSnapQuery() function is not implemented and just returns false. However, if you wanted to support optional Transform Gizmo features like grid snapping, or snapping to other geometry, this would be the place to start.

FRuntimeToolsContextTransactionImpl is the implementation of the IToolsContextTransactionsAPI. Here we just forward the calls directly to the USceneHistoryManager. Currently I have not implemented RequestSelectionChange(), which some Modeling Mode Tools use to change the selection to newly-created objects, and also ignored PostInvalidation() calls, which are used in the UE Editor to force a viewport refresh in non-Realtime mode. Built games always run in Realtime, so this is not necessary in a standard game, but if you are building an app that does not require constant 60fps redraws, and have implemented a scheme to avoid repaints, this call can provide you with a cue to force a repaint to see live Tool updates/etc.

FRuntimeToolsFrameworkRenderImpl is the implementation of the IToolsContextRenderAPI. The main purpose of this API is to provide a FPrimitiveDrawInterface implementation to the Tools and Gizmos. This is one of the most problematic parts of using the Modeling Mode Tools at Runtime, and I will describe how this is implemented in the section below on the UToolsContextRenderComponent. Otherwise, functions here just forward information provided by the RuntimeToolsFrameworkSubsystem.

Finally FRuntimeToolsContextAssetImpl implements IToolsContextAssetAPI, which in our Runtime case is very limited. Many of the functions in this API are intended for more complex Editor usage, because the UE Editor has to deal with UPackages and Assets inside them, can do things like pop up internal asset-creation dialogs, has a complex system for game asset paths, and so on. Several of the functions in this API should perhaps not be part of the base API, as Tools do not call them directly, but rather call utility code that uses these functions. As a result we only need to implement the GenerateStaticMeshActor() function, which Tools do call, to emit new objects (for example the DrawPolygon Tool, which draws and extrudes a new mesh). The function name is clearly not appropriate because we don’t want to emit a new AStaticMeshActor, but rather a new URuntimeMeshSceneObject. Luckily, in many Modeling Mode Tools, the returned AActor type is not used - more on this below.

And that’s it! When I mentioned the “ITF Back-End” or “Editor-Like Functionality”, this is all I was referring to. 800-ish lines of extremely verbose C++, most of it relatively straightforward “glue” between different systems. Even quite a few of the existing pieces are not necessary for a basic ITF implementation, for example if you didn’t want to use the Modeling Mode Tools, you don’t need the IToolsContextAssetAPI implementation at all.

USceneObjectSelectionInteraction and USceneObjectTransformInteraction

When I introduced the ITF, I focused on Tools and Gizmos as the top-level “parts” of the ITF, ie the sanctioned methods to implement structured handling of user input (via InputBehaviors), apply actions to objects, and so on. However, there is no strict reason to use either Tools or Gizmos to implement all user interactions. To demonstrate this I have implemented the “click-to-select-SceneObjects” interaction as a standalone class USceneObjectSelectionInteraction.

USceneObjectSelectionInteraction subclasses IInputBehaviorSource, so it can be registered with the UInputRouter, and then it’s UInputBehaviors will be automatically collected and allowed to capture mouse input. A USingleClickInputBehavior is implemented which collects left-mouse clicks, and supports Shift+Click and Ctrl+Click modifier keys, to add to the selection, or toggle selection. The IClickBehaviorTarget implementation functions just determine what state the action should indicate, and apply them to the scene via the URuntimeMeshSceneSubsystem API functions. As a result, the entire click-to-select Interaction requires a relatively tiny amount of code. If you wanted to implement additional selection interactions, like a box-marquee select, this could be relatively easy done by switching to a UClickDragBehavior/Target and determining if the user has done a click vs drag via a mouse-movement threshold.

The URuntimeToolsFrameworkSubsystem simply creates an instance of this class on startup, registers it with the UInputRouter, and that’s all the rest of the system knows about it. It is of course possible to implement selection as a Tool, although generally selection is a “default” mode, and switching out-of/into a default Tool when any other Tool starts of exits requires a bit of care. Alternately it could be done with a Gizmo that has no in-scene representation, and is just always available when selection changes are supported. This would probably be my preference, as a Gizmo gets Tick() and Render() calls and that might be useful (for example a marquee rectangle could be drawn by Render()).

As the selection state changes, a 3D Transform Gizmo is continually updated - either it moves between the origin of the selected object, to the shared origin if there are multiple selected objects, or disappears if no object is selected. This behavior is implemented in USceneObjectTransformInteraction, which is similarly created by the URuntimeToolsFrameworkSubsystem. A delegate of URuntimeMeshSceneSubsystem, OnSelectionModified, is used to kick off updates as the scene selection is modified. The UTransformGizmo that is spawned acts on a UTransformProxy, which is given the current selection set. Note that any selection change results in a new UTransformGizmo being spawned, and the existing one destroyed. This is a bit heavy, and it is possible to optimize this to re-use a single Gizmo (various Modeling Mode Tools do just that).

One last note is the management of the active Coordinate System. This is handled largely under the hood, the UTransformGizmo will query the available IToolsContextQueriesAPI to determine World or Local coordinate frames. This could be hardcoded, but to support both, we need somewhere to put this bit of state. Currently I have placed it in the URuntimeToolsFrameworkSubsystem, with some BP functions exposed to allow the UI to toggle the option.

UToolsContextRenderComponent

I mentioned above that the IToolsContextRenderAPI implementation, which needs to return a FPrimitiveDrawInterface (or “PDI”) that can be used to draw lines and points, is a bit problematic. This is because in the UE Editor, the Editor Mode that hosts the ITF has it’s own PDI that can simply be passed to the Tools and Gizmos. However at Runtime, this does not exist, the only place we can get access to a PDI implementation is inside the rendering code for a UPrimitiveComponent, which runs on the rendering thread (yikes!).

If that didn’t entirely make sense, essentially what you need to understand is that we can’t just “render” from anywhere in our C++ code. We can only render “inside” a Component, like a UStaticMeshComponent or UProceduralMeshComponent. But, our Tools and Gizmos have ::Render() functions that run on the Game Thread, and are very far away from any Components.

So, what I have done is make a custom Component, called UToolsContextRenderComponent, that can act as a bridge. This Component has a function ::GetPDIForView(), which returns a custom FPrimitiveDrawInterface implementation (FToolsContextRenderComponentPDI to be precise, although this is hidden inside the Component). The URuntimeToolsFrameworkSubsystem creates an instance of this PDI every frame to pass to the Tools and Gizmos. The PDI DrawLine() and DrawPoint() implementations, rather than attempting to immediately render, store each function call’s arguments in a list. The Components SceneProxy then takes these Line and Point parameter sets and passes them on the standard UPrimitiveComponent PDI inside FToolsContextRenderComponentSceneProxy::GetDynamicMeshElements() implementation (which is called by the renderer to get per-frame dynamic geometry to draw).

This system is functional, and allows the Modeling Mode Tools to generally work as they do in the Editor. However one hitch is that the Game and Render threads run in parallel. So, if nothing is done, we can end up with GetDynamicMeshElements() being called before the Tools and Gizmos have finished drawing, and this causes flickering. Currently I have “fixed” this by calling FlushRenderingCommands() at the end of URuntimeToolsFrameworkSubsystem::Tick(), which forces the render thread to process all the outstanding submitted geometry. However, this may not fully resolve the problem.

One other complication is that in the UE Editor, the PDI line and point drawing can draw “hidden lines”, ie lines behind front-facing geometry, with a stipple pattern. This involves using Custom Depth/Stencil rendering in combination with a Postprocess pass. This also does not exist at Runtime. However, in your own application, you actually have more ability to do these kinds of effects, because you are fully in control of these rendering systems, while in the Editor, they need to be added “on top” of any in-game effects and so are necessarily more limited. This article gives a good overview of how to implement hidden-object rendering, as well as object outlines similar to the UE Editor.

FSimpleDynamicMeshComponentTarget

As I described above in the section on PrimitiveComponentTargets, to allow the mesh editing tools from Modeling Mode to be used in this demo, we need to provide a sort of “wrapper” around the UPrimitiveComponents we want to edit. In this case that will be USimpleDynamicMeshComponent. The code for FSimpleDynamicMeshComponentTarget, and it’s associated Factory, is relatively straightforward. You might notice, if you dive in, that the FDynamicMesh3 in the SDMC is being converted to a FMeshDescription to pass to the Tools, which then convert it back to a FDynamicMesh3 for editing. This is a limitation of the current design, which was focused on Static Meshes. If you are building your own mesh editing Tools, this conversion would not be necessary, but to use the Modeling Mode toolset, it is unavoidable.

Note that changes to the meshes (stored in ::CommitMesh()) are saved in the change history as FMeshReplacementChange, which stores two full mesh copies. This is not ideal for large meshes, however the mesh “deltas” that the modeling tools create internally to store changes on their preview meshes (eg in 3D sculpting) do not currently “bubble up”.

Finally, I will just re-iterate that because of the issues with Factory registration described in the section on FPrimitiveComponentTarget, it is not possible to directly edit UStaticMeshComponent or UProceduralMeshComponent at Runtime in UE4.26, with the Modeling Mode toolset. Although, since it’s largely only the ToolBuilders that use the FPrimitiveComponentTargetFactory registry, you might be able to get them to work with custom ToolBuilders that directly create alternate FPrimitiveComponentTarget implementations. This is not a route I have explored.

AToolsFrameworkDemoGameModeBase

The final C++ code component of the tutorial project is AToolsFrameworkDemoGameModeBase. This is a subclass of AGameModeBase, which we will configure in the Editor to be used as the default game mode. Essentially, this is what “launches” our Runtime Tools Framework. Note that this is not part of the RuntimeToolsFramework module, but rather the base game module, and there is no need for you to initialize things this way in your own app. For example, if you wanted to implement some kind of in-game level design/editing Tools, you would likely fold this code into your existing Game Mode (or perhaps launch a new one on demand). You also don’t need to use a Game Mode to do this, although a complication in that case is the default pawn AToolsContextActor, which might need to be replaced too.

Very little happens in this Game Mode. We configure it to Tick, and in the Tick() function, we Tick() the URuntimeToolsFrameworkSubsystem. Otherwise all the action is in AToolsFrameworkDemoGameModeBase::InitializeToolsSystem(), where we initialize the URuntimeMeshSceneSubsystem and URuntimeToolsFrameworkSubsystem, and then register the set of available Tools with the ToolManager. All this code could (and perhaps should) be moved out of the Game Mode itself, and into some utility functions.

ToolsFrameworkDemo Project Setup

If you are planning to set up your own Project based on this tutorial, or make changes, there are various assets involved, and Project Settings, that you need to be aware of. The Content Browser screenshot below shows the main Assets. DefaultMap is the level I have used, this simply contains the ground plane and initializes the UMG User Interface in the Level Blueprint (see below).

BP_ToolsContextActor is a Blueprint subclass of AToolsContextActor, which is configured as the Default Pawn in the Game Mode. In this BP Actor I have disabled the Add Default Movement Bindings setting, as I set up those bindings manually in the Actor. DemoPlayerController is a Blueprint subclass of AToolsFrameworkDemoPlayerController, again this exists just to configure a few settings in the BP, specifically I enabled Show Mouse Cursor so that the standard Windows cursor is drawn (which is what one might expect in a 3D Tool) and disabled Touch Events. Finally DemoGameMode is a BP subclass of our AToolsFrameworkDemoGameModeBase C++ class, here is where we configure the Game Mode to spawn our DemoPlayerController and BP_ToolsContextActor instead of the defaults.

BP_ToolsContextActor Settings

DemoPlayerController Settings

DemoGameMode Settings

Finally in the Project Settings dialog, I configured the Default GameMode to be our DemoGameMode Blueprint, and DefaultMap to the the Editor and Game startup map. I also added various actions in the Input section, I showed a screenshot of these settings above, in the description of AToolsContextActor. And finally in the Packaging section, I added two paths to Materials to the Additional Asset Directories to Cook section. This is necessary to force these Materials to be included in the built Game executable, because they are not specifically referenced by any Assets in the Level.

View fullsize

Packaging settings - these force Material assets to be included in the built game

RuntimeGeometryUtils Updates

In my previous tutorials, I have been accumulating various Runtime mesh generation functionality in the RuntimeGeometryUtils plugin. To implement this tutorial, I have made one significant addition, URuntimeDynamicMeshComponent. This is a subclass of USimpleDynamicMeshComponent (SDMC) that adds support for collision and physics. If you recall from previous tutorials, USimpleDynamicMeshComponent is used by the Modeling Mode tools to support live previews of meshes during editing. In this context, SDMC is optimized for fast updates over raw render performance, and since it is only used for “previews”, does not need support for collision or physics.

However, we have also been using SDMC as a way to render runtime-generated geometry. In many ways it is very similar to UProceduralMeshComponent (PMC), in that respect, however one significant advantage of PMC was that it supported collision geometry, which mean that it worked properly with the UE raycast/linetrace system, and with the Physics/Collision system. It turns out that supporting this is relatively straightforward, so I created the URuntimeDynamicMeshComponent subclass. This variant of SDMC, I guess we can call it RDMC, supports simple and complex collision, and a function SetSimpleCollisionGeometry() is available which can take arbitrary simple collision geometry (which even PMC does not support). Note, however, that currently Async physics cooking is not supported. This would not be a major thing to add, but I haven’t done it.

I have switched the Component type in ADynamicSDMCActor to this new Component, since the functionality is otherwise identical, but now the Collision options on the base Actor work the same way they do on the PMC variant. The net result is that previous tutorial demos, like the bunny gun and procedural world, should work with SDMC as well as PMC. This will open the door for more interesting (or performant) runtime procedural mesh tools in the future.

Using ModelingMode Tools at Runtime

It’s taken quite a bit of time, but we are now at the point where we can expose existing mesh editing Tools in the MeshModelingToolset in our Runtime game, and use them to edit selected URuntimeMeshSceneObject’s. Conceptually, this “just works” and adding the basic ability for a Tool to work only requires registering a ToolBuilder in AToolsFrameworkDemoGameModeBase::RegisterTools(), and then adding some way (hotkey, UMG button, etc) to launch it via the URuntimeToolsFrameworkSubsystem::BeginToolByName(). This works for many Tools, for example PlaneCutTool and EditMeshPolygonsTool worked out-of-the-box.

However, not all Tools are immediately functional. Similar to the global ToolTargetTargetFactory system, various small design decisions that likely seemed insignificant at the time can prevent a Tool from working in a built game. Generally, with a bit of experimentation it is possible to work around these problems with a small amount of code in a subclass of the base Tool. I have done this in several cases, and I will explain these so that if you try to expose other Tools, you might have a strategy for what to try to do. If you find yourself stuck, please post in the Comments with information about the Tool that is not working, and I will try to help.

Note that to make a Tool subclass, you will also need to make a new ToolBuilder that launches that subclass. Generally this means subclassing the base Builder and overriding a function that creates the Tool, either the base ::BuildTool() or a function of the base Builder that calls NewObject<T> (those are usually easier to deal with).

In several cases, default Tool Settings are problematic. For example the URemeshTool by default enables a Wireframe rendering that is Editor-Only. So, it is necessary to override the Setup() function, call the base Setup(), and then disable this flag (There is unfortunately no way to do this in the Builder currently, as the Builder does not get a chance to touch the Tool after it allocates a new instance).

Tools that create new objects, like UDrawPolygonTool, generally do not work at Runtime without modification. In many cases the code that emits the new object is #ifdef’d out, and a check() is hit instead. However we can subclass these Tools and replace either the Shutdown() function, or an internal function of the Tool, to implement the new-object creation (generally from a FDynamicMesh3 the Tool generated). URuntimeDrawPolygonTool::EmitCurrentPolygon() is an example of doing this for the UDrawPolygonTool, and URuntimeMeshBooleanTool::Shutdown() for the UCSGMeshesTool. In the latter case, the override performs a subset of the base Tool code, as I only supported replacing the first selected Input object.

These are the two main issues I encountered. A third complication is that many of the existing Tools, particularly older Tools, do not use the WatchProperty() system to detect when values of their UInteractiveToolPropertySet settings objects have been modified. Instead of polling, they depend on Editor-only callbacks, which do not occur in a built game. So, if you programmatically change settings of these PropertSets, the Tool will not update to reflect their values without a nudge. However, I have coupled those “nudges” with a way to expose Tool Settings to Blueprints, which I will now explain.

Blueprint-Exposed ToolPropertySets

One major limitation of the Tools Framework in 4.26 is that although it is built out of UObjects, none of them are exposed to Blueprints. So, you cannot easily do a trivial thing like hook up a UMG UI to the active Tool, to directly change Tool Settings. However if we subclass an existing Tool, we can mark the subclass as a UCLASS(BlueprintType), and then cast the active Tool (accessed via URuntimeToolsFrameworkSubsystem::GetActiveTool()) to that type. Similarly we can define a new UInteractiveToolPropertySet, that is also UCLASS(BlueprintType), and expose new UProperties marked BlueprintReadWrite to make them accessible from BP.

To include this new Property Set, we will then subclass the Tool ::Setup() function, call the base-class ::Setup(), and then create and register our new PropertySet. For each property we will add a WatchProperty() call that forwards changes from our new PropertySet to the base tool Settings, and then if necessary call a function to kick off a recomputation or update (for example URuntimeMeshBooleanTool will have to call Preview->InvalidateResult()).

One complication is enum-valued Settings, which in the Editor will automatically generate dropdown lists, but this is not possible with UMG. So, in those cases I used integer UProperties and mapped integers to enums myself. So, for example, here is all the PropertySet-related code for the URuntimeDrawPolygonTool of UDrawPolygonTool (I have omitted the EmitCurrentPolygon() override and new ToolBuilder that I mentioned above). This is a cut-and-paste pattern that I was able to re-use in all my Tool overrides to expose Tool Properties for my UMG UI.

UENUM(BlueprintType)
enum class ERuntimeDrawPolygonType : uint8
{
    Freehand = 0, Circle = 1, Square = 2, Rectangle = 3, RoundedRectangle = 4, HoleyCircle = 5
};

UCLASS(BlueprintType)
class RUNTIMETOOLSSYSTEM_API URuntimeDrawPolygonToolProperties : public UInteractiveToolPropertySet
{
    GENERATED_BODY()
public:
    UPROPERTY(BlueprintReadWrite)
    int SelectedPolygonType;
};

UCLASS(BlueprintType)
class RUNTIMETOOLSSYSTEM_API URuntimeDrawPolygonTool : public UDrawPolygonTool
{
    GENERATED_BODY()
public:
    virtual void Setup() override;

    UPROPERTY(BlueprintReadOnly)
    URuntimeDrawPolygonToolProperties* RuntimeProperties;
};

void URuntimeDrawPolygonTool::Setup()
{
    UDrawPolygonTool::Setup();

    // mirror properties we want to expose at runtime 
    RuntimeProperties = NewObject<URuntimeDrawPolygonToolProperties>(this);
    RuntimeProperties->SelectedPolygonType = (int)PolygonProperties->PolygonType;
    RuntimeProperties->WatchProperty(RuntimeProperties->SelectedPolygonType,
        [this](int NewType) { PolygonProperties->PolygonType = (EDrawPolygonDrawMode)NewType; });

    AddToolPropertySource(RuntimeProperties);
}

ToolPropertySet Keepalive Hack

One major hitch I ran into in trying to get the MeshModelingToolset Tools to work in a built game is that it turns out that they do something…illegal…with UObjects. This really gets into the weeds, but I’ll explain it briefly in case it is relevant to you. I previously mentioned that UInteractiveToolPropertySet is used to expose “Tool Settings” in a structured way in nearly all the Tools. One desirable property of a system like this is to be able to save the state of Settings between Tool invocations. To do this, we can just hold on to an instance of the Property Set itself, but we need to hold it somewhere.

Various Editor systems do this by holding a pointer to the saved settings UObject in the CDO of some other UObject - each UObject has a CDO (Class Default Object) which is like a “template” used to construct additional instances. CDOs are global and so this is a handy place to put things. However, in the Editor the CDO will keep this UObject from being Garbage Collected (GC’d), but at Runtime, it will not! And in fact at Runtime, the Garbage Collector does a safety check to determine that this has not been done, and if it detects this, kills the game (!). This will need to be fixed in future versions of UE, but for this demo to function in a binary 4.26 build, we will need a workaround.

First, I had to disable the GC safety check by setting the global GShouldVerifyGCAssumptions = false in URuntimeToolsFrameworkSubsystem::InitializeToolsContext(). This prevents the hard kill, but the saved PropertySets will still be Garbage-Collected and result in crashes later, when the Tool tries to access them and assumes they still exist. So, in the URuntimeToolsFrameworkSubsystem::OnToolStarted() event handler, the AddAllPropertySetKeepalives() function is called, which iterates through the CDOs of all the registered PropertySet UObjects of the new Tool, and adds these “saved settings” UObjects to a TArray that will prevent them from being GC’d.

This is…a gross hack. But fully functional and does not appear to have any problematic side-effects. But I intend to resolve the underlying architectural issues in the future.

User Interface

The point of this tutorial was to demonstrate at-runtime usage of the Interactive Tools Framework and Mesh Modeling Toolset, not to actually build a functional runtime modeling tool. However, to actually be able to launch and use the Tools for the demo, I had to build a minimal UMG user interface. I am not an expert with UMG (this is the first time I’ve used it) so this might not be the best way to do it. But, it works. In the /ToolUI subfolder, you will find several UI widget assets.

ToolTestUI is the main user interface, which lives in the upper-left corner, there is an image below-right. I described the various Tool buttons at the start of the Tutorial. The Accept, Cancel, and Complete buttons dynamically update visibility and enabled-ness, based on the active Tool state, this logic is in the Blueprint. Undo and Redo do what you expect, and the World button toggles between World and Local frames for any active Gizmos. This UI is spawned on BeginPlay by the Level Blueprint, below-right.

View fullsize

There are also several per-tool UI panels that expose Tool settings. These per-Tool UI panels are spawned by the ToolUI buttons after they launch the Tool, see the ToolUI Blueprint, it’s very straightforward. I have only added these settings panels for a few of the Tools, and only exposed a few settings. It’s not really much work to add settings, but it is a bit tedious, and since this is a tutorial I wasn’t too concerned with exposing all the possible options. The screenshots below are from the DrawPolygonToolUI, showing the in-game panel (left) and the UI Blueprint (right). Essentially, on initialization, the Active Tool is cast to the correct type and we extract the RuntimeProperties property set, and then initialize all the UI widgets (only one in this case). Then on widget event updates, we forward the new value to the property set. No rocket science involved.

sincere Apologies for the terrible UI layout and sizing.

View fullsize

Final Notes

I have had many people ask about whether the UE Editor Modeling Mode Tools and Gizmos could be used at Runtime, and my answer has always been “well, it’s complicated, but possible”. I hope this sample project and write-up answers the question! It’s definitely possible, and between the GeometryProcessing library and MeshModelingToolset tools and components, there is an enormous amount of functionality available in UE4.26 that can be used to build interactive 3D content creation apps, from basic “place and move objects” tools, to literally a fully functional 3D mesh sculpting app. All you really need to do is design and implement the UI.

Based on the design tools I have built in the past, I can say with some certainty that the current Modeling Mode Tools are probably not exactly what you will need in your own app. They are a decent starting point, but really what I think they provide is a reference guide for how to implement different interactions and behaviors. Do you want a 3D workplane you can move around with a gizmo? Check out UConstructionPlaneMechanic and how it is used in various Tools. How about drawing and editing 2D polygons on that plane? See UCurveControlPointsMechanic usage in the UDrawAndRevolveTool. An interface for drawing shortest-edge-paths on the mesh? USeamSculptTool does that. Want to make a Tool that runs some third-party geometry processing code, with settings and a live preview and all sorts of useful stuff precomputed for you? Just subclass UBaseMeshProcessingTool. Need to run an expensive operation in a background thread during a Tool, so that your UI doesn’t lock up? UMeshOpPreviewWithBackgroundCompute and TGenericDataBackgroundCompute implement pattern, and Tools like URemeshMeshTool show how to use it.

I could go on, for a long time. There are over 50 Tools in Modeling Mode, they do all sorts of things, far more than I could possibly have time to explain. But if you can find something close to what you want to do in the UE Editor, you can basically copy the Tool .cpp and .h, rename the types, and start to customize it for your purposes.

So, have fun!

Procedural Mesh Blueprints in UE4.26

no static meshes were cooked in the making of this level. Click to play video.

In my previous tutorial, I showed you how to do runtime mesh manipulation in Unreal Engine 4.26 using various components (UStaticMeshComponent, UProceduralMeshComponent, and USimpleDynamicMeshComponent) by way of the GeometryProcessing plugin’s mesh editing capabilities and a sort of “wrapper” Actor called ADynamicMeshBaseActor. An obvious thing that you might want to use this machinery for is to do procedural geometry generation, either in-Editor or in-Game. And since ADynamicMeshBaseActor exposed Blueprint functions for things like mesh generators, Mesh Booleans with another ADynamicMeshBaseActor, and so on, one could imagine building up a procedural system where you generate Actors and combine them to build up complex geometry.

Here’s the problem though: Actors are expensive. Say you want to do something like use an L-System to create plants, or buildings, or whatever. These systems tend to involve lots of small meshes. Making an Actor for each of those small meshes would be…painful.

So, in this tutorial I’m going to show you one way to implement a more effective procedural mesh generation system in Blueprints. I’m not sure this approach is the best way to do it, and it has some serious pitfalls. But the result is that the small level you see on the right is 100% generated by Blueprints - there is not a single static mesh in this project. All the geometry is built up by combining simple primitives, doing operations like Booleans and mesh-wrapping and applying displacement noise, and there is even a simple all-in-Blueprints scattering system for placing rocks on the asteroids. The textures are not procedural, I made them in Quixel Mixer. But they are applied without UV maps, so that’s kinda procedural! And I threw in a little Niagara particle system at the end, to spice up your escape from the Asteroid.

(Mandatory Disclaimer: your author, Ryan Schmidt, is an employee of Epic Games. However, gradientspace.com is his personal website and this article represents his personal thoughts and opinions. In particular, this tutorial code and content is not part of Unreal Engine, and it isn’t supported by Epic Games.)

Getting and Running the Sample Project

Before we begin, this tutorial is for UE 4.26, currently in Preview release (Preview 7 at time of writing). You can install the Preview binaries from the Epic Games Launcher.

The project for this tutorial is on Github in UnrealProceduralMeshesDemo repository (MIT License). I did not include this in the same repository as my previous tutorials because there are ~100mb of binary files. In UE 4.26 this project will probably only work on Windows due to the inclusion of the DynamicSDMCActor class in the RuntimeGeometryUtils plugin. However I did not use any DynamicSDMCActors in this project, so you should be able to make it this demo work on OSX/Linux by deleting the DynamicSDMCActor .cpp and .h files - see the notes at the end of the previous tutorial (If you try that and it works, please send me a note so I can mention it here).

Once you are in the top-level folder, right-click on ProceduralMeshesDemo.uproject in Windows Explorer and select Generate Visual Studio project files from the context menu. This will generate ProceduralMeshesDemo.sln, which you can use to open Visual Studio. You can also open the .uproject directly in the Editor (it will ask to compile), but you probably will want to refer to the C++ code for this tutorial.

Build the solution and start (press F5) and the Editor should open into the sample map. You can test the project in PIE using the large Play button in the main toolbar, or click the Launch button to build a cooked executable. This will take a few minutes, after which the built game will pop up in a separate window. Try to escape the level! You’ll have to press Alt+F4 to exit as there is no menu/UI.

UGeneratedMesh

At the end of the last tutorial, ADynamicMeshBaseActor had some basic Blueprint functions you could use to do single mesh operations, like subtract another Actor, and so on. But to do real mesh generation, we need lots of temporary meshes. Consider something trivial like a “stairs generator”. In it’s simplest form, you would make a box, and then append a bunch of translated copies of that box, each shifted up-and-forward from the last. What I want to be able to do is have a BP_ProceduralStairs Actor, that has parameters for # of steps, width, height, etc, and it’s Blueprint does the mesh generation.

To do this kind of thing, we’re going to need temporary meshes. I want to make a temporary mesh, append a bunch of other meshes to it (which might be temporary meshes themselves), and then transfer that mesh to my ADynamicMeshBaseActor’s Source Mesh. So the question is, how to handle the temporary meshes in Blueprints? In Normal C++ we would just make FDynamicMesh3 objects, use them, and throw them away. But in Blueprints, we can only “use” the meshes by passing them between Blueprint functions. And to pass data between BP Functions, we either have to have them be UStructs or UObjects.

The issue with using UStructs is that Blueprints pass UStructs by value, and are very liberal with making copies of UStructs. So what that means is, if we put our FDynamicMesh3 inside a UStruct, it gets copied. A lot. There are ways to cheat to avoid the copies, like stuffing non-UProperty pointers inside the structs, but to use the cheats safely means that the person making the Blueprints has to understand what going to happen in C++. This is a Bad Idea.

So, our other option is a UObject. UObjects are much heavier than UStructs, and they are managed via garbage collection. What that means is, when a UObject is no longer referenced by any UProperties, it will be deleted (eventually). This garbage collection is going to cause some complications. But otherwise, this approach basically works. I have created a class UGeneratedMesh that is similar to the core of ADynamicMeshBaseActor - it has a FDynamicMesh3, as well as FDynamicMesh3AABBTree and TFastWindingTree for spatial queries.

What it doesn’t have is a Material, Transform, or any of that stuff. It’s just a container for a mesh, with a set of UFunctions to manipulate that mesh. The set of functions is quite large, it includes a bunch of primitive generation functions (Box, Sphere, Cylinder, Cone, Torus, Revolves, and Extrudes), functions to Append and Boolean other UGeneratedMeshes, Cut with a plane, Mirror, Solidify, Simplify, Transform, Recompute Normals, and various Spatial Query functions. It can do a lot of things! And you can spawn them in Blueprints as needed, to do node-graph-style geometry creation.

My basic stacked-boxes Stairs Generator is below. This is all happening in the Construction Script on an ADynamicMeshBaseActor Blueprint called BP_ProceduralStairs in the sample project. I have two temporary meshes, Step Mesh and Stairs Mesh. The Append Box function adds a box to the Step Mesh, and then the Append Tiled function adds Step Mesh to Stairs Mesh multiple times, applying the input Transform each repetition. Finally Copy from Mesh is a function of the ADynamicMeshBaseActor, that copies the UGeneratedMesh geometry into the Actor, which then goes on to it’s Component (I will use ProceduralMeshComponent for everything in this tutorial, we’ll get to that).

View fullsize

All the parameters above are exposed on the Actor, so in the Editor I can select a BP_ProceduralStairs and change it’s various dimensions and number of steps in the Actor Details panel.

Our Nemesis: Garbage Collection

There is one fly in the ointment here, though. In the BP screenshot above, I cropped off the top. That’s where I need to create the UGeneratedMesh instances for StepMesh and StairsMesh. The standard way to do this would be to use the BP version of the C++ NewObject<> function, which spawns a new UGeneratedMesh UObject. The Blueprint nodes below-left do that (click to enlarge). This works fine, until you open up the Task Manager and look at memory usage as you drag one of the parameters in the Details Panel. Look at it go - hundreds of megabytes in a few seconds!

View fullsize

What is happening here is that the temporary UObject meshes we spawned with Construct Generated Mesh are not deleted when the Construction Script completes. They become unreferenced, but they will hang around until the Garbage Collector (GC) runs, which is about once a minute in UE4.26. And sure enough, if you wait for a while, memory usage will drop back down. But this is not acceptable. We could force the GC to run, but this is not a good practice, and creating and then destroying all these UObjects is a lot of overhead.

Generated Mesh Pooling

What to do? Well, this is not a new problem. It comes up all the time, and particularly in procedural systems where you are going to do the same thing over and over as the user interactively changes some parameter. The general way to address this is Caching of some kind or other, and for this specific situation the standard strategy is Pooling. Which means, basically, re-use the things you have already created/allocated instead of making new ones all the time.

Re-using UObjects is a bit tricky in Blueprint Construction Scripts, where we ideally want to do our mesh generation (otherwise it will not be visible in the Editor). We can’t just add some generated-mesh pool UObject in the Construction Script because it will get re-created on each run. So, I put it in the ADynamicMeshBaseActor. There is a UGeneratedMeshPool class, and a UProperty instance of this, but it is not exposed to Blueprints directly. Instead there is a BP function ADynamicMeshBaseActor::AllocateComputeMesh() which will give you a UGeneratedMesh you can use, either by allocating a new one or providing one from the pool. So now our Blueprint looks like below, and we see that memory usage stays reasonable as we edit an instance of the BP_ProceduralStairs Actor:

View fullsize

View fullsize

But, we’re actually not done yet. If you look up at the Stairs Generator BP, you’ll see that the Exec path also goes off to the right. What’s over there? Well, generally when you do Pooling you have to explicitly give any objects you took back to the Pool, and it’s no different here. ADynamicMeshActor also has several functions to return the UGeneratedMesh’s we have taken ownership of. These must be called at the end of the BP, otherwise the mesh will hang around forever! The screenshot below shows two possible paths, there is a Release Compute Meshes function that lets you explicitly release meshes (there is also Release Compute Mesh for a single mesh), as well as a simpler Release All Compute Meshes function that tells the Pool to take back all the meshes it has given out. I tend to use this one when generating meshes in construction scripts, as it requires the least effort, but be aware that it means you can’t hold on to any of the meshes.

This “Return to Pool” mechanism is the one big gotcha of this UGeneratedMesh architecture. I haven’t figured out a way around it. If you forget to release your compute meshes, things could spiral out of control quickly! So, to try to avoid catastrophe, if the pool grows to over 1000 UGeneratedMeshes it will clear the pool and run garbage collection.

DynamicMeshBaseActor Improvements

This tutorial will use the ADynamicMeshBaseActor I created in the previous tutorial to represent procedural meshes in the Level. I have made some improvements, though. First of all, I grouped the various UProperties under a DetailsView category called “Dynamic Mesh Actor”. I have added a new Source Type called Externally Generated. This tells the Actor that you will populate it’s Source Mesh yourself - if you don’t set it to this value, then it will overwrite your generated mesh with a Primitive or Import.

I also exposed a new Runtime Collision mode named Simple Convex Hull. When set to this mode, the DynamicPMCActor subclass will calculate convex hull “simple collision” for the mesh whenever it is updated. This convex hull will optionally be simplified to the Max Hull Triangles triangle count (set to zero to use the full hull). If you use this mode, then it is possible to enable Simulate Physics on the ProceduralMeshComponent and it will work!

We will use this option in a few places. Note, however, that generating the convex hull can be very expensive if the input mesh is large - ie several seconds if you have many thousands of triangles. It might make sense to have some thresholds, ie don’t compute the hull if the mesh is too large, or randomly sample a subset of vertices. I haven’t done any of that, but I would take a PR!

Note that I exclusively used DynamicPMCActor in this project. This is because only the ProceduralMeshComponent supports runtime cooking of Simple or Complex collision, and only with PhysX (the default physics library in UE 4.26 - for now!).

UGeneratedMesh Procedural Mesh Generation API

View fullsize

I won’t say much about the actual UGeneratedMesh Blueprint API, you can skim the header or browse the Generated Mesh category in the BP Editor (search “generatedmesh” in the BP function search bar) to see what is available. There is no particular strategy for completeness, I implemented what I needed to make the tutorial. Most of the functions are just wrappers around calls to the GeometryProcessing plugin.

One slightly unusual thing I did in the BP functions is have most of them return a UGeneratedMesh that is actually just the input mesh. It’s very important to note that this is not a copy. That’s why the output pin is labeled “Input Mesh”. Because a lot of the mesh processing is a sequence of functions applied to the same UObject, I found that this let me make BPs which were much clearer to read, vs having a far-away output feeding multiple inputs. However these output pins are optional and you don’t have to use them!

Another important note: there are not many safety checks in these functions. If you pass in arguments that will produce a huge multi-gigabyte mesh, the code will happily do it and you will have to kill the Editor. It’s kind of tricky to put in these kinds of limits, because what one person considers reasonable may be an eternity to another.

Blueprint Function Libraries

I have placed most of the mesh editing functions directly on UGeneratedMesh. However, this strategy will lead to enormous classes which can be difficult to manage. It also doesn’t work to “extend” the base class with additional functions, which is particularly a challenge if you have your own third-party libraries and you want to use them in some BP function implementations. To handle this kind of situation, we can use a Blueprint Function Library. Making a BP Library is very simple, just define a C++ class that subclasses UBlueprintFunctionLibrary, and then add static UFunctions that are BlueprintCallable. Those functions will then be available in the BP Graph Editor.

I did this for the various mesh deformers that are used below (Sin Wave Displacement, Perlin Noise, Mesh Smoothing, etc). The static UFunctions functions are in the UGeneratedMeshDeformersLibrary class. Organizing things this way allows you to add your own functions that operate on a UGeneratedMesh, and keep your C++ code better-organized (I should have done more of the UGeneratedMesh API this way!)

Blueprint Procedural Mesh Generators

Ok, on to the tutorial. I have already explained the Stairs generator above. In the tutorial there are various other Procedural Mesh BP Actors that I will briefly explain here

BP_Wall

This Blueprint generates a basic rectangular box with configurable width/height/depth, and an optional angled top, created using a plane cut. The angled top allows this BP_Wall to also be used for creating ramps. There really is not much to it. Most of the nodes below are converting the input parameters into the necessary input vectors, rotations, transforms, and so on. So far I have found that this part tends to be more work than setting up the actual mesh operations. That said, I’m also not a Blueprint guru so maybe there are more efficient ways to accomplish some of these basic geometric constructions.

The only thing I wanted to point out below is the Set To Face Normals call. In most of the editing operations, I am not automatically recomputing normals. This is to avoid the cost of repeated normal computation if you are chaining multiple operations together. Note that Copy from Mesh has a Recompute Normals boolean. This does work, but it does not have the ability to compute normals with creases based on a threshold angle, which can be done with the UGeneratedMesh function SetToAngleThresholdNormals (which is not necessary on a wall like this, where Face Normals are sufficient, but I used it in various others below)

View fullsize

BP_Rock

This is another relatively simple generator, that creates a reasonable rock or boulder by displacing the vertices of a sphere using Perlin Noise. Hard Normals are then computed with an angle threshold, which produces some sharp edges and, combined with a nice triplanar texture, makes for a reasonably convincing rock. This is quite a cheap generator, and with a bit of additional effort the rocks could be randomly scaled, and/or multiple frequencies of Noise could be used to produce a wider range of shapes. Note that the Noise is defined by an explicit RandSeed parameter - if you don’t change this, you’ll get the same rock every time. In this kind of procedural generator, we generally want reproducible results so it is necessary to use explicit random seeds.

For these rocks, I have set the Collision Mode to Simple Convex Hull in the Dynamic Mesh Actor properties. This allows the rocks to be simulated, which is why they can drop out of the pipe and roll down the stairs.

Note that in the BP below, the construction script just calls a BP-defined function called Generate Rock. This is a useful way to organize procedural mesh generators, and allows for “sharing code” between the Construction Script and Event Graph (a common thing to want in procedural generators).

View fullsize

The pipe the rocks fall out of is also procedurally generated in BP_RockEmitter, by creating two cylinders and then subtracting a third. There is nothing particularly new in that generator that isn’t in the others, so I did not include it here. BP_RockEmitter also has some Tick functionality, as it spawns new instances of BP_Rock at regular intervals, but this is pretty standard Blueprint gameplay stuff so I won’t explain it here.

BP_DynamicBridge

In the tutorial project level, once you’ve made it past the barrier wall, you’ll have to correctly time a run across a walkway that cycles back and forth to the far platform, as shown on the right. This is a procedural mesh being regenerated each frame, with the “bridge length” parameter being driven based on elapsed game time.

The geometry generation is not that complex, just a box with two Sin wave deformers applied to it. However the BP is structured somewhat differently than BP_Rock, in a way that I think is better. The geometry is created in a BP Function called Generate Bridge. However instead of allocating and releasing it’s own compute mesh, Generate Bridge takes the UGeneratedMesh as an argument, and returns it at the end of the function. This means that if you wanted to re-use this code, you can easily cut-and-paste it into any other BP without having to worry about the Copy from Mesh and Release All Compute Meshes at the end of Generate Rock above (which might conflict with other geometry generation you want to do).

Another benefit to putting the generation code in a BP Function is you can then easily run the same code in-Editor and in-Game. The Construction Script (below-middle) and the Event Graph Tick version (at the bottom) are basically the same, except for the time-dependent animation of the length parameter.

View fullsize

BP_BlobGenerator

The Blueprint that creates the two large asteroid pieces is BP_BlobGenerator, and it has quite a lot going on. I won’t fully explain every node, it but here’s a simplified description of what happens along the exec flow below

1. Create a mesh called Subtract Shape (a sphere in this case)

2. Create another Sphere and deform it with two Sin Wave deformers, this is our final Accumulated mesh we will build up

3. For Loop to repeat the following “NumBlobs” times

   1. Generate a random point on a sphere

   2. find the closest point on the Accumulated mesh

   3. subtract the Subtract Shape from the Accumulated mesh at that point, with a random scale

4. Update the Actor’s source mesh

5. Optionally apply the Make Solid BP function below

By default, BP_BlobGenerator has no collision enabled, however the asteriods in the tutorial level are configured to use Complex as Simple collision. This works quite well to create landscape-type elements that are not too high-resolution. In the level, I increased the simplification triangle count to 4000, because at large scale things felt a bit too triangulated. A pass of mesh smoothing might also have helped to reduce that.

View fullsize

The Make Solid function is straightforward, basically we are going to copy the current Source Mesh from the Actor, call the Solidify function, apply Perlin noise displacement if necessary, and then Simplify the result and re-update the Actor Source Mesh. This generates a more compelling rock-like/organic object, by removing all the sharp creases and “perfect” geometry.

One issue with using this BP in practice is that the Solidify and Simplify functions are (relatively) slow, for real-time use. This is all vastly more expensive than BP_Rock, for example. So, in practice when I am positioning and configuring one of these Actors, I would disable the Solidify setting in the “Blob Generator” DetailsView properties, and then toggle it on to see the final result. However I did one other thing in this BP, which is to enable the “Call in Editor” flag on the MakeSolid BP function (shown below-right). This leads to the BP Function being exposed as a button labeled “Make Solid” in the Blob Generator Details panel section (shown below-left). If you click that button, the Make Solid function will be applied. This is why I in the Make Solid function I initialized a GeneratedMesh from the current Actor Source Mesh - so that it can be called without re-running the Construction Script.

View fullsize

These kinds of UI-exposed functions are extremely useful as a way to extend the capabilities of an Actor in the Editor. For example it might be nice to have a button on the ADynamicMeshActorBase that could emit a new StaticMeshActor - it would effectively turn these Procedural Mesh BPs into asset authoring Tools in the Editor. Maybe something to tackle in the future. However, do be aware that the next time the Construction Script runs it will wipe out any result created by incrementally updating the Source Mesh!

BP_SpaceShip

View fullsize

At the end of the tutorial video, the character jumps off the edge of the asteroid, lands on a ship, and flies away. Making this ship was the most time-consuming part of creating the demo, but in many ways it’s both visually and technically the least interesting. The BP is below, you can click to see a larger version, but you’ll discover that’s really just a straight-ahead generate-primitives-and-append-them situation. The generator could be parameterized in many ways, which might make it more useful. But I think I would have been able to hand-model a more interesting ship in much less time.

This BP does have a nice Niagara particle system attached, though, that emits particles from the rear engine when they fire up. I followed the “Create a GPU Sprite Effect” tutorial on from the UE4 documentation to start, and then made changes. It was very straightforward and looks pretty good, I think.

View fullsize

BP_RockScatter

This Actor is the most complex in this tutorial as it isn’t really a single “object”, but rather a generator for a set of rocks covering the asteroid surface. The basic idea here is not complicated - I am going to generate random points on a 3D plane above the asteroid, shoot rays downwards, and if the rays hit a relatively flat spot, I’ll add a new randomly-deformed rock at that location.

That might sounds costly, but actually this actor is surprisingly inexpensive - it updates very quickly as I drag it in the Editor viewport. This is because the computation involved amounts to a raycast and then a mesh copy. It’s much cheaper than even a single mesh boolean, let alone voxel operations or simplification. All the rocks are accumulated into a single mesh, so this doesn’t add a tick or draw call per-rock (however it also means there is no instanced rendering). That said, if I crank up the rock count, things will start to get chunky (on my quite powerful computer, 50 rocks is interactive but 200 takes a second or so to update).

Note that this definitely isn’t the most efficient way to implement scattering like this. At minimum all the ray-shooting and mesh generation/accumulation could easily be ported to a single C++ UFunction, and much of it done in parallel. But it’s kind of nice to prototype it in Blueprints, where an artist can easily go in and make changes.

View fullsize

You might note in the gif above there is a yellow box that is moving with the scattered rocks. This is a Box Collision component that I added in the Actor to define the scattering bounds, as shown in the BP editor screenshot to the right. The ray origin points are randomly generated on the top face of the box, and shot down the -Z axis. The box dimensions/etc could have been done with BP parameters, but having a visual box is a nice design guide and helps with debugging.

The scattering in the Actor BP is computed in the Do Scatter function, the Construction Script just calls this, like we saw in other examples above. Most of this BP is actually just generating the ray origin and mapping between coordinate systems. Basically we generate one rock and then inside the For loop, at each valid intersection point, we apply a random rotation and scale to break up the repetition. Although Append Tiled is used, it’s called once for each rock, with 1 repeat - so in this context it’s just a way to append a single mesh, because we don’t have an Append Once function. Finally in the bottom-right we return the accumulated Scatter Objects mesh.

View fullsize

One question is, what to scatter on. We could have used world linetraces to find hits, but then we would have had to filter out all the non-asteroid level elements. Instead I used the ADynamicMeshBaseActor spatial query function IntersectRay. The BP_RockScatter has a public array UProperty of ADynamicMeshBaseActor targets, as shown on the right. The Find Intersection Point BP function (pasted below) calculates the nearest ray-intersection point on each of these objects and returns the nearest point. Although I have hardcoded the ray direction in this point, that could easy be exposed as an argument, making this a very general BP function (that doesn’t rely on physics!)

View fullsize

One important note about working with this BP in the Editor. The rocks only update when the Construction Script runs. So, if you change the Asteroids, you need to force the Scattering to be recomputed by changing a setting of the Actor instance (ie, “jiggle it” a bit, change the Scatter Count, etc).

UVs and Triplanar Textures

I have avoided the question of UVs so far, because UVs potentially make all this procedural-mesh stuff a lot more complicated. The basic shape-generator functions for spheres, boxes, toruses, etc, all generate decent UVs. The underlying generators have more UV options that could be exposed. Boolean operations will combine UVs properly, and perations like Plane Cut will (for example) generate UVs on the cut surface, and there are controls for those UVs in the underlying C++ code (also not BP-exposed here). However none of these operations will update the UV Layout, ie make sure that islands are unique/etc. And there is no way (currently) to recompute UVs for an object after something like the Solidify operation. (This can be done in the Editor, but it requires calls to Editor-Only code, and only works on Windows currently).

This limits the options for texturing, but again, in a procedural context this is all pretty common. If you want to generate procedural worlds, you aren’t going to be hand-painting them. The most basic procedural texturing one can do is simply to apply tiled UV maps, based on the UV set. This can work surprisingly well if the “scale” of the UV islands on the mesh are all relatively similar, however (for example) if you subtract a small sphere from a big one, this won’t be the case by default. The UVs in the subtracted-small-sphere area will be denser, and so the texture pattern will be different sizes.

An alternative is to use local or world-space Projection. What this means at a conceptual level is, we take a geometric shape with known “good” UVs, eg like a plane/box/cylinder/sphere, and put the texture map on that shape. Then to find the color at a point on our detailed shape, we map that position onto the simple shape (eg by casting a ray, nearest point, etc) and use the color/etc at the mapped point. So for example with Planar Projection, to find the color for point P we would project P onto the tangent axes of the plane, ie U = Dot( P-PlaneOrigin, PlaneTangentX ) and V = Dot( P-PlaneOrigin, PlaneTangentY ), and then sample the Plane’s Texture Map at (U,V).

Planar Projection is cheap but only works if your surface is basically pointing at the plane, otherwise it just kind of smears out. A more complex version is TriPlanar Projection, where you have 3 planes and use the surface normal to choose which plane to project onto, or as a way to compute a blend weight between the 3 colors sampled from each plane. This works particularly well for things like rocks, cliffs, and so on. In UE there are special Material nodes to create TriPlanar texture, called WorldAlignedTexture, WorldAlignedNormal, etc (this technique does work for both world-space and local-space, despite the name). The Quixel website has an excellent tutorial on setting up these kinds of Materials. In this tutorial I have used TriPlanar materials extensively, for all the rocky surfaces. Quixel’s Megascans library has tons of great tileable textures to use with TriPlanars (all free if you are using UE4). And you can also use Quixel Mixer to create new ones - I made the textures in the tutorial sample by messing around with the Mixer sample projects.

Finally it is also possible to procedurally generate fully 3D materials, ie 3D patterns that the mesh “cuts through” to define the surface texture. These can work quite well if you are going to cut into the shape using booleans. I have used one of these for the destructible brick wall, but I won’t say this is necessarily the greatest example - I am a novice at creating UE Materials!

Gotchas, Pitfalls, Caveats, and so on

I quite enjoyed playing with procedural shape generators like this in the Editor. I’m not a great artist, so being able to generate shapes and mash them up let me build a level I probably would not have put in the effort to model by hand. But, I do want to be super clear that although the approach to procedural content generation that I have outlined here might be fine for a demo or small indie game, it won’t scale up to a large procedural world. There are some major problems.

For one, UProceduralMeshComponent is serialized in the Level. So every time I save the level, all my generated PMCActor meshes are being saved, and when I restart the Editor they are de-serialized, right before my Construction Scripts run and regenerate them all! It’s not only inefficient (it takes quite a long time), it also requires a lot of disk space - the AA_AsteroidLevel.umap file is 38MB, just for this small level. And if the content is all in the Level, you can’t have someone else working on it, as you can’t merge changes to the binary .umap file. So, saving and loading the Level is a bottleneck. (Note that this would not be an issue if the Actors were only spawned at runtime)

Second, I relied very heavily on Construction Scripts. This is mostly fine, as the objects are generally independent. However, it is a major problem for the BP_RockScatter scattering Actor. The issue is that this Construction script must run after the scripts for the Asteroids, or there is nothing for the raycasts to hit. However, it is not possible to explicitly specify the ordering of Construction scripts. So it’s easy for this to break. If you google you’ll find various tricks, like order of adding to the level, naming, etc, that can make this work, but these are not guarantees and the ordering could easily change in future Engine versions. If you wanted to build a complex sequence of procedural generators, with dependencies on eachother, this would not be possible to do with Construction scripts. It is possible to better handle these kind of dependencies at runtime, by making the generation dependent on events fired between Actors. But then you have to PIE to see what your generators are doing.

Third, it’s pretty easy to accidentally lock up the Editor when playing with these Blueprints. The computation all happens on the main thread, and cannot be interrupted, so if you do something that is going to take 20 minutes (or 20GB of RAM) to compute, you’ve just got to wait. In particular, dragging sliders in the Details panel is a problem, it tends to lock things up while it recomputes the BP for each slider update. I got in the habit of typing in new values instead of just scrubbing to see what happens, like you might in a Material.

Basically, we’re trying to wedge a procedural geometry dataflow graph into a system that was not designed for this. A real procedural node-graph system for doing expensive geometry operations has a very different underlying architecture than a runtime gameplay framework like Blueprints. For example, a procedural node-graph system usually automatically handles things like our mesh pool, caches expensive computations, evaluates nodes in parallel when possible, and so on. Then again, those DCC-tool node-graph systems don’t work at Runtime…

Wrapping Up

That’s it for this tutorial. If you make neat things with procedural meshes using the system I have developed here, please @ me on twitter (@rms80) with screenshots, comments, and suggestions, or send an email.

If you want to dive deeper into Procedural Content Authoring/Generation inside UE, I highly recommend watching the Getting Started with Procedural Mesh Generation twitch stream by Arran Langmead (twitter). In this video Arran explains how he builds a procedural tree generator using the basic UProceduralMeshComponent API (which allows you to build up meshes directly from vertex positions and triangle index arrays). There is an accompanying written article which walks through much of the same content on 80.lv. Unlike myself, Arran is a UE master and his tree sample can show you how to do awesome things like make procedural shaders and procedurally animate generated meshes with materials.

After I watched Arran’s video I realized there is so much more that could be added to UGeneratedMesh, particularly with vertex colors. I have not made any effort to support vertex colors. But, FDynamicMesh3 supports vertex colors, and it would not be difficult to add vertex color support so that (for example) vertex colors on different parts of a Boolean are preserved, or to add BP functions to calculate vertex colors from spatial fields, 3D noise, etc. Exciting!

Mesh Generation and Editing at Runtime in UE4.26

In my last tutorial, I showed you how to use the new experimental GeometryProcessing plugin in UE4.26 to do useful meshy things like mesh generation, remeshing, simplification, and Mesh Booleans (zomg!). When people first learn that we are building these capabilities in Unreal, one of their immediate questions is usually “Can I use it in a game?”. The short answer is, yes. However there is no Blueprint API to that plugin, so there are some hoops to jump through to do it.

A related question that comes up very frequently is how to implement runtime mesh creation in a UE4 game or application. UProceduralMeshComponent (API docs link) is historically how one would do such a thing in Unreal Engine. As of UE4.25, it is now also possible to “build” and update a UStaticMesh at Runtime, which can then be used in UStaticMeshComponent/Actor. So we now have a new question, which one should you use? In addition, there are third-party solutions like RuntimeMeshComponent (link) that provide more functionality than UProceduralMeshComponent and might be a better choice in some situations. (For the rest of this tutorial I am going to abbreviate UProceduralMeshComponent as PMC and UStaticMesh/Component as SMC, to save my fingers).

Unfortunately there is no “best” option - it depends on what you need. And it’s not immediately obvious how to hook any of them up to our GeometryProcessing plugin, which uses FDynamicMesh3 to represent meshes without any connection to Components or Actors. So, in this tutorial I will show you one way to implement runtime-generated-and-edited meshes that will work with any of these options.

The video above-right shows a small “runtime geometry gym” demo that I built using the Actors and utility code in this tutorial. As you can see there are Booleans, mesh operations, and some spatial-query demos. These are just a few things I exposed via Blueprints (BP), and by the end of this tutorial you should understand how it is very straightforward to expose other GeometryProcessing-based mesh editing code on the ADynamicMeshBaseActor I will describe below.

(Mandatory Disclaimer: your author, Ryan Schmidt, is an employee of Epic Games. However, gradientspace.com is his personal website and this article represents his personal thoughts and opinions. About triangles.)

Translation for Chinese Users: https://zhuanlan.zhihu.com/p/345724236

Getting and Running the Sample Project

Before we begin, this tutorial is for UE 4.26, currently in Preview release (Preview 4 at time of writing). You can install the Preview binaries from the Epic Games Launcher.

The project for this tutorial is on Github in the UnrealMeshProcessingTutorials repository (MIT License), in the UE4.26/RuntimeGeometryDemo subfolder. Unfortunately in UE4.26 this project will only work on Windows. It should be possible to get this tutorial working on OSX or Linux with some selective deleting, I will describe this at the end of the post. If you don’t want to check out with git, you can grab a zip of the current repository.

Once you are in the top-level RuntimeGeometryDemo folder, right-click on RuntimeGeometryDemo.uproject in Windows Explorer and select Generate Visual Studio project files from the context menu. This will generate RuntimeGeometryDemo.sln, which you can use to open Visual Studio. You can also open the .uproject directly in the Editor (it will ask to compile), but you probably will want to refer to the C++ code for this tutorial.

Build the solution and start (press F5) and the Editor should open into the sample map. You can test the project in PIE using the large Play button in the main toolbar, or click the Launch button to build a cooked executable. This will take a few minutes, after which the built game will pop up in a separate window. Run around and shoot the walls! You’ll have to press Alt+F4 to exit as there is no menu/UI.

The Code

There are two sides to this tutorial. First, I will describe an architecture for at-runtime dynamic/editable mesh Actors implemented in C++, and then the sample project that uses these Actors with Blueprints to do interesting things. None of the “game logic” is in C++, just the core functionality. I have put this all a plugin named RuntimeGeometryUtils, which you could easily copy to your own projects. Just to mention up front, in RuntimeGeometryUtils I have also included updated versions of DynamicMeshOBJReader/Writer from my previous tutorial, but I have changed the API to be static functions.

PMC or SMC?

If you are going to set about building dynamic in-game geometry, the immediate question is whether to use UProceduralMeshComponent (PMC) or UStaticMeshComponent (SMC). There are various small differences between these two, but the biggest is in terms of performance. To update a mesh at runtime, it needs to be “built”. By this we meant that the Rendering representation of the mesh needs to be created or updated. Unreal does not directly render from the Sections in a PMC or the FMeshDescription in a SMC. For any UMeshComponent (of which both PMC and SMC are), a FPrimitiveSceneProxy subclass is created which is the rendering representation of the Component. That Proxy will create one or more FMeshBatch from the Component data.

In PMC this is relatively straightforward and I would suggest you just go skim the code that does it in ProceduralMeshComponent.cpp, the FProceduralMeshSceneProxy class is at the top. You will see that in the constructor, FProceduralMeshSceneProxy converts the FProcMeshSection’s that you create externally and initializes a FStaticMeshVertexBuffers and FLocalVertexFactory. These store things you would expect the GPU to need, like vertex positions, triangle index buffer, Normals and Tangents, UVs, Vertex Color, etc. To the GPU this data is not different between a PMC or SMC, the only difference is how it gets there.

SMC is much more complex. I am playing a bit loose with terminology here, because a UStaticMeshComponent does not store a mesh itself - it references a UStaticMesh, and the UStaticMesh stores the mesh. Until UE4.25 it was not possible to update a UStaticMesh at runtime. This is because traditionally your “source mesh” in the Unreal Editor, stored as a FMeshDescription inside UStaticMesh, is “cooked” into a preprocessed, optimized “rendering mesh” that is used to initialize the FStaticMeshSceneProxy (done in StaticMeshRender.cpp). The source FMeshDescription is not used after cooking, so it is stripped out in your built game. This cooking process, kicked off by UStaticMesh::Build(), depends on various Editor-only functions and data, and that’s why you couldn’t update the UStaticMesh geometry at runtime.

But, in 4.25 a new function was added - UStaticMesh::BuildFromMeshDescriptions(). This function takes a list of FMeshDescriptions and initializes the rendering mesh data, allowing a UStaticMesh to be built at runtime. This build is not the same as the Editor-only ::Build() path - it skips various complex steps that are too slow to do at runtime (eg build Distance Field Lighting data) or not useful (eg generate lightmap UVs, which would be useless as you can’t bake new lightmaps at runtime).

Calling BuildFromMeshDescriptions() is more expensive than updating Sections in a ProceduralMeshComponent. The trade-off is that you can re-use the generated UStaticMesh in multiple StaticMeshComponents (and may even get instanced rendering), while if you want to have multiple of the “same” PMC, you need to copy the mesh into each one. In addition you get extra rendering features with SMC. For example the FProcMeshVertex used in the PMC only supports 4 UV channels, but a UStaticMesh supports up to 7. UStaticMesh also supports LODs and Sockets, and generally SMC is better-supported throughout the Engine.

One more fundamental difference has to do with how these different types use the renderer. PMC uses what’s called the “Dynamic Draw” path, which means it rebuilds and re-submits FMeshBatches every frame. This basically tells the renderer “my vertex/index buffers might change at any frame, so don’t bother caching anything” and this has a peformance cost. SMC uses the “Static Draw” path, which tells the renderer that the render buffers are not going to be changing and so it can do more aggressive caching and optimization. If you are interested in more details here, Marcus Wassmer gave an excellent GDC 2019 talk on the current UE4 Rendering Pipeline..

These are not the only differences, however in terms of at-runtime generated meshes, this is the core trade-off. SMC will give you better rendering performance but has a higher up-front cost. So if you are just loading generated meshes, or are “done” dynamically editing or changing a mesh, you will benefit from building it as an SMC. But if you are actively changing a mesh, you will really suffer if you are updating a SMC every frame. As a simple test, I used the infrastructure I will describe below to re-generate a sphere every frame, at “tessellation level 32”, which generates 2046 triangles and 1025 vertices. Using PMC this runs at 90-100fps in PIE. With SMC it drops down to 30fps. If I crank up the tessellation to 128 (32k triangles), it’s still hitting about 15fps with PMC, while with SMC it’s an unworkable 3fps.

A Third Option: USimpleDynamicMeshComponent

Having to deal with two different Components does make things complicated…and I’m about to make it worse! PMC has a fundamental limitation in that it cannot have split normals, UVs, or other attributes at a vertex. So if you have a cube with connected mesh topology - ie it has no boundary edges - but 3 normals at each corner vertex, or a separate UV chart/island for each face, you have to split that cube up into 6 rectangles before you initialize the PMC. This is because the PMC wants to be as efficient as possible at initializing the RenderProxy, and the GPU doesn’t support split-anything. Splitting up vertices and rewriting triangle indices is complicated and would make the PMC API much more complex, so you have to do it yourself.

But, this means that the PMC is not suitable to directly use for any kind of mesh editing. For example lets say you want to plane-cut a cube and fill the cut surface. Well, if you were to try to cut the PMC sections directly, you don’t actually have a cube, you have 6 disconnected triangle patches. So now instead of getting a closed boundary loop after the cut, which is ideal for hole-filling, you get some 3D lines that need to be chained up to identify the hole. This is all very bad for editing reliability. Similarly if you pull on a “cube” corner, you’re just going to create a crack. I could go on…just trust me, it’s a nightmare.

So, to implement the Mesh Modeling Editor Mode in UE 4.25, we introduced another type of mesh Component, USimpleDynamicMeshComponent. This component is like a PMC in that it uses the Dynamic Draw path and is designed to be fast to update. However unlike the PMC, it stores a more complex mesh representation that supports split-attributes at vertices, and internally handles rewriting that mesh as something suitable for the GPU. And what is this more complex mesh representation? It’s a FDynamicMesh3 of course.

(Why “Simple”? Because there is another variant - UOctreeDynamicMeshComponent - that is designed for efficiently updating sub-regions of huge meshes. That one is beyond the scope of this tutorial.)

USimpleDynamicMeshComponent (let’s call it SDMC) has many features that PMC lacks that are nice for interactive mesh editing situations. For example it supports overriding the material on a subset of triangles, or hiding them, without re-structuring the mesh. It also has various functions for “fast updates” of the different rendering buffers, for example if you only changed vertex positions, colors, or normals. It also supports “chunking” the mesh into multiple render buffers for faster updating, and can auto-compute tangents on mesh updates. We won’t be using any of these capabilities in this tutorial.

In terms of trade-offs, SDMC will generate larger render buffers than PMC. So, if GPU memory in your dynamic-geometry-game is a concern, it may not be the best choice. In interactive-mesh-editing contexts, this memory usage will generally pale in comparison to the many mesh copies and ancillary data structures you will need, though. SDMC also does not currently have any Physics support. And finally it cannot be serialized - you should only use SDMC in contexts where your generated mesh is saved some other way.

Runtime Geometry Architecture and ADynamicMeshBaseActor

Ok, so now we have 3 options - SMC, PMC, and SDMC. Each stores the mesh in a different way (FMeshDescription, FProcMeshSection, and FDynamicMesh3). Which one should we use? Let’s choose all of them!

The core architectural question is, where is your generated mesh coming from? If you are generating it completely procedurally, or just loading it from a file, then you can easily use any of these options, the only question is whether it’s static after being generated, frequently updated, or you want to have the convenience of not having to build sections yourself.

If you need to change the meshes after they are generated, then I strongly recommend you do not think of the mesh representation inside any of these Components as the canonical representation of your application’s data model. For starters, none of them will serialize your mesh data. And you probably have additional metadata beyond just the mesh vertices and triangles, that you would like to keep track of (and even if you don’t yet, you probably will eventually). So, I think the way you should think of the different Components is strictly as different ways to render your mesh data.

In this tutorial, “my mesh data” will be stored as FDynamicMesh3. This is a good choice if you don’t have your own mesh, in my opinion. However do know that currently there is no native serialization for FDynamicMesh3, you will need to implement that yourself. The next question is where to put this data. I am going to have it live on a C++ Actor class, ADynamicMeshBaseActor. If I were building a real app, say a mesh sculpting tool, I would probably have the mesh live in some other place, and just pass it to the ADynamicMeshBaseActor when it is modified. But for now I will have it live directly on the Actor:

UCLASS(Abstract)
class RUNTIMEGEOMETRYUTILS_API ADynamicMeshBaseActor : public AActor
{
protected:
    /** The SourceMesh used to initialize the mesh Components in the various subclasses */
    FDynamicMesh3 SourceMesh;
};

This Actor has no way to display this mesh, it needs a Component. Rather than have to make a choice, I’m going to make 3 Actor subclasses, one for each type of mesh Component:

UCLASS()
class RUNTIMEGEOMETRYUTILS_API ADynamicSMCActor : public ADynamicMeshBaseActor
{
    UPROPERTY(VisibleAnywhere)
    UStaticMeshComponent* MeshComponent = nullptr;
};

UCLASS()
class RUNTIMEGEOMETRYUTILS_API ADynamicPMCActor : public ADynamicMeshBaseActor
{
    UPROPERTY(VisibleAnywhere)
    UProceduralMeshComponent* MeshComponent = nullptr;
};

UCLASS()
class RUNTIMEGEOMETRYUTILS_API ADynamicSDMCActor : public ADynamicMeshBaseActor
{
    UPROPERTY(VisibleAnywhere)
    USimpleDynamicMeshComponent* MeshComponent = nullptr;
};

Now on the ADynamicMeshBaseActor we are going to have the following function, which has no implementation (but cannot be C++-abstract, ie = 0, because UE does not support that on UObjects):

protected:
    /**
     * Called when the SourceMesh has been modified. Subclasses override this function to 
     * update their respective Component with the new SourceMesh.
     */
    virtual void OnMeshEditedInternal();

Finally we implement this function in each of the subclasses. Essentially what each of those functions has to do, is generate and update their Component’s mesh data from the FDynamicMesh3. In the file MeshComponentRuntimeUtils.h I have added converter functions to do this, UpdateStaticMeshFromDynamicMesh() and UpdatePMCFromDynamicMesh_SplitTriangles(). The former uses the FDynamicMeshToMeshDescription class, part of GeometryProcessing, to do the conversion. For the PMC the function currently just splits each triangle. This is not the most efficient in terms of GPU memory, but is the fastest on the CPU side (in interactive mesh editing we tend to focus on the CPU side, as that’s generally the bottleneck).

For SDMC, it’s just a straight copy into the Component. Note that the SDMC does support directly editing it’s internal FDynamicMesh3. If I was cleverer, I could have allowed the base-class to access this mesh and sometimes avoid a copy. However, then I would not be using the Component-owned data as my canonical Source Mesh, which I said above was a bad idea. In some performance-sensitive situations it might make sense, but FDynamicMesh3 is very fast to copy and so just to keep things clean, I didn’t do it here.

Finally, we have one top-level API function on ADynamicMeshBaseActor that we use to actually modify the SourceMesh:

/**
 * Call EditMesh() to safely modify the SourceMesh owned by this Actor.
 * Your EditFunc will be called with the Current SourceMesh as argument,
 * and you are expected to pass back the new/modified version.
 * (If you are generating an entirely new mesh, MoveTemp can be used to do this without a copy)
 */
virtual void EditMesh(TFunctionRef<void(FDynamicMesh3&)> EditFunc);

Basically you call this function with a C++ lambda that does your actual edit. This pattern allows us to better control access to the SourceMesh, so we could in theory do things like “borrow” it from somewhere else. Another function, ::GetMeshCopy(), allows you to extract a copy of the SourceMesh from the Actor, which is needed in cases where you want to combine two meshes, for example.

And that’s basically it. If we have an instance of any of the 3 ADynamicMeshBaseActor subclasses above, we can update it in C++ by doing something like the following:

SomeMeshActor->EditMesh([&](FDynamicMesh3& MeshOut)
{
    FDynamicMesh3 NewMesh = (...);
    MeshOut = MoveTemp(NewMesh);
});

and the underlying PMC, SMC, or SDMC will be automatically updated.

One valid design question here is, why is this all done on the Actor, instead of the Component? It could of course be done on the Component too. It’s question of what you intend to do with your meshes. One complication with UE is that one Actor might have many Components, and then if you want to do things like “combine meshes”, you would have to decide how to handle these multiple, possibly hierarchical Components, some of which might not be your editable meshes. And what happens to the Actors if the Components are your primary “thing”? (These are conceptual problems we continue to struggle with in the design of the Modeling Editor Mode!!). For this tutorial I want to think of each mesh as an “object” and so organizing via Actors made sense. In addition, Actors have blueprints and this will make it easier to do fun things below.

Mesh Generation

Unfortunately our current ADynamicMeshBaseActor won’t do anything interesting in the Editor unless we initialize it in C++. So I have added some basic mesh generation functionality to it. A top-level UProperty SourceType determines whether the mesh is initialized with a generated primitive (either a box or sphere), or an imported mesh in OBJ format. The relevant sections of the Actor properties are shown below (click to enlarge). I also added the ability to control how Normals are generated, what Material is assigned to the underlying Component, and additional options for the Generation and Import. In addition there are options to enable automatic building of an AABBTree and FastWindingTree, which I will discuss below.

View fullsize

View fullsize

One note on the “Imported Mesh” option - the import path can either be a full C:\-style path, or a path relative to the project Content folder. I have included the Bunny.obj mesh with the project. This mesh will be included in the packaged build because I have added the SampleOBJFiles folder to the “Additional Non-Asset Directories to Copy” array in the Project Settings under Project - Packaging.

With these additions, we now have fully dynamic generated and imported meshes. If you open the Editor, you can find ‘Dynamic SMCActor”, as well as the PMC and SDMC actors, in the Place Actors panel (via the search box is easiest), drop one into the scene, and then change it’s parameters in the Actor DetailsView and the mesh will update, as shown in the short clip above-right.

Blueprint API

The final part of ADynamicMeshBaseActor is a small set of UFunction’s for executing mesh import/copy, spatial queries, booleans, solidification, and simplification. These are all marked BlueprintCallable and I will explain them in more detail below. But let’s look at the code for one of them, just to see what it’s like:

void ADynamicMeshBaseActor::CopyFromMesh(ADynamicMeshBaseActor* OtherMesh, bool bRecomputeNormals)
{
    // the part where we generate a new mesh
    FDynamicMesh3 TmpMesh;
    OtherMesh->GetMeshCopy(TmpMesh);

    // apply our normals setting
    if (bRecomputeNormals)
    {
        RecomputeNormals(TmpMesh);
    }

    // update the mesh
    EditMesh([&](FDynamicMesh3& MeshToUpdate)
    {
        MeshToUpdate = MoveTemp(TmpMesh);
    });
}

This is a simple function, but the point is that you can basically cut-and-paste this function, rename it, and plug any GeometryProcessing code in at “the part where we generate a new mesh”, and you’ll have access to that operation in Blueprints. The other BP API functions that apply mesh updates all do exactly that. A useful exercise would be to add the a Remesh() function by cutting-and-pasting the remeshing call from the Command-Line Geometry Processing Tutorial.

The Project

Then RuntimeGeometryDemo project has a single map, AA_RuntimeGeometryTest, with 4 “stations”. Each area does something different with runtime geometry based on the BP API I have added to ADynamicMeshBaseActor. From right-to-left we have a boolean operation, mesh simplification/solidification, spatial queries, and then another boolean demo where you get to chew away at some blocks!

The embedded video at the top of the tutorial shows a quick run-through of the different stations. We’ll walk through them in detail below.

View fullsize

Booleans!

In the first area, there are two dynamic mesh actors (both SMC in this case, but it doesn’t really matter). The red sphere is animated, and when you step on one of the three buttons on the ground labeled Union, Difference, and Intersection, that Boolean operation will be applied to these two objects, and the sphere will be deleted. The Blueprint Actor BP_ApplyBooleanButton is where the action is, and is shown below. Basically when you overlap the respective green box, it changes material, then we get both the Target and ‘Other’ Actors and call ADynamicMeshBaseActor::BooleanWithMesh(Target, Other). Then the Other Actor is Destroyed. Easy!

View fullsize

The objects for each station are grouped into a folder, shown for this part below-left. Below-right is the DetailsView properties for the above blueprint on the Button_XYZ actors. Here the various parameters in the Button BP can be configured for each Button Actor instance. Basically, you set the “Target” and “Other” Actors to two mesh actors in the current level, and pick an operation. In addition you can set the “Pressed” and “Not Pressed” materials. The SM_ButtonBox static mesh used in the BP is configured to only register Pawn overlaps, and so the ActorBeginOverlap above will only fire when the player steps on it, and kick off the Boolean operation.

View fullsize

One little BP tip, the “Validated Get” used above to only continue execution if BooleanOtherActor “Is Valid”, can be quickly created by right-clicking on a normal parameter-Get node and selecting ‘Convert to Validated Get’ at the bottom of the context menu. I only learned this after wasting countless minutes wiring up explicit IsValid branches.

Mesh Algorithms!

At the next station you can repeatedly jump or walk on/off the Simplify button to simplify the green Bunny mesh by 50% each time, or step on Solidify to run the Fast Mesh Winding Number-based remeshing. These are both essentially cut-and-pastes of the BP_ApplyBooleanButton, but they don’t take a second Actor. Below I have zoomed in on the relevant “action” section of BP_ApplySimplifyButton.

View fullsize

Spatial Queries!

The third station is not really interactive, although if you get in there and jump at just the right time, you can knock the flying sphere around. There are two objects “attached” to the semitransparent bunny. Neither are our editable mesh actors, they are just normal blueprinted StaticMeshActors, BP_MagnaSphere and BP_RotatoSphere. Their BPs are more complicated, I have shown BP_RotatoSphere below. Basically this one just moves around in a circle (“Then 0” off the Sequence node), moves a little sphere sub-Component to the nearest-point on the Bunny mesh surface (“Then 1” branch) and then changes it’s color depending on whether the StaticMesh’s location is inside or outside the Bunny (“Then 2” branch).

These latter two steps call the ContainsPoint() and DistanceToPoint() functions on the target ADynamicMeshBaseActor. These are just utility functions that query an AABBTree and FastWindingTree built for the SourceMesh automatically, if the bEnableSpatialQueries and bEnableInsideQueries UProperties are true, respectively. Otherwise they will just return false. Building these data structures can be expensive if the mesh is changing every frame, and should be disabled unless they are needed.

There is also a ADynamicMeshBaseActor::IntersectRay() function exposed to BP, which is not used in any of the examples. You might find this capability useful as runtime-generated meshes can’t necessarily be hit by LineTraces. For PMC and SMC it requires runtime physics cooking, which is not fully supported by Chaos and is generally somewhat expensive, and SDMC doesn’t support it at all. In addition ::IntersectRay() is implemented based on the double-precision DynamicMesh3/AABBTree, which can be helpful with huge meshes. (We rebuild AABBTree’s per-frame during many Modeling Mode operations in the Editor, it is not unreasonably expensive in an “editing tool” context.)

View fullsize

The BP_MagnaSphere is similar, except it uses the nearest point to add an impulse to the sphere, basically “attracting” it to the Bunny surface. This sometimes goes flying off into the world, so you might not always see it.

Shoot This!

In the final station, you can left-click to fire spheres at the red and green walls, and the spheres will be Boolean subtracted when they hit the wall. There is no physics involved here, the blueprint below detects the hits using the ADynamicMeshBaseActor::DistanceToPoint() query, between the center of the BP_Projectile actor and the wall mesh. The distance is compared to a fraction (0.25) of the radius of the projective bounding-box - this is the input to the CompareFloat node below. If it’s within range, the projectile sphere mesh is subtracted from the wall, and then destroyed. The red wall is an SDMC and the green wall is a PMC, and the projectile sphere is an SDMC, but the whole point here is it really doesn’t matter in this context.

If you recall from above, after each Subtract operation, the mesh has to be updated and the AABBTree recomputed so that the Distance to Point can be calculated. This might sound slow, but for me it generally runs at 90-100fps in PIE (run ‘stat fps’ in the console to see framerate) even if I’m shooting as fast as I can. This is surprisingly fast (I was surprised!). Turning up the tessellation level on the wall or projectile will have a noticeable effect, and it’s easy to end up getting noticeable hitches. One note, if you just hit ‘play’ and go to the wall, the pulsing red sphere at the first station will still be regenerating every frame, which slows things down (see PMC/SMC discussion above, the red sphere is an SMC). You will notice that shooting is snappier if you disable ‘Regenerate on Tick’ on the red sphere, or jump on one of the Boolean buttons first.

View fullsize

Frequent use of Get All Actors Of Class is not recommended…but it is easy!

View fullsize

Using physics for this kind of thing could be problematic because generally physics collision detection requires “simple collision” geometry, which has to be boxes spheres, capsules, or convex hulls. Decomposing the complex result of a half-blasted wall into those shapes is very difficult (read: slow). “Complex Collision” can also be used, but in that case the collision tests are much more expensive than a single distance-to-point query. A raycast could be used, and in fact I did this initially, but it meant that the ball could easily “go through” tiny holes. The distance threshold makes it “feel” much better (and can be decreased to allow each ball to do more damage). Finally it’s kind of neat to play around with the projectile mesh and/or wall mesh, like I did on the right…

Note that it would also be possible to test for exact overlaps between the two meshes here, too. TMeshAABBTree has a TestIntersection() function that takes a second TMeshAABBTree (and optional transform), and this could easily be exposed in BP, just like DistanceToPoint() is. However in this case, it would mean that the ball might not get a chance to penetrate into the wall before being subtracted (which would also be the case with complex collision). And this function is much more expensive than a simple distance query.

One final note, the BP_Projectile’s are emitted by the Fire() function on ARuntimeGeometryDemoCharacter. This could probably also be done in BP. But note that it is the only code I added to the auto-generated Third Person template project - everything else in C++ was done by the RuntimeGeometryUtils plugin.

What About Collisions?

You will notice if you run the demo that you can run right through any of the runtime-generated meshes, there is no collision. As I described above, the Boolean-gun does not use the collision system, even for line traces. Physics “cooking” is separate from rendering mesh cooking, and support for runtime physics cooking is more complicated. UProceduralMeshComponent does support runtime physics cooking, but currently only with PhysX. The situation is murkier with SMC, since the runtime Build option is very new, it’s not clear that runtime physics cooking will work (and would have the similar PhysX requirement). And SDMC does not support physics at all, as it is meant for fast visualizing during editing, and physics cooking is expensive!

However even if we do want physics, we still have a complication because as I mentioned above, “Simple Collision” is required to support physics simulation (ie things that move) and only allows for spheres, capsules, boxes, and convex hulls. So it’s necessary to approximate a complex object with a set of these simpler shapes for it to be simulated at runtime. This is essentially an unsolved problem to solve well automatically - just look at that wall above riddled with bunny-bullet-hits and imagine trying to split it up into boxes and spheres! The other option is “Complex Collision”, which is limited to static objects (ie that can be collided with but aren’t simulated) and is expensive both to cook and to test for collisions. It will do in a pinch or a prototype, but you probably don’t want to build your game around complex collision.

This is without a doubt a major thorn in the side of any game or app dependent on runtime-generated-geometry. As I have shown, it is possible to implement some physics-like effects without the full physics system. And it would be possible to implement the UPrimitiveComponent LineTrace and Overlap tests against the FDynamicMeshAABBTree, which would allow many things to “just work” (but not physics simulation). Perhaps a topic for a future article!

Late Breaking Update!

As of UE4.26 Preview 5, the default Physics system has been switched back to PhysX in the binary builds. You can still enable Chaos if you are building from source, and experiment with all the great new features that are coming in the future. However one result of this switch is Runtime Physics Cooking for the UProceduralMeshComponent is available again. So, I have added a Collision Mode field to the ADynamicMeshBaseActor, with options for Complex as Simple and Complex as Simple Async. These options will only currently have an effect on the ADynamicPMCActor, not with the SMC or SDMC. It may be possible to also get this to work with the SMC, I haven’t tried yet myself (maybe you will figure it out and submit a PR!)

The video on the right shows what happens if you switch the BP_ShootableBlock_PMC object to use either of these new modes (I also changed the material in this video, to a volumetric procedural brick texture, which is why it keeps looking like brick as it is destroyed). Since it’s not happening every frame, the physics cooking is actually quite snappy and I can’t say I noticed any real performance hit. Exciting!

A short note about the ‘Async’ variant. What this means is that the Collision cooking is done on a background thread and so it doesn’t block the game thread. This improves performance but the trade-off is that the updated collision geometry isn’t necessarily available immediately. As a result if you have quickly-changing geometry and fast-moving objects, they could more easily end up “inside” the generated meshes before the collision is finished updating.

Should you make your own Component?

This is a question that frequently comes up when looking more deeply into this problem of runtime-generated geometry. The most common case I have heard about is, you are generating meshes using an existing C++ library that has it’s own YourMeshFormat. You want to see this mesh in Unreal Engine. So as we’ve done in this tutorial, you are going to have to stuff it into a PMC or SMC to get it on the screen. This will involve one or more copies/translations of YourMeshFormat into something the Engine can ingest - either PMC Sections, an FMeshDescription, or now a FDynamicMesh3 with SDMC.

But if you dive into the PMC code, you’ll see that there is not much to the PMC and it’s SceneProxy. It won’t take you long to figure out that you could “cut out the middleman” and have a custom UYourMeshComponent that directly initializes the RenderBuffers from an instance of YourMeshFormat. In fact this is essentially what the SDMC does, where the alternative mesh format is FSimpleDynamicMesh3.

So, should you? My opinion would be, that depends on how much time you want to invest in saving a few mesh copies. If you have a large team of skilled engineers, it’s not a huge commitment, but you will have to do continual work keeping your Component up-to-date as the engine evolves. If you are a small team or indie, the small performance win may not be worth the effort. I would definitely want to profile the cost of the mesh copies/conversions, because in our experimentation, this has not been the bottleneck. Uploading the new render buffers to the GPU is still the main expense, and this doesn’t change with different Component types.

And if you organize your “mesh architecture” as I have in this tutorial, it wouldn’t necessarily matter - if you don’t depend on specific Component features, you could swap in a new Component type as needed.

(Addendum: khammassi ayoub has written a very detailed set of Medium articles about creating your own Mesh component - links here to Part 0, Part 1, and Part 2. However just a small note, much of the effort in that tutorial goes into being able to have a custom vertex shader, which is not necessary if you just want to make a “PMC but with my mesh format to avoid copies” Component. In particular you don’t need to make your own Vertex Factory/etc if you are just rendering “normal” meshes. But it’s a great read as he covers all the critical parts of Rendering-side Component/Proxy infrastructure)

Wrapping Up

That’s the end of this tutorial. I focused on Booleans here because they don’t require any parameter manipulation. However GeometryProcessing and the ModelingOperators module in the MeshModelingToolset plugin have operations like Extrusion, Offset, Bend/Twist/Taper space deformers, and more, that could easily be added to something like ADynamicMeshBaseActor and manipulated interactively in-game, with UMG or by in-world actions.

Although you could directly use ADynamicMeshBaseActor and the RuntimeGeometryUtils plugin, I really see this more of a guide for how you could build your own versions. If you are creating a game or app that involves storing content created at runtime, I would encourage you to spend some time thinking about how you want to organize this data. Storing the source meshes in the Actor was convenient for this demo, but if I were building something real, I would move ownership of those source meshes out of the Actor to some more centralized place, like a UGameInstanceSubsystem. It might seem like you can “do this later”, but if you build a lot of Blueprints on top of this current system, it will be messy to refactor later (I initially had ADynamicMeshBaseActor in the game folder, and just being able to move the .h/.cpp to the plugin without completely breaking everything involved spending an afternoon learning about Redirectors…)

It’s also not strictly necessary to have the separate PMC/SMC/SDMC Actors. Since I wanted to compare them, it was useful to split them up this way. But it would be possible to have one base actor that dynamically spawns the different Component types based on an enum. This might make life easier for working with Blueprints, as right now if you make a BP for the one of the Actor subclasses, and want to switch it to a different one, you have to jump through some hoops to change the base type, and you can’t share between multiple types (that’s why there are two shootable-wall BP’s in the tutorial, one for PMC and one for SDMC).

Finally I mentioned earlier that this will not currently work on OSX. That’s because of the USimpleDynamicMeshComponent - it is part of the MeshModelingToolset plugin, which depends on some editor-only modules that require third-party DLLs currently only available on Windows. It should be possible to get the ModelingComponents module that contains the SDMC working with some strategic #ifdef’s, but that would require recompiling the engine. A more immediate solution would be to remove ADynamicSDMCActor .h/.cpp and the “ModelingComponents” reference in the RuntimeGeometryUtils.build.cs file, and only use the SMC or PMC variants. I have verified that on Windows everything still compiles if you do this, which means it should work on OSX, but I have not tested it myself. (Note that this also breaks most of the sample project)

Thanks for reading! Don’t hesitate to post questions in the comments, or on twitter @rms80.

Command-Line Mesh Processing with Unreal Engine 4.26

This is the first of several tutorials that will (hopefully) teach you how to do advanced Mesh/Geometry Processing with Unreal Engine. Past Gradientspace Tutorials focused on the open-source Geometry3Sharp library that I developed in C#, and demonstrated how to do things like Mesh Simplification, Remeshing, Implicit Surface Modeling, and so on. G3Sharp became quite powerful, I used it to create the Cotangent 3D printing tool, and helped others use it to build Archform (a tool for designing dental aligners) and NiaFit (for designing 3D printable prosthetics). Quite a few other developers seemed to like it, too.

However, ultimately C# became a limiting factor for G3Sharp. I really like coding in C#, but the performance cost can be high, and critical math libraries like sparse linear solvers are missing from the C# ecosystem. The thought of porting even more WildMagic/GTEngine code, it was just too much! So, in December 2018 I joined Epic Games and started a C++ port of G3Sharp. Thanks to the hard work of the UE Geometry Team, this library - the GeometryProcessing plugin - has now far surpassed G3Sharp in capability. So, I think it’s about time to start showing you how to use it.

In this tutorial, I am going to walk you through a single example that generates all the meshes in the image below. In doing so, we will cover the main content of most of the previous GradientSpace G3Sharp tutorials, but in C++, in Unreal Engine. To avoid any UE complexity in this intro tutorial, we’re going to do everything in a simple command-line tool. But keep in mind that everything we’re going to do is available both in the Editor, and in-game at Runtime.

(Mandatory Disclaimer: your author, Ryan Schmidt, is an employee of Epic Games)

Translation for Chinese users: https://zhuanlan.zhihu.com/p/343789879

View fullsize

Preliminaries / UE4 Setup

View fullsize

Click to Enlarge

One small hurdle we have to overcome is that binary UE4 engine installations cannot build command-line executables. So, we’ll need to use the UE4 source, which you can get on Github once you have joined the Epic Games Github Organization (click link for instructions - it’s free for anyone who accepts the UE4 license agreement). This tutorial depends on code only available in version 4.26 or later, so I suggest you use the 4.26 branch (https://github.com/EpicGames/UnrealEngine/tree/4.26) directly (this tutorial should also work against the Release branch by the time you read it).

The simplest thing to do (in my opinion) is to use the Download Zip option, available under the Code drop-down button (see image to the right). Download and unzip (this will require about 1.7 GB of disk space). After that, you’ll need to run the Setup.bat file in the top-level folder, which will download another ~11GB of binary files and then run an installer that unpacks that into another ~40 GB. Unfortunately there is no more compact variant. Time for coffee!

The code for this tutorial is available on GitHub in the gradientspace UnrealMeshProcessingTools repository (click for link), in a folder named CommandLineGeometryTest in the UE4.26 samples subfolder. Again, you can download a zip of the top-level repository (click for direct link), or you can sync with a git client, too.

View fullsize

Assuming you unzipped the UE repository into a folder named “UnrealEngine-4.26”, then you’ll need to copy or move the sample code folder UE4.26\CommandLineGeometryTest to the path UnrealEngine-4.26\Engine\Source\Programs\, as shown in the image on the right. This folder contains various other command-line tools and apps that UE uses. You might be able to put it in other places, but this is where I tested it from, and where the sample HoleyBunny.obj file paths are hardcoded relative to.

For reference, I created this sample project based on the BlankProgram command-line executable that is included with the Engine (you can see it in the list on the right). This is a minimal Hello-World example program and a good starting point for any command-line executable based on UE (eg for unit testing, etc). The only modification I had to make to get things to work was to add references to several of the modules in the GeometryProcessing plugin, in the CommandLineGeometryTest.Build.cs file:

PrivateDependencyModuleNames.Add("GeometricObjects");
PrivateDependencyModuleNames.Add("DynamicMesh");

If you wanted to use these modules in other projects, you will have to do the same. Note that many parts of the Engine are not immediately available in a command-line or “Program” target type. For example in BlankProgram the UObject system is not initialized. The GeometryProcessing plugin modules have minimal engine dependencies, and do not define UObjects, so this is not a problem for this tutorial. (It is possible to initialize various engine systems, see for example the SlateViewer program.)

View fullsize

click to enlarge

Once you have the files in the right place, run the top-level GenerateProjectFiles.bat file. This will generate a Visual Studio 2019 UE4.sln file. Oh, by the way, you probably want to have Visual Studio 2019 installed, if you are on Windows. If you are on Linux or OSX, there are .command/.sh versions of the batch files I mentioned above, and this tutorial should work on those platforms, too. (GeometryProcessing has already been used in shipping games on desktop, mobile, and console platforms!!)

Open up UE4.sln, and you will find a long list of projects in the Solution Explorer subwindow. Find our CommandLineGeometryTest project, right-click on it, and select the Set as Startup Project option that appears in the context menu. Then click the Start Debugging button or hit F5. This will build for a minute or so, then pop up a command-line dialog box and print a bit of info as the tutorial runs (should only be a few seconds, though).

Note that this is not a full build of UE4. Since we are building a simple command-line app, we don’t have any dependencies on the majority of the Engine, or the Editor. A full build would take much longer - from ~15 minutes on my 24-core Threadripper to well over 2 hours on my 4-core laptop. So, make sure you don’t do a “Build Solution” or “Rebuild All”, or you are in for a long wait.

Tutorial Files

View fullsize

The sample code contains just a few files, all the code we care about is in CommandLineGeometryTest.cpp. The CommandLineGeometryTest.Build.cs and CommandLineGeometryTest.Target.cs are configuration files for the CommandLineGeometryTest UE4 module, and the CommandLineGeometryTest.h is empty.

The GeometryProcessing module does not natively support any file I/O, so the DynamicMeshOBJReader.h and DynamicMeshOBJWriter.h are necessary to read/write OBJ mesh files. The OBJ Reader is just a wrapper around the tinyobjloader library (https://github.com/tinyobjloader/tinyobjloader, source is embedded) which constructs a FDynamicMesh3 (the mesh class we will use). The OBJ Writer is minimalist, but does the basics.

CommandLineGeometryTest.cpp just contains #includes and a main() function, and I'm going to paste the entire tutorial code below. We'll step through the blocks afterwards, but I think it's instructive to skim through it all first. In less than 150 lines, this code demonstrates normals calculation, sphere and box generators, Mesh AABBTree setup and queries (nearest-point and ray-intersection), appending meshes, fast-winding-number-based resampling, implicit morphological operations, mesh simplification, isotropic remeshing, mesh hole filling, and mesh booleans (twice) ((yes, MESH BOOLEANS zomg!!)) :

// import an OBJ mesh. The path below is relative to the default path that Visual Studio will execute CommandLineGeometryTest.exe,
// when using a normal UE4.26 auto-generated UE.sln file. If things change you might need to update this path
FDynamicMesh3 ImportMesh;
FDynamicMeshOBJReader::Read("..\\..\\Source\\Programs\\CommandLineGeometryTest\\HoleyBunny.obj", ImportMesh, true, true, true);
// flip to UE orientation
ImportMesh.ReverseOrientation();

// print some mesh stats
UE_LOG(LogBlankProgram, Display, TEXT("Mesh has %d vertices, %d triangles, %d edges"), ImportMesh.VertexCount(), ImportMesh.TriangleCount(), ImportMesh.EdgeCount());
UE_LOG(LogBlankProgram, Display, TEXT("Mesh has %d normals"), ImportMesh.Attributes()->PrimaryNormals()->ElementCount());
UE_LOG(LogBlankProgram, Display, TEXT("Mesh has %d UVs"), ImportMesh.Attributes()->PrimaryUV()->ElementCount());

// compute per-vertex normals
FMeshNormals::QuickComputeVertexNormals(ImportMesh);

// generate a small box mesh to append multiple times
FAxisAlignedBox3d ImportBounds = ImportMesh.GetBounds();
double ImportRadius = ImportBounds.DiagonalLength() * 0.5;
FMinimalBoxMeshGenerator SmallBoxGen;
SmallBoxGen.Box = FOrientedBox3d(FVector3d::Zero(), ImportRadius * 0.05 * FVector3d::One());
FDynamicMesh3 SmallBoxMesh(&SmallBoxGen.Generate());

// create a bounding-box tree, then copy the imported mesh and make an Editor for it
FDynamicMeshAABBTree3 ImportBVTree(&ImportMesh);
FDynamicMesh3 AccumMesh(ImportMesh);
FDynamicMeshEditor MeshEditor(&AccumMesh);

// append the small box mesh a bunch of times, at random-ish locations, based on a Spherical Fibonacci distribution
TSphericalFibonacci<double> PointGen(64);
for (int32 k = 0; k < PointGen.Num(); ++k)
{
    // point on a bounding sphere
    FVector3d Point = (ImportRadius * PointGen.Point(k)) + ImportBounds.Center();

    // compute the nearest point on the imported mesh
    double NearDistSqr;
    int32 NearestTriID = ImportBVTree.FindNearestTriangle(Point, NearDistSqr);
    if (ImportMesh.IsTriangle(NearestTriID) == false)
        continue;
    FDistPoint3Triangle3d DistQueryResult = TMeshQueries<FDynamicMesh3>::TriangleDistance(ImportMesh, NearestTriID, Point);

    // compute the intersection between the imported mesh and a ray from the point to the mesh center
    FRay3d RayToCenter(Point, (ImportBounds.Center() - Point).Normalized() );
    int32 HitTriID = ImportBVTree.FindNearestHitTriangle(RayToCenter);
    if (HitTriID == FDynamicMesh3::InvalidID)
        continue;
    FIntrRay3Triangle3d HitQueryResult = TMeshQueries<FDynamicMesh3>::TriangleIntersection(ImportMesh, HitTriID, RayToCenter);

    // pick the closer point
    bool bUseRayIntersection = (HitQueryResult.RayParameter < DistQueryResult.Get());
    FVector3d UsePoint = (bUseRayIntersection) ? RayToCenter.PointAt(HitQueryResult.RayParameter) : DistQueryResult.ClosestTrianglePoint;

    FVector3d TriBaryCoords = (bUseRayIntersection) ? HitQueryResult.TriangleBaryCoords : DistQueryResult.TriangleBaryCoords;
    FVector3d UseNormal = ImportMesh.GetTriBaryNormal(NearestTriID, TriBaryCoords.X, TriBaryCoords.Y, TriBaryCoords.Z);

    // position/orientation to use to append the box
    FFrame3d TriFrame(UsePoint, UseNormal);

    // append the box via the Editor
    FMeshIndexMappings TmpMappings;
    MeshEditor.AppendMesh(&SmallBoxMesh, TmpMappings,
        [TriFrame](int32 vid, const FVector3d& Vertex) { return TriFrame.FromFramePoint(Vertex); },
        [TriFrame](int32 vid, const FVector3d& Normal) { return TriFrame.FromFrameVector(Normal); });
}

// make a new AABBTree for the accumulated mesh-with-boxes
FDynamicMeshAABBTree3 AccumMeshBVTree(&AccumMesh);
// build a fast-winding-number evaluation data structure
TFastWindingTree<FDynamicMesh3> FastWinding(&AccumMeshBVTree);

// "solidify" the mesh by extracting an iso-surface of the fast-winding field, using marching cubes
// (this all happens inside TImplicitSolidify)
int32 TargetVoxelCount = 64;
double ExtendBounds = 2.0;
TImplicitSolidify<FDynamicMesh3> SolidifyCalc(&AccumMesh, &AccumMeshBVTree, &FastWinding);
SolidifyCalc.SetCellSizeAndExtendBounds(AccumMeshBVTree.GetBoundingBox(), ExtendBounds, TargetVoxelCount);
SolidifyCalc.WindingThreshold = 0.5;
SolidifyCalc.SurfaceSearchSteps = 5;
SolidifyCalc.bSolidAtBoundaries = true;
SolidifyCalc.ExtendBounds = ExtendBounds;
FDynamicMesh3 SolidMesh(&SolidifyCalc.Generate());
// position the mesh to the right of the imported mesh
MeshTransforms::Translate(SolidMesh, SolidMesh.GetBounds().Width() * FVector3d::UnitX());

// offset the solidified mesh
double OffsetDistance = ImportRadius * 0.1;
TImplicitMorphology<FDynamicMesh3> ImplicitMorphology;
ImplicitMorphology.MorphologyOp = TImplicitMorphology<FDynamicMesh3>::EMorphologyOp::Dilate;
ImplicitMorphology.Source = &SolidMesh;
FDynamicMeshAABBTree3 SolidSpatial(&SolidMesh);
ImplicitMorphology.SourceSpatial = &SolidSpatial;
ImplicitMorphology.SetCellSizesAndDistance(SolidMesh.GetCachedBounds(), OffsetDistance, 64, 64);
FDynamicMesh3 OffsetSolidMesh(&ImplicitMorphology.Generate());

// simplify the offset mesh
FDynamicMesh3 SimplifiedSolidMesh(OffsetSolidMesh);
FQEMSimplification Simplifier(&SimplifiedSolidMesh);
Simplifier.SimplifyToTriangleCount(5000);
// position to the right
MeshTransforms::Translate(SimplifiedSolidMesh, SimplifiedSolidMesh.GetBounds().Width() * FVector3d::UnitX());

// generate a sphere mesh
FSphereGenerator SphereGen;
SphereGen.Radius = ImportMesh.GetBounds().MaxDim() * 0.6;
SphereGen.NumPhi = SphereGen.NumTheta = 10;
SphereGen.bPolygroupPerQuad = true;
SphereGen.Generate();
FDynamicMesh3 SphereMesh(&SphereGen);

// generate a box mesh
FGridBoxMeshGenerator BoxGen;
BoxGen.Box = FOrientedBox3d(FVector3d::Zero(), SphereGen.Radius * FVector3d::One());
BoxGen.EdgeVertices = FIndex3i(4, 5, 6);
BoxGen.bPolygroupPerQuad = false;
BoxGen.Generate();
FDynamicMesh3 BoxMesh(&BoxGen);

// subtract the box from the sphere (the box is transformed within the FMeshBoolean)
FDynamicMesh3 BooleanResult;
FMeshBoolean DifferenceOp(
    &SphereMesh, FTransform3d::Identity(),
    &BoxMesh, FTransform3d(FQuaterniond(FVector3d::UnitY(), 45.0, true), SphereGen.Radius*FVector3d(1,-1,1)),
    &BooleanResult, FMeshBoolean::EBooleanOp::Difference);
if (DifferenceOp.Compute() == false)
{
    UE_LOG(LogBlankProgram, Display, TEXT("Boolean Failed!"));
}
FAxisAlignedBox3d BooleanBBox = BooleanResult.GetBounds();
MeshTransforms::Translate(BooleanResult, 
    (SimplifiedSolidMesh.GetBounds().Max.X + 0.6*BooleanBBox.Width())* FVector3d::UnitX() + 0.5*BooleanBBox.Height()*FVector3d::UnitZ());

// make a copy of the boolean mesh, and apply Remeshing
FDynamicMesh3 RemeshBoolMesh(BooleanResult);
RemeshBoolMesh.DiscardAttributes();
FQueueRemesher Remesher(&RemeshBoolMesh);
Remesher.SetTargetEdgeLength(ImportRadius * 0.05);
Remesher.SmoothSpeedT = 0.5;
Remesher.FastestRemesh();
MeshTransforms::Translate(RemeshBoolMesh, 1.1*RemeshBoolMesh.GetBounds().Width() * FVector3d::UnitX());

// subtract the remeshed sphere from the offset-solidified-cubesbunny
FDynamicMesh3 FinalBooleanResult;
FMeshBoolean FinalDifferenceOp(
    &SimplifiedSolidMesh, FTransform3d(-SimplifiedSolidMesh.GetBounds().Center()),
    &RemeshBoolMesh, FTransform3d( (-RemeshBoolMesh.GetBounds().Center()) + 0.5*ImportRadius*FVector3d(0.0,0,0) ),
    &FinalBooleanResult, FMeshBoolean::EBooleanOp::Intersect);
FinalDifferenceOp.Compute();

// The boolean probably has some small cracks around the border, find them and fill them
FMeshBoundaryLoops LoopsCalc(&FinalBooleanResult);
UE_LOG(LogBlankProgram, Display, TEXT("Final Boolean Mesh has %d holes"), LoopsCalc.GetLoopCount());
for (const FEdgeLoop& Loop : LoopsCalc.Loops)
{
    FMinimalHoleFiller Filler(&FinalBooleanResult, Loop);
    Filler.Fill();
}
FAxisAlignedBox3d FinalBooleanBBox = FinalBooleanResult.GetBounds();
MeshTransforms::Translate(FinalBooleanResult,
    (RemeshBoolMesh.GetBounds().Max.X + 0.6*FinalBooleanBBox.Width())*FVector3d::UnitX() + 0.5*FinalBooleanBBox.Height()*FVector3d::UnitZ() );

// write out the sequence of meshes
    FDynamicMeshOBJWriter::Write("..\\..\\Source\\Programs\\CommandLineGeometryTest\\HoleyBunny_processed.obj", 
    { AccumMesh, SolidMesh, SimplifiedSolidMesh, BooleanResult, RemeshBoolMesh, FinalBooleanResult }, true);

Import and Attributes

Ok lets step through the code. The first block just reads a mesh, prints some information, and computes normals (just a reminder, as I mentioned above, FDynamicMeshOBJReader is not part of UE4, this is a class included with the sample code). Note the call to FDynamicMesh3::ReverseOrientation(). Whether this is necessary depends on your input file, but generally, UE4 uses a left-handed coordinate system, while most content tools are right-handed. This means that a right-handed mesh, when imported into UE4, will be “inside-out”, and so (for example) the positive/outward surface normal direction for that mesh would point inwards. If we ReverseOrientation() on import, and again on export, then things will be fine.

// import an OBJ mesh. The path below is relative to the default path that Visual Studio will execute CommandLineGeometryTest.exe,
// when using a normal UE4.26 auto-generated UE.sln file. If things change you might need to update this path
FDynamicMesh3 ImportMesh;
FDynamicMeshOBJReader::Read("..\\..\\Source\\Programs\\CommandLineGeometryTest\\HoleyBunny.obj", ImportMesh, true, true, true);
// flip to UE orientation
ImportMesh.ReverseOrientation();

// print some mesh stats
UE_LOG(LogBlankProgram, Display, TEXT("Mesh has %d vertices, %d triangles, %d edges"), ImportMesh.VertexCount(), ImportMesh.TriangleCount(), ImportMesh.EdgeCount());
UE_LOG(LogBlankProgram, Display, TEXT("Mesh has %d normals"), ImportMesh.Attributes()->PrimaryNormals()->ElementCount());
UE_LOG(LogBlankProgram, Display, TEXT("Mesh has %d UVs"), ImportMesh.Attributes()->PrimaryUV()->ElementCount());

// compute per-vertex normals
FMeshNormals::QuickComputeVertexNormals(ImportMesh);

There is nothing special about the logging calls, I just wanted to have a reason to mention the calls to Attributes(), which return a FDynamicMeshAttributeSet. The design of FDynamicMesh3 is quite similar to DMesh3 from Geometry3Sharp, to the point where this documentation on DMesh3 basically applies directly to FDynamicMesh3. However, one major addition that has been made in the GeometryProcesing implementation is support for arbitrary Attribute Sets, including per-triangle indexed Attributes which allow for representation of things like split normals and proper UV islands/atlases/overlays (depending on your preferred terminology). Generally, mesh editing operations in GeometryProcessing (eg like the mesh edge splits/flips/collapses, the FDynamicMeshEditor, the Simplifier and Remeshers, change tracking, etc) handle updating the Attribute Sets automatically.

Generating a Box Mesh

The next step is to generate a simple box, that we are going to append to the imported bunny a bunch of times. There are a variety of mesh generators in the /Public/Generators/ subfolder in the GeometricObjects module. FMinimalBoxMeshGenerator makes a box with 12 triangles, and we’ll use FGridBoxMeshGenerator later to generate a subdivided box. The GeometricObjects module also includes a library of basic geometry and vector-math types, templated on Real type, with typedefs for float and double. So FAxisAlignedBox3d is a 3D double-precision axis-aligned bounding box, while FAxisAlignedBox2f is a 2D float variant. Conversions to the standard FVector/FBox/etc UE4 types are defined wherever possible (implicit where safe, otherwise via casts). However generally the GeometryProcessing library will calculate in double precision if not templated on Real type.

// generate a small box mesh to append multiple times
FAxisAlignedBox3d ImportBounds = ImportMesh.GetBounds();
double ImportRadius = ImportBounds.DiagonalLength() * 0.5;
FMinimalBoxMeshGenerator SmallBoxGen;
SmallBoxGen.Box = FOrientedBox3d(FVector3d::Zero(), ImportRadius * 0.05 * FVector3d::One());
FDynamicMesh3 SmallBoxMesh(&SmallBoxGen.Generate());

You will note that nearly every type is prefixed with “F”. This is a UE4 convention, generally all structs and classes have an F prefix. Similarly the code here basically follows the UE4 coding standard (which includes quite a bit more whitespace than I generally prefer, but it is what it is).

Making an AABBTree

This is a one-liner, the constructor for FDynamicMeshAABBTree3 will automatically build the AABBTree (this can be disabled with an optional argument). The AABBTree construction is quite fast and there generally is no excuse to use something less reliable (or, horror of horrors, a linear search). Similarly copying a FDynamicMesh3 is very quick, as the storage for the mesh does not involve per-element pointers, it is all in chunked arrays (see TDynamicVector) that can be memcopied. Finally this block creates a FDynamicMeshEditor, which implements a many common low-level mesh editing operations. If you need to do something it doesn’t do, it’s generally a better idea to try and break your problem down into operations that are already implemented, even at the cost of some efficiency, as handling Attribute Set updates gets quite hairy.

// create a bounding-box tree, then copy the imported mesh and make an Editor for it
FDynamicMeshAABBTree3 ImportBVTree(&ImportMesh);
FDynamicMesh3 AccumMesh(ImportMesh);
FDynamicMeshEditor MeshEditor(&AccumMesh);

If you were to look at the code for FDynamicMeshAABBTree3, you would find that it is just a typedef for TMeshAABBTree3<FDynamicMesh3>. The AABBTree is templated on mesh type, as only a few functions on the mesh are required. The FTriangleMeshAdapterd struct can be used to wrap virtually any indexed mesh in an API that will work with TMeshAABBTree3, as well as TMeshQueries<T> which supports many types of generic mesh…queries.

AABBTree Queries

This is a large block because we’re going to do a bit of logic, but the critical parts are the calls to FDynamicMeshAABBTree3::FindNearestTriangle() and FDynamicMeshAABBTree3::FindNearestHitTriangle(). These are two of the most common queries on an AABBTree. Note that in both cases, the query only returns an integer triangle ID/index, and then TMeshQueries<T> is used to execute and return a FDistPoint3Triangle3d/FIntrRay3Triangle3d object. Those classes can also be used directly. They return various information calculated for a point-triangle distance query, or ray-tri intersection. Distance and Intersection queries in GeometryProcessing are generally implemented in this style, and the calculation objects store any useful intermediate information which otherwise might be discarded. In some cases the FDistXY / FIntrXY class has static functions that will do a more minimal computation. The AABBTree class also has a ::FindNearestPoint() helper function (but no similar ray-intersection variant).

// append the small box mesh a bunch of times, at random-ish locations, based on a Spherical Fibonacci distribution
TSphericalFibonacci<double> PointGen(64);
for (int32 k = 0; k < PointGen.Num(); ++k)
{
    // point on a bounding sphere
    FVector3d Point = (ImportRadius * PointGen.Point(k)) + ImportBounds.Center();

    // compute the nearest point on the imported mesh
    double NearDistSqr;
    int32 NearestTriID = ImportBVTree.FindNearestTriangle(Point, NearDistSqr);
    if (ImportMesh.IsTriangle(NearestTriID) == false)
        continue;
    FDistPoint3Triangle3d DistQueryResult = TMeshQueries<FDynamicMesh3>::TriangleDistance(ImportMesh, NearestTriID, Point);

    // compute the intersection between the imported mesh and a ray from the point to the mesh center
    FRay3d RayToCenter(Point, (ImportBounds.Center() - Point).Normalized() );
    int32 HitTriID = ImportBVTree.FindNearestHitTriangle(RayToCenter);
    if (HitTriID == FDynamicMesh3::InvalidID)
        continue;
    FIntrRay3Triangle3d HitQueryResult = TMeshQueries<FDynamicMesh3>::TriangleIntersection(ImportMesh, HitTriID, RayToCenter);

    // pick the closer point
    bool bUseRayIntersection = (HitQueryResult.RayParameter < DistQueryResult.Get());
    FVector3d UsePoint = (bUseRayIntersection) ? RayToCenter.PointAt(HitQueryResult.RayParameter) : DistQueryResult.ClosestTrianglePoint;

    FVector3d TriBaryCoords = (bUseRayIntersection) ? HitQueryResult.TriangleBaryCoords : DistQueryResult.TriangleBaryCoords;
    FVector3d UseNormal = ImportMesh.GetTriBaryNormal(NearestTriID, TriBaryCoords.X, TriBaryCoords.Y, TriBaryCoords.Z);

    // position/orientation to use to append the box
    FFrame3d TriFrame(UsePoint, UseNormal);

    // append the box via the Editor
    FMeshIndexMappings TmpMappings;
    MeshEditor.AppendMesh(&SmallBoxMesh, TmpMappings,
        [TriFrame](int32 vid, const FVector3d& Vertex) { return TriFrame.FromFramePoint(Vertex); },
        [TriFrame](int32 vid, const FVector3d& Normal) { return TriFrame.FromFrameVector(Normal); });
}

The final call in the block above appends the SmallBoxMesh we created above, via the FDynamicMeshEditor. The two lambdas transform the vertices and normals of the box mesh (which is centered at the origin) to be aligned with the surface position and normal we calculated using the distance/ray-intersection. This is done via a FFrame3d, which is a class that is heavily used in GeometryProcessing.

A TFrame3<T> is a 3D position (referred to as the .Origin) and orientation (.Rotation), which is represented as a TQuaternion<T>, so essentially like a standard FTransform without any scaling. However the TFrame3 class has an API that allows you to treat the Frame as a set of 3D orthogonal axes positioned in space. So for example the X(), Y(), and Z() functions return the three axes. There are also various ToFrame() and FromFrame() functions (in the case of FVector3<T> you must differentiate between Point and Vector, but for other types there are overloads). ToFrame() maps geometry into the local coordinate system of the frame, so for example ToFramePoint(P) returns returns a new position that measures the distance from P to the Frame.Origin along each of it’s three axes. FromFrame() does the inverse, mapping points “in” the Frame into “world space” (as far as the Frame is concerned). So in the code above, we are treating the cube as being “in” the Frame coordinate system, and mapping it onto the mesh surface.

A final note, the FFrame3d(Point, Normal) constructor used above results in a Frame with “Z” aligned with the Normal, and somewhat arbitrary X and Y axes. In many cases you might wish to construct a Frame with specific tangent-plane axes. There is a constructor that takes X/Y/Z, but a frequent case is where you have a Normal and another direction that is not necessarily orthogonal to the Normal. In that case you can construct with Z=Normal and then use the ::ConstrainedAlignAxis() function to best-align one of the other Frame axes (eg axis X/0) with a target direction, by rotating around the Z.

“Solidification” with the Fast Mesh Winding Number

Several previous Gradientspace tutorials [1] [2] used the Fast Mesh Winding Number to reliably compute Point Containment (ie inside/outside testing) on meshes. An implementation of the Fast Mesh Winding Number is available in GeometricObjects as TFastWindingTree<T>, where T is FDynamicMesh3 or a MeshAdapter. This data structure is built on top of a TMeshAABBTree<T>. In the code below we construct one of these, and then use a TImplicitSolidify<T> object to generate a new “solidified” mesh. TImplicitSolidify interprets the inside/outside values produced by TFastWindingTree as an Implicit Surface (see previous tutorial) and uses the FMarchingCubes class to generate a triangle mesh for that implicit.

// make a new AABBTree for the accumulated mesh-with-boxes
FDynamicMeshAABBTree3 AccumMeshBVTree(&AccumMesh);
// build a fast-winding-number evaluation data structure
TFastWindingTree<FDynamicMesh3> FastWinding(&AccumMeshBVTree);

// "solidify" the mesh by extracting an iso-surface of the fast-winding field, using marching cubes
// (this all happens inside TImplicitSolidify)
int32 TargetVoxelCount = 64;
double ExtendBounds = 2.0;
TImplicitSolidify<FDynamicMesh3> SolidifyCalc(&AccumMesh, &AccumMeshBVTree, &FastWinding);
SolidifyCalc.SetCellSizeAndExtendBounds(AccumMeshBVTree.GetBoundingBox(), ExtendBounds, TargetVoxelCount);
SolidifyCalc.WindingThreshold = 0.5;
SolidifyCalc.SurfaceSearchSteps = 5;
SolidifyCalc.bSolidAtBoundaries = true;
SolidifyCalc.ExtendBounds = ExtendBounds;
FDynamicMesh3 SolidMesh(&SolidifyCalc.Generate());
// position the mesh to the right of the imported mesh
MeshTransforms::Translate(SolidMesh, SolidMesh.GetBounds().Width() * FVector3d::UnitX());

The TImplicitSolidify code is relatively straightforward, and we could have easily used FMarchingCubes here directly. However, you will find that there are many such “helper” classes like TImplicitSolidify in the GeometryProcessing modules. These classes reduce the amount of boilerplate necessary to do common mesh processing operations, making it easier to implement “recipes” and/or user interfaces that expose certain parameters.

Mesh Morphological Operations and Mesh Simplification

We’ve now generated a “solid” mesh of our holey-bunny-plus-boxes. The next step is to offset this mesh. Offset can be done directly on the mesh triangles, but it can also be considered a Morphological Operation, sometimes referred to as ‘Dilation’ (and a negative offset would be an ‘Erosion’). There are also more interesting Morphological Operations like ‘Opening’ (Erode, then Dilate) and ‘Closure’ (Dilate, then Erode), which is particularly useful for filling small holes and cavities. These are generally quite difficult to implement directly on a mesh, but easily done with implicit surface / level-set techniques in the TImplicitMorphology<T> class. Similar to TImplicitSolidify, this class builds the necessary data structures and uses FMarchingCubes to generate an output mesh.

// offset the solidified mesh
double OffsetDistance = ImportRadius * 0.1;
TImplicitMorphology<FDynamicMesh3> ImplicitMorphology;
ImplicitMorphology.MorphologyOp = TImplicitMorphology<FDynamicMesh3>::EMorphologyOp::Dilate;
ImplicitMorphology.Source = &SolidMesh;
FDynamicMeshAABBTree3 SolidSpatial(&SolidMesh);
ImplicitMorphology.SourceSpatial = &SolidSpatial;
ImplicitMorphology.SetCellSizesAndDistance(SolidMesh.GetCachedBounds(), OffsetDistance, 64, 64);
FDynamicMesh3 OffsetSolidMesh(&ImplicitMorphology.Generate());

// simplify the offset mesh
FDynamicMesh3 SimplifiedSolidMesh(OffsetSolidMesh);
FQEMSimplification Simplifier(&SimplifiedSolidMesh);
Simplifier.SimplifyToTriangleCount(5000);
// position to the right
MeshTransforms::Translate(SimplifiedSolidMesh, SimplifiedSolidMesh.GetBounds().Width() * FVector3d::UnitX());

Marching Cubes meshes are generally very dense, and so it’s a common pattern to Simplify the mesh afterwards. This can be done in a few lines using the FQEMSimplification class, which has a variety of simplification criteria you can specify. There are also several other Simplifier implementations, in particular FAttrMeshSimplification will consider the normal and UV Attribute Overlays.

Mesh Booleans (!!!)

It’s the moment you’ve been waiting for - Mesh Booleans! In this block we first use FSphereGenerator and FGridBoxGenerator to generate two meshes, and then use FMeshBoolean to subtract the Box from the Sphere. The FMeshBoolean constructor takes a transform for each input mesh, but only supports two input meshes (ie it’s not an N-way Boolean). If you would like to use multiple input meshes, you will have to use repeated FMeshBoolean operations, but if the inputs are not intersecting it can be more efficient to combine them using FDynamicMeshEditor::AppendMesh() first.

// generate a sphere mesh
FSphereGenerator SphereGen;
SphereGen.Radius = ImportMesh.GetBounds().MaxDim() * 0.6;
SphereGen.NumPhi = SphereGen.NumTheta = 10;
SphereGen.bPolygroupPerQuad = true;
SphereGen.Generate();
FDynamicMesh3 SphereMesh(&SphereGen);

// generate a box mesh
FGridBoxMeshGenerator BoxGen;
BoxGen.Box = FOrientedBox3d(FVector3d::Zero(), SphereGen.Radius * FVector3d::One());
BoxGen.EdgeVertices = FIndex3i(4, 5, 6);
BoxGen.bPolygroupPerQuad = false;
BoxGen.Generate();
FDynamicMesh3 BoxMesh(&BoxGen);

// subtract the box from the sphere (the box is transformed within the FMeshBoolean)
FDynamicMesh3 BooleanResult;
FMeshBoolean DifferenceOp(
    &SphereMesh, FTransform3d::Identity(),
    &BoxMesh, FTransform3d(FQuaterniond(FVector3d::UnitY(), 45.0, true), SphereGen.Radius*FVector3d(1,-1,1)),
    &BooleanResult, FMeshBoolean::EBooleanOp::Difference);
if (DifferenceOp.Compute() == false)
{
    UE_LOG(LogGeometryTest, Display, TEXT("Boolean Failed!"));
}
FAxisAlignedBox3d BooleanBBox = BooleanResult.GetBounds();
MeshTransforms::Translate(BooleanResult, 
    (SimplifiedSolidMesh.GetBounds().Max.X + 0.6*BooleanBBox.Width())* FVector3d::UnitX() + 0.5*BooleanBBox.Height()*FVector3d::UnitZ());

Note that the MeshBoolean is not 100% reliable. Below I will show how to (try to) handle failures.

Remeshing

Next we apply a pass of isotropic triangular remeshing to the Boolean result. This is a standard step if you are planning on doing further mesh processing like deformations/smoothing/etc, as output of a Mesh Boolean often has highly variable triangle size/density (which constraints how the mesh can move) and sliver triangles that can cause numerical issues. The standard approach is to use FRemesher and run a fixed number of passes over the full mesh. Below I used FQueueRemesher, which produces nearly the same result, but rather than full-mesh passes, an “active queue” of meshes that need to be processed is tracked. This can be significantly faster (particularly on large meshes).

// make a copy of the boolean mesh, and apply Remeshing
FDynamicMesh3 RemeshBoolMesh(BooleanResult);
RemeshBoolMesh.DiscardAttributes();
FQueueRemesher Remesher(&RemeshBoolMesh);
Remesher.SetTargetEdgeLength(ImportRadius * 0.05);
Remesher.SmoothSpeedT = 0.5;
Remesher.FastestRemesh();
MeshTransforms::Translate(RemeshBoolMesh, 1.1*RemeshBoolMesh.GetBounds().Width() * FVector3d::UnitX());

I covered the basics of Isotropic Remeshing in a previous G3Sharp tutorial. That tutorial basically applies directly to the UE4 GeometryProcessing implementation, down to the type and field names (don’t forget to add F). However the UE4 version is quite a bit more capable, for example the FQueueRemesher is much faster, and there is also FSubRegionRemesher which can remesh a portion of a larger mesh.

Two notes about the block above. First, I did not use a Projection Target (described in the linked tutorial), so the nice crisp edges of the sphere-minus-cube will be smoothed away. Setting up a Projection Target only takes a few lines, search for any usage of FMeshProjectionTarget in the Engine code. Second, the first thing I did after making the mesh copy above is to call RemeshBoolMesh.DiscardAttributes(). This call removes all attribute overlays from the mesh, specifically the per-triangle the UV and Normal layers. The Remeshers do support remeshing with per-triangle attributes, however it is more complex because those attributes have additional topological constraints that must be preserved. The utility function FMeshConstraintsUtil::ConstrainAllBoundariesAndSeams() can be used to more-or-less automatically set all that up, but even just calling that is a bit complicated, so I thought I would save it for a future tutorial (look up FRemeshMeshOp if you want to see an example).

Hole Filling and Boolean Failure Handling

View fullsize

Finally, we are going to compute the Boolean Intersection of the smoothed-out-sphere-minus-cube with the solidified-offset-cubesbunny. This is again just a single construction of a FMeshBoolean object and a call to Compute(). However, in this case the input objects are quite complex and it’s relatively likely that the Mesh Boolean output is not fully closed.

Why? Well, Mesh Booleans are notoriously difficult to compute reliably. If you have used tools like Maya or Max over many years, you will recall that Mesh Booleans used to be extremely unreliable, and then at some points they switched to being somewhat more reliable. This is mainly due to those packages changing which third-party Mesh Boolean library they were using. There actually aren’t that many to choose from. The CGAL library has quite powerful Polyhedral Booleans, but they are very, very slow, and cannot be redistributed with a commercial game engine. Carve is used by Blender, and is quite good, but it is GPL-licensed. Cork is reasonably capable but not actively maintained. The Mesh Booleans in LibIGL use the recently-introduced Mesh Arrangements technique and are basically the current state-of-the-art, but are also somewhat slow on large meshes, and depend on some CGAL code. If you dig, you will find that the Booleans available in many commercial tools or open-source libraries are using one of these four.

Another complication with third-party mesh boolean libraries is they generally don’t support arbitrary complex mesh atttributes, like the indexed per-triangle overlays I mentioned above. So, in UE4.26 we wrote our own. One benefit to writing an implementation specifically for FDynamicMesh3 is that we could take advantage of some modern triangle-mesh-processing techniques. For example when previous Mesh Booleans failed catastrophically, ie with parts of the output disappearing, it was often because they couldn’t geometrically tell what was “inside” and “outside”. Now that we have the Fast Mesh Winding Number, this is basically a solved problem, and as a result UE4’s FMeshBoolean tends to fail in a way that is localized, and often recoverable. For example in the images above-right, the sphere mesh has a giant hole in it, usually a no-go for a Mesh Boolean, but as long as the intersection curve is well-defined, FMeshBoolean will usually work. Even if it does (lower image), the Boolean no longer really makes sense, but the failure is not catastrophic, we just get a hole where the hole is.

So, all of that is a long-winded way of saying that if your FMeshBoolean::Compute() returns false, it’s probably got some holes and you can fill them. The FMeshBoundaryLoops object will extract a set of FEdgeLoop objects that represent the open boundary loops of a mesh (another surprisingly difficult problem…) and then FMinimalHoleFiller will fill them (we also have FPlanarHoleFiller and FSmoothHoleFiller but they likely aren’t applicable in this context). Note that most of the “holes” are zero-area cracks along the intersection curve between the two objects, so it can be helpful to collapse away degenerate triangles (something the library does not do automatically, yet).

// subtract the remeshed sphere from the offset-solidified-cubesbunny
FDynamicMesh3 FinalBooleanResult;
FMeshBoolean FinalDifferenceOp(
    &SimplifiedSolidMesh, FTransform3d(-SimplifiedSolidMesh.GetBounds().Center()),
    &RemeshBoolMesh, FTransform3d( (-RemeshBoolMesh.GetBounds().Center()) + 0.5*ImportRadius*FVector3d(0.0,0,0) ),
    &FinalBooleanResult, FMeshBoolean::EBooleanOp::Intersect);
FinalDifferenceOp.Compute();

// The boolean probably has some small cracks around the border, find them and fill them
FMeshBoundaryLoops LoopsCalc(&FinalBooleanResult);
UE_LOG(LogGeometryTest, Display, TEXT("Final Boolean Mesh has %d holes"), LoopsCalc.GetLoopCount());
for (const FEdgeLoop& Loop : LoopsCalc.Loops)
{
    FMinimalHoleFiller Filler(&FinalBooleanResult, Loop);
    Filler.Fill();
}
FAxisAlignedBox3d FinalBooleanBBox = FinalBooleanResult.GetBounds();
MeshTransforms::Translate(FinalBooleanResult,
    (RemeshBoolMesh.GetBounds().Max.X + 0.6*FinalBooleanBBox.Width())*FVector3d::UnitX() + 0.5*FinalBooleanBBox.Height()*FVector3d::UnitZ() );

The GeometryProcessing Library

The examples above have shown you how to use a handful of the data structures and algorithms in the GeometryProcessing Plugin. There are many, many more, and even the ones used above have many more options and capabilities. You will find all the code in \Engine\Plugins\Experimental\GeometryProcessing\, there are four modules:

• GeometryObjects: templated vector-math and geometry types, Distance and Intersection computations, spatial data structures like AABBTrees/Octrees/Grids/HashTables, and Grids, Mesh Generators, 2D graphs and polygons, generic mesh algorithms (not specific to FDynamicMesh3), and Implicit Surfaces

• DynamicMesh: FDynamicMesh3 and related data structures, booleans and cutting, editing operations like extrusions and offsets, deformation, sampling, parameterization, baking, shape fitting, and so on. Nearly all the Mesh Processing code is here

• GeometryAlgorithms: Computational-Geometry algorithm implementations like Delaunay Triangulation, Convex Hulls, Line Segment Arrangement, and so on. This module uses the third-party boost-licensed GTEngine library in some places (included in /Private/ThirdParty/) and also Shewchuck’s Exact Predicates.

• MeshConversion: Helper classes for converting between mesh types. Currently main for converting to/from FMeshDescription, the other main mesh format used in Unreal Engine.

I encourage you to explore. If you find a class or function that looks interesting but aren’t sure exactly how to use it, you will almost certainly find some usage elsewhere in the Engine codebase. One great thing about Unreal Engine is you have the code for literally everything, including the Editor. So if you see something interesting when using the Modeling Tools Editor Mode, and want to know how to do it yourself, you can find the code in the MeshModelingToolset plugin. This is built on top of GeometryProcessing and implements nearly all the interactive in-Editor Tools.

Many of those Tools (particularly the ones that are more “set options and process” and less “pointy-clicky”) are split into the Tool-level code (in the MeshModelingTools module) and what we call an “Operator” (in the ModelingOperators module). An Operator is basically an object that executes a more complex multi-step mesh processing recipe, with higher-level parameters exposed. So for example the FBooleanMeshesOp operator ultimately runs a FMeshBoolean on two input FDynamicMesh3, however it will automatically do the hole-filling repair step above if bAttemptFixHoles is set to true. Operators are safe to run on background threads, and take a FProgressCancel object, which can be used to safely abort their computation if they take too long.

Create your own In-Editor Geometry Processing Tools

This tutorial has shown you how to use the GeometryProcessing modules in a a command-line tool. However, as I mentioned above, this plugin can be used to implement the same mesh processing in the Editor. My previous tutorial on using LibIGL in UE 4.24 to make an interactive mesh smoothing tool in the Editor already showed how to do this!! That tutorial ultimately reduced the problem to implementing a MakeMeshProcessingFunction() function that returned a TUniqueFunction<void(FDynamicMesh3&)>, ie a lambda that processed the input FDynamicMesh3. In that tutorial we wanted to call LibIGL code so we converted to/from the LibIGL mesh format. But now you know that you can also just skip LibIGL and edit the input mesh using GeometryProcessing code directly!

I have updated that tutorial for UE 4.26, there is a small addendum explaining the necessary changes. In UE 4.26 we added a new “base tool” class UBaseGeometryProcessingTool which makes the code for that tutorial much simpler.

As I mentioned, the GeometryProcessing Plugin is made up of Runtime modules, so there is nothing stopping you from using it in your games, either. I will be exploring this in future tutorials - stay tuned!

Interactive Mesh Processing with libigl in Unreal Engine 4.24

The code for this tutorial has been updated for UE 4.26 - see the update notes below!

View fullsize

This tutorial describes how to embed libigl triangle mesh processing code inside an Interactive Tool in the Unreal Editor. Interactive Tools are an Experimental feature in UE 4.24, meaning they are not documented or officially supported. However this doesn’t mean they aren’t useful! Using the sample project below, you will be able to select a StaticMesh Actor/Component in the scene, and then start a Tool that applies Laplacian Mesh Smoothing to the underlying StaticMesh Asset. The smoothing parameters (ie sliders, checkboxes, etc) are automatically exposed in a DetailsView panel. The gif to the right shows a quick demo of applying this smoothing to one of the chairs in the default UE scene.

(Of course the point of this tutorial is not just to provide a mesh smoothing Tool, it’s to show you how to create your own Tools inside the Editor. Libigl is just a convenient (and awesome) example.)

(Mandatory Disclaimer: your author, Ryan Schmidt, is an employee of Epic Games)

What is libigl? Why would I want to do this?

Libigl (https://libigl.github.io/) is an open-source C++ mesh processing library (github) initially created by two computer graphics PhD students (now professors) Alec Jacobson (UToronto) and Danielle Panozzo (NYU). If you see a cool SIGGRAPH paper that does something crazy with meshes, there is a good chance it is based on libigl. For example, Alec and his student Derek Liu had a paper Cubic Stylization at SIGGRAPH Asia 2019 that was reviewed on the popular 2-Minute Papers youtube channel (click here to watch). Their code is open-source (github) and built on top of libigl.

So, if you wanted to Cubic-Stylize some of your game assets, you could try to download and compile their software and run it on your objects. However as research software, it’s UI has…limitations (your assets are all OBJ files, right?). If you could get this code running in the Editor as an Interactive Tool, you could apply it to your already-created-and-configured Unreal Assets.

Did I mention that libigl has an enormous tutorial that demonstrates a ton of amazing geometry processing algorithms, with sample code easily cut-and-pasteable? In fact the mesh smoothing in the gif above is just the 205_Laplacian sample code [link] and the purpose of this tutorial is to get you as close as possible to literally being able to cut-and-paste libigl-based code into an Editor Tool.

(By the way, libigl is not the only C++ mesh processing library out there, and the plugin provided as part of this tutorial should be adaptable to any of those, too - details later on.)

UE 4.24 Sample Project & MeshProcessingPlugin

This is the tab for the Mesh Processing Mode

The Tool Buttons will activate when a valid object is selected

You must Accept the active Tool to commit the current preview to the underlying StaticMesh asset

To make it trivial to use libigl inside of UE, I have written a small plugin that provides two things. First, it’s an Editor Mode plugin, which means it adds its own tab in the Editor Modes tab panel on the left-hand side of the Editor. The tab for this mode will have a default “Wrench-Brush-Pencil” icon (see images to right).

When you select this new mode, a small toolbar of buttons will appear below the main toolbar. It will contain two buttons labeled IGLSmooth and Export. You have to select a StaticMesh object in the viewport to start these Tools. The Export Tool will allow you to export the selected mesh as an OBJ file, but I won’t cover this in more detail in the tutorial.

When you start a tool, two additional buttons will appear, labeled Accept and Cancel. A tool is like a mini-mode with a live preview, so what you see in the chair-smoothing gif above is not actually affecting the input StaticMesh Asset yet. Selecting Accept will commit the current preview to the Asset (or Cancel to discard it). Note that clicking Accept edits the Asset but does not save it! You must save the Asset manually (eg Save-All from the File menu, select the Asset and click Save, etc, etc)

And that’s it. This sample project is on Github at github.com/gradientspace/UnrealMeshProcessingTools in the UE4.24 subfolder. The project itself is just an empty default Game C++ project, all the actual code is in the Editor Mode Plugin located in the subdirectory /Plugins/MeshProcessingPlugin/. You should be able to copy that Plugin to another UE 4.24 Project without difficulty, if you prefer.

Note that the Plugin contains copies of libigl and Eigen (an awesome header-only math library that libigl is built on). If you want to use your own copies of these at some other system path, you can edit the quoted strings in Plugins/MeshProcessingPlugin/Source/MeshProcessingPlugin/MeshProcessingPlugin.Build.cs (you can use absolute paths).

Details on how to install UE4.24 and import this sample project are provided below, if you need it. But lets cover the libigl-related parts first.

IGLSmoothingTool Code

The code you need to care about for this tutorial is located in /Plugins/MeshProcessingPlugin/Source/MeshProcessingPlugin/Public/Tools/IGLSmoothingTool.h and Private/Tools/IGLSmoothingTool.cpp. There are roughly 40 lines of non-whitespace/comment code between these two files, so there really is not much to it beyond the actual libigl code. I’ll describe all the salient parts. It will feel a bit like magic that this works. Don’t worry about it - that’s the whole point of the Editor Tools Framework! There are 3 objects we have to provide for the magic to happen - a ToolBuilder, and PropertySet, and the Tool itself:

The ToolBuilder

UIGLSmoothingToolBuilder is a trivial factory class that creates instances of UIGLSmoothingTool. The ToolBuilder code should be straightfoward, it’s just allocating a new object using NewObject<T>, which is an Unrealism for creating new UObject-derived subclasses (many things in Unreal are UObjects). If this is your first exposure to Unreal Engine you might note that all classes/structs are prefixed with U or F. U means UObject, F is everything-else (just go with it for now). Similarly just ignore the UCLASS() and GENERATED_BODY() macros. Unreal has a custom pre-processor/code-generator that parses these macros, so they have to be there.

The PropertySet

Next is UIGLSmoothingToolProperties. This is a UInteractiveToolPropertySet implementation, which means it provides a list of configuration variables that will appear in the properties panel on the left-hand side of the Editor when the Tool is active. When you change a property the Tool will recompute the preview. The properties have to be annotated with UPROPERTY() macros but again you can cut-paste what is there if you wanted to add more (only certain types are supported though - stick to int/float/boolean, and enums if you need it, see the MeshExportTool.h header for an example). Note that as long as you initialize the values in the header you don’t need anything for this class in the cpp file.

The Tool

Finally we have UIGLSmoothingTool. The header declaration of this class isn’t really important, if you want to change the libigl code all that matters is the implementation of the ::MakeMeshProcessingFunction() function. I have included the code for this function below (slightly edited to compact it vertically).

The basic task of this function is to create and return a lambda that updates the input FDynamicMesh3 (our editable Unreal mesh). To do that with libigl we will first have to convert the mesh vertices and triangles to matrix format. The rest of the code is taken from libigl sample Laplacian_205 (github), lightly edited to remove the origin scaling/translation in that sample (otherwise we would need to invert that transformation).

TUniqueFunction<void(FDynamicMesh3&)> UIGLSmoothingTool::MakeMeshProcessingFunction()
{
    // make local copies of current settings
    int SolveIterations = Properties->Iterations;
    float Smoothness = Properties->Smoothness;

    // construct compute lambda
    auto EditFunction = [Smoothness, SolveIterations](FDynamicMesh3& ResultMesh)
    {
        Eigen::MatrixXd V;      Eigen::MatrixXi F;    
        iglext::DynamicMeshToIGLMesh(ResultMesh, V, F);    // convert FDynamicMesh3 to igl mesh representation

        Eigen::SparseMatrix<double> L;
        igl::cotmatrix(V, F, L);    // Compute Laplace-Beltrami operator L

        Eigen::MatrixXd U = V;      // smoothed positions will be computed in U

        for (int k = 0; k < SolveIterations; ++k)
        {
            Eigen::SparseMatrix<double> M;     // Recompute mass matrix on each step
            igl::massmatrix(U, F, igl::MASSMATRIX_TYPE_BARYCENTRIC, M);

            const auto& S = (M - Smoothness * L);
            Eigen::SimplicialLLT<Eigen::SparseMatrix<double>> solver(S);
            U = solver.solve(M * U).eval();    // Solve (M-delta*L) U = M*U
        }
        
        iglext::SetVertexPositions(ResultMesh, U);   // copy updated positions back to FDynamicMesh3
    };

    return MoveTemp(EditFunction);  // return compute lambda
}

The conversion between the FDynamicMesh3 and the libigl V/F mesh format is done by utility code in /Tools/IGLUtil.h. If you want to use some other mesh library, you will need to write similar converters. FDynamicMesh3 supports many capabilities not available in most open-source mesh libraries, such as split normals, multiple layers of UV maps, deleting vertices and triangles, etc. This class is a ported and evolved version of the DMesh3 format from the geometry3Sharp library, which I documented here, if you are interested in using it directly.

Note also that the local copies of the configuration settings SolveIterations and Smoothness are pretty important (see extended comments in the github cpp). The lambda we return here (TUniqueFunction is an Unreal version of std::function) will be called from a different thread. So we cannot reference the Properties object directly.

Note also that multiple instances of the lambda may execute simultaneously, as the Tool framework will spawns new computations when the user edits settings, rather than wait for the previous one to finish. If your code is going to depend on global variables/etc, you will need to use a lock. Unreal’s FCriticalSection is an easy way to do this (please post in the comments if you would like me to add instructions on doing this, or handle it in the parent MeshProcessingTool).

That’s it! (p.s. Live Coding)

You now know everything you need to do to write libigl code that will be run inside an Interactive Tool in the Unreal Editor. That’s really all there is to it. You can edit the libigl code in Visual Studio, hit Play to recompile and launch the Editor, and try out your changes. This is the development loop for Unreal Engine.

….except, Unreal Engine also supports hot reloading of C++. This feature is called Live Coding and it will let you iterate much more quickly. Live Coding is not enabled by default, but if you click on the Compile drop-down arrow in the main toolbar, you can toggle on Enable Live Coding (see image to the right).

Then you just hit Ctrl+Alt+F11 in the Unreal Editor to recompile and patch the currently-running Editor. The Live Coding Log Window will show compiler output and/or errors. Note that some changes cannot be made with Live Coding. In particular, you cannot add/edit UPROPERTY() fields in the Properties object - changes to those require a restart. (In this context I do recommend Cancelling out of the Tool before doing the hot-reload, otherwise you might get crashes when you Cancel later).

Detailed Setup Instructions

The instructions above explain how to do the libigl part of this tutorial. However if you are new to UE4, it might not be obvious how to get to the point where you will be able to edit that block of libigl code. So, this section will walk you through it step by step.

Step 0 - Pre-requisites

To do anything with Unreal Engine you will need to have a C++ development environment set up. Currently this tutorial only works on Windows, so you will need to install Visual Studio - I used Visual Studio Community 2019 which is free and can be downloaded here.

You also need to get the sample project from Github, at https://github.com/gradientspace/UnrealMeshProcessingTools. The simplest way to get the project is to just download a zip of the repository, which you can do by clicking here. Alternately you can clone out and/or fork the repository - you’ll have to find instructions on doing that elsewhere, though.

Between Visual Studio and Unreal Engine you will probably need about 30 GB of free disk space. If you are hurting for space, you should be able to customize the configuration of each to get it down to maybe 15GB total.

Step 1 - Install Unreal Engine version 4.24.x

You will need to have installed Unreal Engine 4.24 to use this tutorial. Go to https://www.unrealengine.com and click the Download button in the top-right. You will have to sign up for an Epic Games account, then download and run the Epic Games Launcher, and sign in to the launcher with that account.

Once you have done that, follow the instructions to the right to install Unreal Engine. Click the thumbnails on the bottom to see instructions for the various steps in the image captions. You can save some disk space by disabling support for the various platforms (Android, iOS, etc), as we’re only using the Editor. But don’t disable the Starter Content Packs, or you won’t have anything in the scene to work with.

I also strongly recommend that you check the box to include the full Engine/Editor source (not shown in the images). Being able to look at the Engine source can be extremely useful both for debugging and just for seeing “how things are done” when you want to do something similar. Entrian Source Search is quite good for this kind of Engine code spelunking.

Browse to the Unreal Engine tab on the left, and then the Library tab on the top bar. Click the + button to add a new engine version.

Select a version of UE 4.24 and click the Install button

Click through the EULA and then choose an install location (the default works fine). You can save some disk space by going into Options and disabling all the optional platforms. Then click Install and get a coffee.

Click the More button to get to a different dialog where you can open an existing project

On this screen click the Browse button to pop up a file browser

Browse to wherever you have your local copy of the project, select the IGLMeshProcessing.uproject file, and click Open

This dialog will pop up. Click Yes. A progress bar will appear for a while, and after that the UE Editor will open.

Once the compile finishes, Unreal Editor will launch. The Modes Tabs are in the top-left and the highlighted button will get you into the Mesh Processing Mode

Open the File menu and click on Generate Visual Studio Project

Click on the File menu again and select Open Visual Studio to launch Visual Studio with the generated project file

Step 2 - Open The Sample Project

Once you have installed Unreal Engine, the greyed-out Launch button in the top-right of the Epic Launcher will turn orange. Click that button to start the Unreal Editor. The first screen you see will be the one on the right, to Select or Create New Project. Follow the instructions in the images. You will open and compile the project you downloaded from Github, and then the Editor will launch with that project.

Once the Editor launches you can run the IGL Smooth Tool described above, by switching to the Mesh Processing Editor Mode. To get to the C++ code, you will need to first run Generate Visual Studio Project from the File menu, then Open Visual Studio, again from the File menu, to launch VS with the project code open.

Once you have launched Visual Studio, close the instance of the Unreal Editor that you already opened. When you are working with the C++ code it is likely that you will need to relaunch the Editor frequently (eg after code changes…or crashes), and also use the debugger, and this is all much smoother if launching from Visual Studio.

Step 3 - Working in Visual Studio

View fullsize

The image to the right shows the expanded code tree for the IGLMeshProcessing plugin in Visual Studio (click to enlarge it). /Plugins/ is the top-level plugins directory, below that /MeshProcessingPlugin/ is our plugin, then /Source/ is where the source code modules are, and then we have another /MeshProcessingPlugin/ folder, this is the Module and probably should have been named differently (my mistake). A Plugin is a collection of Modules, and each Module will become a separate DLL (Modules link to other Modules, Plugins are just an organizational structure).

Finally we have the /Private/ and /Public/ folders. Generally headers containing types we may need to export (in the DLL) go in /Public/ and everything else goes in /Private/. The IGLSmoothingTool code is in the /Tools/ subdirectory. We could (should?) have put the actual Tools code in a separate Module from our Editor Mode but that would complicate this sample project.

Click the Start Debugging button in Visual Studio (green triangle in the top toolbar) to launch the Editor with the IGLMeshProcessing project.

If you want to add additional Tools, the simplest thing to do is just duplicate the IGLSmoothingTool.h and .cpp, and string-replace “IGLSmoothing” with something else. If you do this, you may also need to manually “Regenerate Project Files”. You can do this by right-clicking on the IGLMeshProcessing.uproject file and selecting Generate Visual Studio project files. Visual Studio will prompt you to reload the solution, click Yes to All.

Step 4 - Import, Export, and Assets

to import a mesh file (FBX or OBJ) as a StaticMesh asset, right-click in the Content Browser and select Import To /Game

For OBJ files, configure the FBX Import options as shown here

Your mesh file will be imported as a new StaticMesh Asset in the Content Browser

Drag-and-drop the Mesh Asset into the scene. Your bunny appears!

At this point you can edit the code, run the project, and apply your libigl code to the default meshes in the scene. If you want to import other objects, you will need to know a tiny bit about the UE4 Asset system. UE4 does not have a “scene file” that contains mesh data, like you might have in Maya or Blender. In UE4 every mesh is a separate “Asset” with is stored in a .uasset file.

To Import a mesh file (OBJ, FBX, and a few other formats) as an asset, right-click in the Content Browser (the part on the bottom) and select Import to /Game. Select your file and then the FBX Import Options dialog will pop up (the FBX importer is used for OBJ files too). For “geometry processing” meshes, I recommend disabling creating of Materials. You should also un-check Remove Degenerates, or small triangles will be deleted on import (yes this is really the default). Also check Combine Meshes unless you are sure you want multiple objects (any ‘g’ or ‘o’ lines in an OBJ file will result in a separate mesh asset). Finally click Import All. If you were to do this with this bunny OBJ file (click), you would end up with a StaticMesh Asset named UEBunny. You can then drag-and-drop that mesh into the scene.

Click through to the last image on the right, and you will see a star indicator ( ☆ ) on the imported UEBunny Asset icon. This means the Asset is in the modified-but-unsaved state. Assets are saved separately from the actual scene in UE, and an imported Asset is not explicitly saved. You must either Save All (Ctrl+Shift+S) or select the Asset itself and hit Ctrl+S to Save (or use Save from the context menu). When you do this, you create a new file UEBunny.uasset in the top-level Content folder. Similarly when you edit the Asset with the IGL Smooth Tool, it will be modified but unsaved. If you shut down the Editor, you will be prompted to save any modified-but-unsaved Assets. You can skip this if you don’t want to save it. In any case, your imported mesh will not be modified or affected by what you do inside Unreal, because you are working with the Asset file, not the OBJ/FBX/etc (which is called the “Source File” in UE terminology).

Note that you can easily Re-import an external mesh by right-clicking on the Asset and selecting Reimport. Unreal remembers the import settings in the uasset file. If your mesh doesn’t import correctly (for example if there are “cracks”) you might try some of the mesh repair tools in Modeling Mode (see below).

You can Export a StaticMesh Asset by right-clicking on it and selecting Asset Actions and then Export. FBX and OBJ format are supported, however the OBJ exporter exports each triangle separately (ie more like an STL file). For your Geometry Processing experiments you might want that connectivity. In that case, use the Export button in the Mesh Processing Editor Mode, which will allow you to export an OBJ file using the MeshExportTool I have provided.

Finally, note that Unreal uses a left-handed Z-Up coordinate system (more details here). It’s unlikely that you are using this same system (it is a bit unusual outside of game engines). For example Maya and Meshmixer use right-handed Y-Up, while Blender and 3DS Max use right-handed Z-Up. The FBX Importer will convert OBJ format from right-to-left handed (and FBX specifies handedness in the file) but will not modify the Up direction. The Mesh Export Tool defaults will export right-handed Y-Up suitable for use in Maya and Meshmixer (my preferred external tools), but you can change the options.

Finally Finally, the default units in Unreal are centimeters. So an object on the scale of a unit-box (ie good for algorithms) will be tiny compared to the default ground-plane-box (which is 5 meters wide). Also note that if you scale an object in the 3D viewport using the 3D gizmo, that’s just modifying a Transform on the StaticMeshComponent, not the actual local coordinates of the mesh vertices in the Asset.

MeshProcessingPlugin Details

IGLSmoothingTool is the “end-goal” for the MeshProcessingPlugin, allowing you to write a tiny bit of libigl code that will drive a relatively complex underlying system. Much of that system is part of UE 4.24, however a few parts were written specifically for this tutorial. If you want to know how they work, here are some details (and if you don’t - skip this section!)

MeshProcessingTool

The UIGLSmoothingTool class shown above is based on a parent Tool implementation called UMeshProcessingTool. This Tool provides the “glue” that allows our libigl code to manipulate a UE4 StaticMesh Asset without really having to know anything about UE4. This interface is provided by way of the Interactive Tools Framework. UMeshProcessingTool implements UInteractiveTool (by way of USingleSelectionTool), which is the base interface for Interactive Tools in the Framework. You can think of an Interactive Tool as a “mini-mode” in the Editor - when a Tool is active it is Ticked each frame, has the opportunity to do things like debug line drawing in it’s Render function, provides sets of editable properties that are shown in the Mode Panel, and can do things like handle mouse input or create 3D gizmos (we aren’t using those capabilities in this tutorial, though).

UMeshProcessingTool::Setup() creates an instance of an object called UMeshOpPreviewWithBackgroundCompute, which does most of the heavy lifting. This object (lets called it MOPWBC for short :P) implements a common UX pattern in which we want to edit a mesh based on a set of properties, and show a live preview, and we want the edit computation to run in a background thread so it doesn’t freeze the Editor. The “edit a mesh” operation is expressed as a FDynamicMeshOperator, and MeshProcessingTool.h defines a subclass FMeshProcessingOp that basically just owns and runs a lambda that edits the mesh. This lambda is provided by the ::MakeMeshProcessingFunction() we defined above. MOPWBC takes the FDynamicMeshOperator instance created by UMeshProcessingTool::MakeNewOperator(), gives it to a background thread to execute, and when the execution finishes, it uses the result mesh that was returned to update a UPreviewMesh instance that the MOPWBC also creates and manages (magic!).

UPreviewMesh is an independent utility object, that can be used to display the result of interactive mesh editing - it creates and manages a temporary Actor with a special mesh component (USimpleDynamicMeshComponent) that is faster to update than a normal UStaticMeshComponent. The input StaticMeshActor/Component are hidden while the Tool is active, and updated when you click the Accept button. If you click Cancel, they are never modified.

MeshProcessingPlugin Editor Mode, and Adding New Tools

The Mesh Processing Editor Mode is also provided by the Plugin. This is really getting into Unreal Editor dark arts, however what I have done here is basically a stripped down version of several built-in experimental Editor Modes - the ModelingToolsMode and SampleToolsMode Plugins (both located in /Engine/Plugins/Experimental/). The Modeling Tools Mode in particular has many more features than our basic mode (including icons!). In each case we have a subclass of FEdMode, which is how we get our own tab in the Editor Modes tab panel. You can create an FEdMode from scratch by selecting the Editor Mode type in the New Plugin dialog (like I said, dark arts).

The vast majority of the code at this level is boilerplate code you will won’t have any reason to modify. However if you want to add additional Tools - ie more buttons like “IGL Smooth” and “Export” in the toolbar - you will have to add 5 lines of code in specific places. Lets say your new tool is called UIGLSimplifyTool (perhaps based on libigl sample 703_Decimation) and you want the button to say “IGL Simplify”. Then you add the following lines:

1) In MeshProcessingPluginCommands.h, add a new command:

TSharedPtr<FUICommandInfo> BeginIGLSimplifyTool;

2) Configure that command in MeshProcessingPluginCommands.cpp:

UI_COMMAND(BeginIGLSimplifyTool, "IGLSimplify", "Start the LibIGL Simplify Tool", EUserInterfaceActionType::Button, FInputChord());

3) In MeshProcessingPluginEdMode.cpp, include your tool header and then in ::RegisterModeTools(), add a call to RegisterToolFunc() for your new Tool and Command:

#include "Tools/IGLSimlpifyTool.h"
...(snip)....
RegisterToolFunc(PluginCommands.BeginIGLSimplifyTool, TEXT("IGLSimplifyTool"), NewObject<UIGLSimplifyToolBuilder>());

4) in MeshProcessingPluginEdModeToolkit.cpp, in function ::BuildToolPalette(), add your new Command to the Toolbar.

ToolbarBuilder.AddToolBarButton(Commands.BeginIGLSimplifyTool);

That’s it! Done! New Tool appears in the Toolbar!

Modeling Mode in UE 4.24

I mentioned Modeling Mode several times above. This is another new Experimental feature of UE 4.24, which is built on the same Interactive Tools Framework that we used to create the IGLSmoothTool. It is much more extensive, and includes a variety of Mesh Editing and Repair Tools. By default this Mode Plugin is not enabled, and you have to open the Plugins browser and enable the Modeling Tools Editor Mode Plugin to turn it on. However in the IGLMeshProcessingProject I have already enabled it in the settings, so if you switch to the tab with the “Sphere-Cone-Box” icon, you will get a set of tabs and icons in the Mode Toolbar (Most Tools will be disabled unless you have a suitable object(s) selected):

View fullsize

Modeling Mode is relevant to this tutorial because there is a reasonable chance you might need to massage input data to experiment with mesh processing code. For example the FBX/OBJ importer frequently leaves “cracks” in objects along material boundaries. You can use the Inspector Tool to see if this is the case (it will highlight boundary edges in red), and if there are cracks, the Weld Edges Tool can probably repair them.

Another problem you might come across is that many game Assets have very coarse “low-poly” triangulations that are not suitable for most research geometry processing algorithms. The Remesh Tool can be used to add triangle density to these meshes without ruining the UV coordinates or hard normals. The image on the right shows the result of Remeshing the table from the standard UE scene.

The images below show the results of the IGLSmooth Tool on the default Table asset (left) and the remeshed triangulation (right). The remeshed version has better (ie more consistent and “smoother”) behavior because there is more freedom to move the vertices, and because numerically the “energy” that is minimized to compute the smoothed version is better-behaved numerically on more even triangulations. You can always use the Simplify Tool to get rid of these extra triangles after the processing. (PS: the Remesher in Unreal is incredibly powerful and you might find it quite useful to mix with libigl code! Check out RemeshMeshTool.cpp in the MeshModelingToolset plugin for sample code.)

View fullsize

Using Other Mesh Processing Code/Libraries

In this tutorial I focused on libigl because it is widely used, has a lot of neat sample code you can easily drop into an Unreal Editor Tool, and it’s header-only. Other header-only C++ libraries should be similarly-easy to use, the main “gotcha” is that Unreal configures the C++ compiler to consider all Warnings as Errors. As a result it may be necessary to disable some warnings to get your code to build. That’s what the file IGLIncludes.h does in the Mesh Processing Plugin, for example. (Of course you could also fix the warnings!)

If you want to use existing cpp files, things get a bit trickier. You have two options. One is to compile those separately into static libraries or DLLs and link to them. To do that you would add them in the MeshProcessingPlugin.Build.cs file. This is not trivial but there is a nice tutorial here (link) explaining how to do it for PCL, the Point Cloud Library. It’s from 2017 so the instructions should still apply (if you google you will likely find many older tutorials that are now out-of-date…).

The second option is to include your code (cpp and headers) directly within the /Private/ and/or /Public/ subdirectories of the Plugin Source folder. Then when you Regenerate Project Files (see step 3 above) your code will be automatically picked up by Unreal Build Tool (UBT) and included in the generated Visual Studio solution, and built into the Plugin’s DLL. This is in fact how many third-party libraries are included in Unreal Engine (there is actually a version of Eigen used in Unreal, it’s just quite out-of-date). Note however that there is not an easy way to exclude cpp files within the /Source/ subdirectory. This can trip up your attempts to just drop a full repository into the Plugin (for example Eigen’s repo includes a lot of cpp test and example code that UBT will try to build and link, unless those files are literally deleted.

Finally, UE4.24 by default configures the compiler to C++14. If your code requires C++17 or higher, it’s not hopeless. You can configure the Build.cs to enable this level of support for your plugin. The first post in this thread explains the lines to add. However that thread also describes some problems you might encounter. Good luck!

UE 4.26 Update

UE 4.26 introduced some changes to the Interactive Tools framework API, and the 4.24 tutorial code will no longer compile. I have ported it to 4.26 and made some updates. I kept the 4.24 version in the Github repo, and put the the 4.26 version in a separate path. If you would like to see the minimal changes necessary, see this commit. After that, I made further updates I will describe below. I also fixed an issue where the Eigen third-party code did not include any files or folders named ‘Core’, due to the .gitignore.

Another UE 4.26 addition is a “Base Tool” class named UBaseMeshProcessingTool. This was inspired by UMeshProcessingTool above, but with some useful additions. In the current 4.26 sample I have ported UMeshProcessingTool to use UBaseMeshProcessingTool. This intermediate might not be necessary anymore, but removing it meant adding quite a bit of boilerplate to the IGL tool, and rewriting some of the tutorial above, so I left it in. From the perspective of the IGL code, not much changed - the diff is here. The main difference is that I used the new PropertySet Watcher API to watch for changes in the properties, rather than a universal event handler.

One other important thing to know is that by default UBaseMeshProcessingTool will scale the input mesh to a unit box before calling any mesh processing code (and inverse-scale the corresponding live-preview and output meshes appropriately). This helps to “normalize” for the vast differences in scale that occur in a game environment, where we can easily have tiny rocks and enormous cliffs in the same scene. You can disable this behavior by overriding UBaseMeshProcessingTool::RequiresScaleNormalization(). Note that because of this normalization, I had to scale the 0-100 range of the Smoothness parameter presented in the UI (a standard complication in Laplacian mesh processing).

Finally two practical notes. First, make sure you install the Starter Content with the Engine, or you will have an empty level when you open the project. Second, UE has a file path limit when building. If you put the sample code in an already-long path, you might find that it won’t build, with errors about being unable to find files. If this is the case try moving the whole folder to a shorter path.