Seeing Through Things:
Alpha Cutouts, Glass, and Deferred Rendering
Up until now, every single object in RoboEngine’s world was 100% solid. Great for marble vases and animated foxes, not so great if you want leafy trees, chain-link fences, stained glass, or wispy smoke.
After landing our massive Engine/Editor split, I pushed to teach RoboEngine’s materials how to see through things. Now, foliage can have crisp holes (that even show up in cast shadows!), glass panes can catch PBR reflections while revealing what sits behind them, and everything fits cleanly into both our Forward and Deferred renderers.
The trickiest puzzle? Fitting transparency into a Deferred pipeline, which by design can only store one surface per screen pixel.
Three Alpha Modes
(Straight From glTF)
MaterialComponent gained glTF 2.0’s three-tier alpha model: adding an AlphaMode, an opacity multiplier, and an alphaCutoff threshold:
// How a material's alpha (opacity × Base Color map's alpha) is used; matches glTF's alphaMode
enum class AlphaMode
{
Opaque, // Alpha is ignored completely
Mask, // Cut-out: pixels below alphaCutoff are discarded; the rest are solid (leaves, fences)
Blend, // See-through: alpha-blended over what's behind, drawn after all opaque meshes (glass, smoke)
};
AlphaMode alphaMode = AlphaMode::Opaque;
float opacity = 1.0f; // Mask / Blend: multiplied with the Base Color map's alpha
float alphaCutoff = 0.5f; // Mask: pixels with lower alpha get clipped out
A pixel’s final alpha is always opacity × baseColorMap.a. All three fields save to .scene.json files, show up as live controls in the Inspector, and import directly from glTF files (where base_color_factor[3] now maps cleanly to opacity instead of being ignored):
switch (source->alpha_mode)
{
case cgltf_alpha_mode_mask: material.alphaMode = AlphaMode::Mask; break;
case cgltf_alpha_mode_blend: material.alphaMode = AlphaMode::Blend; break;
default: material.alphaMode = AlphaMode::Opaque; break;
}
material.alphaCutoff = source->alpha_cutoff;
Each mode flows through the frame’s render passes differently:
| Alpha Mode | Opaque / G-Buffer Pass | Shadow Pass | Post-Opaque Pass |
Opaque | Drawn normally | Fast solid depth | — |
Mask | Drawn; low-alpha pixels discarded | Cutout depth (preserves holes!) | — |
Blend | Skipped entirely | Solid depth | Forward-lit, alpha-blended, back-to-front |
Alpha Cutouts (Mask):
Sharp Holes & Leafy Shadows
On Screen: Zero Extra Pipelines
Mask is the easy win: a pixel is either 100% solid or completely gone, meaning it needs no depth sorting and no alpha blending. Both our forward shader (mesh.frag) and G-buffer shader (gbuffer.frag) simply call one helper at the top of main():
// opacity × Base Color map's alpha
float sampleAlpha(vec2 uv)
{
return meshOpacity() * texture(baseColorMap, uv).a;
}
// Mask materials: drop pixels below the cutoff (cutoff is 0.0 for Opaque, so nothing discards)
void alphaCutout(vec2 uv)
{
if (sampleAlpha(uv) < meshAlphaCutoff()) { discard; }
}
Because Opaque materials pass a cutoff of 0.0, both modes share the exact same graphics pipeline on screen.
Smuggling Alpha Into Push Constants
Our mesh push constants were already packed tight with a model matrix and a combined normal-plus-material matrix. Rather than expanding the push constant block, we smuggled both the opacity and alphaCutoff into the unused .w slots of the model matrix’s bottom row (which is always (0, 0, 0, 1) for standard affine transforms):
// Alpha packed into the model matrix's bottom row (always 0, 0, 0 for affine transforms)
const bool usesAlpha = material.alphaMode != AlphaMode::Opaque;
push.modelMatrix[0].w = usesAlpha ? glm::clamp(material.opacity, 0.0f, 1.0f) : 1.0f;
push.modelMatrix[1].w = material.alphaMode == AlphaMode::Mask ? glm::clamp(material.alphaCutoff, 0.0f, 1.0f) : 0.0f;
Our GLSL shaders unpack those floats and scrub the matrix back to clean (0, 0, 0, 1) inside meshModelMatrix().
In Shadows: Punching Holes in Light
Our standard shadow pass only reads vertex positions which would turn a leafy branch into a solid rectangular shadow. To fix that, each mesh system gained a dedicated cutout shadow pipeline that reads UVs, binds the base color texture, and runs a tiny shadow_cutout.frag shader:
// shadow_cutout.frag
void main()
{
if (push.alpha.x * texture(baseColorMap, fragUV).a < push.alpha.y) { discard; }
}
During renderShadowDepth(), RoboEngine blasts through all solid shadow casters first using the ultra-fast position-only pipeline. Only if it spots a Mask material along the way does it bind the cutout pipeline for a second pass, and because skinned meshes got skeletal_mesh_shadow_cutout.vert, an animated character’s tattered cloak or hair casts accurate holes too!
See-Through Surfaces (Blend):
Hybrid Forward Rendering
Why Deferred Can’t Hold Glass
A G-buffer stores a single depth and material per pixel, as the updated ReSceneRenderer header puts it, it “can’t hold glass and what’s behind it.”
The classic solution is a hybrid pass: Blend materials skip the G-buffer completely. Once the opaque scene finishes (whether via Forward or Deferred lighting), a dedicated renderTransparentMeshes() pass draws all translucent surfaces forward-style right on top of the lit image:
// Deferred path
deferredLightingSystem->render(frameInfo, gbuffer->getDescriptorSet());
renderTransparentMeshes(frameInfo, {staticMeshSystem.get(), skeletalMeshSystem.get()}); // Forward-lit over the result
lightBillboardSystem->render(frameInfo);
// Forward path
skeletalMeshSystem->render(frameInfo);
renderTransparentMeshes(frameInfo, {staticMeshSystem.get(), skeletalMeshSystem.get()}); // Back-to-front
lightBillboardSystem->render(frameInfo);
Sorting Across Static and Skinned Meshes
Alpha blending is strictly order-dependent: a translucent window in front of a ghostly skinned character has to draw after the character. To sort across both static and skeletal mesh systems in one unified list, both implement a TransparentMeshDrawer interface:
class TransparentMeshDrawer
{
public:
virtual void collectTransparent(const FrameInfo &frameInfo, std::vector<TransparentDraw> &out) = 0;
virtual void beginTransparent(FrameInfo &frameInfo) = 0; // Bind pipeline + frame descriptor sets
virtual void drawTransparent(FrameInfo &frameInfo, const TransparentDraw &draw) = 0;
};
renderTransparentMeshes() collects every Blend draw across all drawers, sorts them farthest-first using the squared distance from the camera to each mesh’s bounding-box center, and swaps pipelines only when the drawer type changes:
std::stable_sort(draws.begin(), draws.end(),
[](const TransparentDraw &a, const TransparentDraw &b) { return a.distanceSq > b.distanceSq; });
TransparentMeshDrawer *bound = nullptr;
for (const TransparentDraw &draw : draws)
{
if (draw.drawer != bound) { draw.drawer->beginTransparent(frameInfo); bound = draw.drawer; }
draw.drawer->drawTransparent(frameInfo, draw);
}
Each mesh system compiles a third lit pipeline with alpha blending enabled, depth testing on, and depth writes off (so opaque walls occlude glass, while panes of glass don’t occlude each other). Inside mesh_transparent.frag, it calls the exact same shadeSurface() function as every opaque object—meaning glass still catches PBR specular highlights, HDR environment reflections, and cast shadows!
Traps & Mobs Along the Way
Blend materials still cast solid shadows.
- Right now, the shadow pass skips
Maskmaterials in its solid loop, butBlendmeshes still run through the solid depth pass, meaning a clear glass window casts a shadow as dark as a brick wall. Passable for heavily tinted glass, though true colored/attenuated translucent shadows are a future quest.
Sorting happens per mesh, not per triangle.
- Sorting by bounding-box center works well for separate panes of glass, but intersecting transparent meshes, or a concave translucent object, can still exhibit order-dependent artifacts from certain camera angles.
Depth writes off means billboards draw on top.
- Because transparent surfaces don’t write to the depth buffer, anything drawn afterward (like editor light billboards) renders cleanly over them rather than getting clipped.
Always read the model matrix via meshModelMatrix().
- With opacity and cutoff riding in
.wof the first two columns, any shader reading rawpush.modelMatrixdirectly will get garbage transform math.
Descriptor sets shift in skinned cutout shadows.
- Because
skeletal_mesh_shadow_cutout.vertneedsset 0for the material’sbaseColorMaptexture, the skinning storage buffer slides over toset 1(SKINNING_SET 1). Any future shader combining skinning and material textures has to follow the same convention.
Zero overhead when unused.
- The solid shadow pass tracks an
anyCutoutflag; if a scene has zeroMaskmaterials, the second cutout shadow loop and pipeline switch are skipped completely.
Next Up on the Quest
With transparency unlocked, RoboEngine can now render just about anything a glTF 2.0 file throws at it! Next, we’re shifting gears from graphics programming to repository architecture: in my next DevBlog, I’ll tackle extracting the game content out into its own standalone repository and turning RoboEngine into a reusable Git submodule ready to power real game projects!
