From CPU Frustum Culling to GPU-Driven Rendering

CreatorInDeep • 7/29/2026 5:47:25 AM

A year ago the first article on this site introduced Tsar's frustum culling. It ended with a promise: "a solid starting codebase that you can build upon"

We build on it, and then we deleted it.

Frustum::IsVisible(const AABB&) still exists in the engine today, but nothing in the mesh pipeline calls it. Every mesh in a Tsar scene is now culled by a compute shader, and the CPU never learns which objects survived. This article is the path between those two states - what forced each step, what got wrong , and the one number that says this was wroth doing.

1. Where we left off

The original design was the textbook one. Extract six planes from the view-projection matrix, test each object's world AABB against them with the positive-vertex trick, and skip the ones that fail:

bool Frustum::IsVisible(const AABB& worldAABB)
{
  for (const auto& plane : m_Planes)
  {
    glm::vec3 positiveVertex = worldAABB.Min;
    const glm::vec3 normal = glm::vec3(plane);

    if (normal.x >= 0) positiveVertex.x = worldAABB.Max.x;
    if (normal.y >= 0) positiveVertex.y = worldAABB.Max.y;
    if (normal.z >= 0) positiveVertex.z = worldAABB.Max.z;

    // The whole box is outside if even its furthest point is behind the plane.
    if (glm::dot(normal, positiveVertex) + plane.w < 0)
      return false;
  }
  return true;
}

Six dot products per object. That function was never the problem.

2. Why it ran out of road

The cost was never in IsVisible. It was in everything the CPU had to do around the answer.

Passing the test meant an object got pushed into a draw list. Each entry in that list became a vkCmdDrawIndexed with its own descriptor binds and push constants. So the CPU's per-frame work scaled with visible submeshes, not with the cull itself.

Worse, it scaled with views. A frame doesn't have one frustum — it has:

the camera
four shadow cascades
one frustum per shadow-casting spot light
six faces per shadow-casting point light

We had a separate CPU draw list for the camera and another for the cascades. And here is the part that is easy to miss until you look closely: point and spot light shadow passes were reusing the camera's visibility results. That is not a performance bug, it is a correctness bug — an object behind the camera can still cast a shadow into view from a light in front of it. We were culling shadow casters against the wrong frustum entirely.

The honest summary: the CPU wasn't slow at deciding what was visible. It was slow at being the only thing allowed to decide.

3. Step one: make the draws batchable first

You cannot move culling to the GPU while the CPU still decides draw calls. If every submesh needs its own bind, the GPU producing a visibility list doesn't help — someone still has to walk that list on the CPU and issue the calls.

So the first work wasn't culling at all. It was three changes that made a batch possible:

A shared mega vertex/index arena. All static mesh geometry lives in one vertex buffer and one index buffer. A submesh becomes an offset pair (FirstIndex, VertexOffset) instead of a buffer binding.
Bindless materials. Textures and material parameters move into one large descriptor set bound once per pass, indexed per instance.
A material index riding in the instance stream. Each instance's transform row carries the index of its material, so the vertex shader can look it up without a rebind.

One design decision worth calling out: bindless is a shader variant, not a replacement. Mesh-PBR-Static.TShader gets a BINDLESS variant used only by the opaque geometry pass. The default variant, the material editor, serialization, and thumbnail rendering are all untouched. Bindless is invasive enough without letting it leak into asset tooling.

With that in place, the opaque pass is a single vkCmdDrawIndexedIndirect — and the question becomes: who fills the indirect buffer?

4. Step two: the cull becomes a compute shader

A cull candidate is a submesh instance flatten into 48 bytes.

struct CullObject
{
  glm::vec4 BoundsMin;   // xyz = world AABB min
  glm::vec4 BoundsMax;   // xyz = world AABB max
  u32 IndexCount;
  u32 FirstIndex;      // global, into the shared index buffer
  u32 VertexOffset;     // global, into the shared vertex buffer
  u32 FirstInstance;    // slot into the unculled instance buffer
};

Bounds plus draw parameters, nothing else. The CPU builds this array once per frame, unculled, and uploads it. That is now the entire per-frame CPU cost of visibility.

MeshCull.TShader runs one thread per candidate. The frustum test is a deliberate line-for-line port of the CPU one — same plane order, same positive-vertex construction, same sign convention:

bool AabbVisible(vec3 mn, vec3 mx)
{
  for (int i = 0; i < 6; ++i)
  {
    vec3 n = cull.Planes[i].xyz;
    vec3 pv = vec3(n.x >= 0.0 ? mx.x : mn.x,
            n.y >= 0.0 ? mx.y : mn.y,
            n.z >= 0.0 ? mx.z : mn.z);
    if (dot(n, pv) + cull.Planes[i].w < 0.0)
      return false;
  }
  return true;
}

Keeping exact parity was worth the discipline. When something disappeared from the frame, we could rule out "the GPU test disagrees with the CPU test" immediately — and as section 7 shows, that mattered.

The output is a tightly packed VkDrawIndexedIndirectCommand per candidate:

uint base = idx * 5u;
draws.cmds[base + 0u] = obj.IndexCount;
draws.cmds[base + 1u] = visible ? 1u : 0u;   // instanceCount — 0 means culled
draws.cmds[base + 2u] = obj.FirstIndex;
draws.cmds[base + 3u] = obj.VertexOffset;
draws.cmds[base + 4u] = obj.FirstInstance;

Why mask instead of compact. The tidier approach is to compact survivors into a dense buffer and use vkCmdDrawIndexedIndirectCount so the GPU also supplies the draw count. We chose to issue a fixed drawCount and mask culled entries with instanceCount = 0 instead. A zero instance draw is retired almost free, and masking has no device-feature dependency, no drawIndirectCount, no atomic compaction pass, no readback. For the cost of a handful of dead commands we got a path that runs everywhere.

The render graph inserts the StorageWrite -> IndirectArgument barrier between the cull dispatch and the draw. That's the whole handoff - the CPU is not in this conversation.

5. Step three: one candidate set, many views

This is where the architecture pays for itself, and it's the part I'd most want a reader to take away.

The candidate buffer is view-agnostic — it is just "every submesh in the scene, with its world bounds." So a view becomes a small object: six planes, a view-projection matrix, and its own indirect draw buffer.

struct CullView
{
  ts::string Name;
  Ref<ShaderStorageBuffer> Draw;  // this view's indirect commands
  ts::vector<Ref<ComputeData>> Data;
  glm::vec4 Planes[6];
};

Culling for a new view is now: dispatch MeshCull over the shared candidates with that view's planes, write to that view's draw buffer, then draw it. The camera, each of the four shadow cascades, and each shadow casting spot light all became one dispatch and one indirect draw each and, critically, each is culled against its own frustum, which fixed the correctness bug from section 2.

Point lights needed one extra idea. They're omnidirectional, so no single frustum applies. The shader gained a sphere test and a switch to disable the plane test:

bool SphereVisible(vec3 mn, vec3 mx)
{
  vec3 c = cull.LightSphere.xyz;
  vec3 closest = clamp(c, mn, mx);
  vec3 d = closest - c;
  return dot(d, d) <= cull.LightSphere.w * cull.LightSphere.w;
}

A point light culls by range, once per light, and the existing multiview shadow pass replays that single indirect draw across all six cube faces. Six per-face frusta would be tighter; one sphere test is a fraction of the cost and fixes the actual bug. We took the cheap correct thing.

6. Step four: deleting the CPU cull

Once every pass drew from a GPU-produced buffer, the CPU-side machinery was dead weight that still ran every frame. Removing it deleted, among others:

m_DrawList and m_CascadedShadowMapDrawList
the per-frame IsVisible loop in BeginFrame
the CSM cull pre-pass in UpdateLighting
the per-key transform map and its duplicated instance buffer
five now-unreachable renderer draw methods

BeginFrame now does exactly one thing for visibility: build the candidate array.

7. The bug that ate a day: correct culling, stale data

Objects started vanishing. Specifically: objects scaled at runtime from C# or by physics would render fine, then pop out of existence as the camera turned toward them.

The obvious suspect is the new code. A GPU cull, freshly written, dropping scaled objects clearly the shader mishandles scale. Maybe it's treating the AABB as a bounding sphere and not scaling the radius by the maximum axis. That's the classic version of this bug.

The shader was innocent. It consumes a world-space AABB that the CPU builds by transforming all eight corners of the local box through the full T*R*S matrix and refitting. Scale and rotation are already baked in before the GPU ever sees it.

The real defect was one layer up. MeshComponent::BoundingBox was only recomputed when the entt on_update<TransformComponent> signal fired. The editor gizmo fired it. The inspector fields fired it. The runtime setters SetScale, SetTranslation, SetRotation and their local variants did not. They mutated the transform, propagated it down the hierarchy, and never touched the bounds.

So a scripted object scaled 10× kept its original scale-1 AABB sitting near the origin. The GPU then correctly culled a box that genuinely was outside the frustum. The cull was right. The data was a lie.

The fix was to refresh bounds in the two transform-propagation chokepoints in Scene, for the moved entity and every traversed child so any script, physics, inspector, or gizmo move updates bounds exactly once, on change, rather than every frame.

The lesson is the one worth publishing: when you move a decision to the GPU, you also move your debugging to the GPU, and the instinct is to distrust the new code. Exact CPU/GPU parity in the visibility test (section 4) is what let us skip that entire dead end if both implementations agree, the disagreement is upstream. Build the parity even when you're deleting the CPU version; it's a debugging tool, not just a migration aid.

8. The part most articles leave out

GPU-driven rendering is fixed per-frame overhead that only pays off at scale.

You add a candidate build, a compute dispatch per view, and a compute -> graphics barrier every frame, regardless of scene complexity. On a scene with a handful of objects, that overhead is larger than the draw calls it eliminates. We measured this the embarrassing way: a test scene that ran comfortably before got measurably worse the day GPU culling landed.

That is not a bug and it is not a reason to back out. It's the shape of the trade. Batched indirect rendering wins when you have thousands of submeshes and a dozen views, which is exactly where the CPU path was collapsing.

The same lesson applied harder to occlusion culling. We implemented a two-phase Hi-Z occlusion pass cull against last frame's depth pyramid, draw the survivors, rebuild the pyramid from this frame's depth, then re-test and recover anything wrongly dropped. Getting it correct took several rounds, including two genuine render-graph synchronisation bugs worth knowing about:

SampledTexture reads were declared as fragment-stage only, but a compute pass that samples a texture needs the barrier to cover the compute stage too.
Storage reads carried SHADER_STORAGE_READ but not SHADER_SAMPLED_READ, so a compute pass sampling a sampler2D written by a previous compute pass read stale data through an uninvalidated texture cache.

It is correct now, and it ships off by default because on our current scenes it costs two pyramid builds and two cull dispatches to occlude nothing. It waits for content dense enough to earn it. Shipping a working feature disabled is a legitimate outcome.

9. Where this goes next

A single-pass depth downsampler (SPD) to make the Hi-Z pyramid one dispatch instead of eleven, so occlusion becomes cheap enough to default on.
A sorted transparent path - everything here is opaque-only.
Per-face point light culling, once there's a scene where six frusta beat one sphere.

The frustum test in this engine hasn't changed since that first article. It's the same six dot products and the same positive vertex. Everything around it move

Resources: