Precomputed Visibility Masks: Hacking the Pipeline to Skip Overdraw

Introduction: The Overdraw Problem and Why Traditional Solutions Fall Short

Overdraw remains one of the most persistent performance bottlenecks in real-time rendering. Every time a fragment is shaded but later discarded because another fragment occludes it, GPU cycles are wasted. Traditional solutions like the depth prepass (Z-prepass) help by decoupling depth testing from shading, but they still shade fragments that are ultimately invisible. Early-Z rejection on modern GPUs can skip shading for fragments that fail the depth test, but it is often ineffective for transparent objects or when shaders modify depth. Tile-based deferred rendering (TBDR) reduces overdraw by shading only visible fragments per tile, but it introduces memory bandwidth costs for G-buffers and struggles with complex lighting. Precomputed visibility masks (PVMs) offer a different approach: by encoding visibility information offline, the GPU can skip entire objects or triangles before rasterization, effectively eliminating overdraw at the source. This guide is written for experienced rendering engineers who already understand the basics of culling and occlusion. We will assume familiarity with GPU pipelines, shader stages, and common rendering techniques. Our goal is to provide a deep, practical understanding of PVMs—their mechanisms, trade-offs, and implementation strategies—so you can decide whether they fit your project. This overview reflects widely shared professional practices as of May 2026; verify critical details against current driver documentation where applicable.

Core Mechanism: How Precomputed Visibility Masks Work

At its core, a precomputed visibility mask is a per-object or per-triangle bitmask that indicates which screen-space tiles or pixels the object is visible in, from a given viewpoint. These masks are generated offline during a preprocessing step, where the scene is rendered from many sampled camera positions and the visibility of each object is recorded. At runtime, instead of sending all objects to the GPU, the application queries the mask for the current camera pose: if the mask indicates the object is not visible in any tile, the object is culled entirely. This skips all GPU work for that object, including vertex shading and rasterization. The mask itself is typically a compact bitfield—one bit per tile—compressed using run-length encoding or hierarchical bitmaps to minimize memory footprint. For example, a 64×64 tile grid yields 4096 bits (512 bytes) per object. With tens of thousands of objects, memory can become a concern, so compression is essential. The key insight is that visibility is often sparse: an object may be visible in only a few tiles, so compression can be very effective. PVMs differ from traditional occlusion culling (like hardware occlusion queries) in that they are precomputed and deterministic, not runtime-dependent. This means there is no CPU overhead for issuing queries or waiting for results. However, it also means the masks are static: they cannot account for dynamic objects or deformable geometry without regeneration. The preprocessing step can be integrated into the asset pipeline, similar to lightmap baking. For static scenes, PVMs can achieve near-perfect culling, eliminating overdraw almost entirely.

Mask Generation Pipeline

Generating PVMs involves three stages: sampling, rasterization, and encoding. First, the scene is sampled from a set of representative camera positions. For a first-person game, these positions might be points on a grid at typical player height, with rotations covering the most common view directions. The number of samples directly affects mask accuracy: too few samples cause aliasing (objects that should be visible are culled), too many increase preprocessing time. A common heuristic is to use one sample per square meter for indoor scenes and one per 4 square meters for outdoor scenes. Second, for each sample, the scene is rendered using conservative rasterization—a technique where any triangle that touches a pixel is considered to cover that pixel, even if the coverage is subpixel. This avoids missing small objects. The visibility of each object is recorded per tile: a tile is marked visible if any pixel of the object is present. Third, the per-sample masks are aggregated into a single mask per object. This can be a union (object visible if visible in any sample) or an intersection (visible only if visible in all samples). Union yields conservative culling (no false negatives but more false positives), while intersection is aggressive (may cull objects that are actually visible). Most pipelines use a union with a small bias to handle edge cases.

Comparison of Overdraw Reduction Techniques

To understand where PVMs fit, we compare them against four common alternatives: depth prepass (Z-prepass), early-Z rejection, tile-based deferred rendering (TBDR), and hardware occlusion queries. Each technique has distinct trade-offs in GPU cost, memory footprint, scene complexity suitability, and ease of integration.

PVMs excel in scenarios where the scene is mostly static and contains many small, overlapping objects—like vegetation, debris, or architectural details. In such environments, overdraw can be extreme, and traditional techniques still shade many invisible fragments. PVMs reduce GPU work to nearly the theoretical minimum: only objects that are actually visible are processed. However, they are not a silver bullet. The preprocessing cost is non-trivial, and dynamic objects cannot benefit. For dynamic scenes, a hybrid approach is common: use PVMs for static geometry and fall back to depth prepass or early-Z for dynamic objects. Another limitation is that PVMs do not handle moving camera positions well if the mask resolution is too coarse. Fine-grained masks (e.g., 128×128 tiles) improve accuracy but increase memory. The table above summarizes the key trade-offs; choose based on your scene’s static/dynamic ratio and overdraw severity.

Step-by-Step Guide to Implementing Precomputed Visibility Masks

Implementing PVMs in a real pipeline requires careful planning. Below is a detailed step-by-step guide that assumes you have a working renderer and asset build system. The steps cover mask generation, compression, runtime lookup, and shader integration.

Step 1: Define Tile Grid and Sampling Strategy

First, choose the tile resolution. A 64×64 grid is a good starting point for 1080p (each tile covers about 30×30 pixels). For higher resolutions, consider 128×128. The sampling strategy depends on your game’s camera movement. For a corridor shooter, sample along the center of each corridor at 1-meter intervals, with rotations covering ±30 degrees horizontally and ±15 degrees vertically. For an open-world game, use a hierarchical grid: coarse samples at 10-meter intervals, then refine around points of interest. The total number of samples should be kept under 100,000 for reasonable preprocessing times (a few hours for a large scene).

Step 2: Generate Masks with Conservative Rasterization

For each sample, render the scene using conservative rasterization. Modern GPUs support this via the `GL_NV_conservative_raster` extension or Vulkan’s `VK_EXT_conservative_rasterization`. If hardware support is unavailable, you can emulate it by rendering slightly larger triangles (e.g., scale triangles by 1.01 in object space). For each object, accumulate a bitmask: one bit per tile. Use a compute shader or CPU-based rasterization for flexibility. The output is a raw mask per object per sample.

Step 3: Aggregate and Compress Masks

Combine per-sample masks into a single mask per object using a union operation. Then compress the mask. Run-length encoding (RLE) works well for sparse masks: store runs of consecutive zeros or ones. Alternatively, use a hierarchical bitmap (like a quadtree) where each node stores whether any child is visible. This allows fast lookup: for a given tile, traverse the tree to determine visibility. Compression ratios of 10:1 to 50:1 are common. Store the compressed mask as a binary blob in the object’s metadata.

Step 4: Integrate Runtime Lookup

At runtime, for each frame, compute the current camera’s tile grid coordinates. For each object, decompress the mask (or use the hierarchical representation) to check visibility. If the object is invisible in all tiles, skip it entirely. Otherwise, pass it to the renderer. This lookup must be fast; ideally, it runs in a compute shader that culls objects in parallel. For CPU culling, use SIMD instructions to process multiple objects at once. The overhead should be less than 0.1 ms per frame for 10,000 objects.

Step 5: Handle Dynamic Objects and Transitions

Dynamic objects cannot use PVMs. Instead, render them with a standard depth prepass. To avoid popping when an object transitions from static to dynamic (e.g., a static crate that gets destroyed), invalidate its mask and regenerate it in the background. For partially dynamic scenes, consider a two-tier system: static objects use PVMs, dynamic objects use a separate list culled by frustum and occlusion queries. This hybrid approach maintains high culling efficiency while supporting interactivity.

Real-World Scenario 1: Dense Forest with Thousands of Vegetation Objects

Consider a forest scene with 20,000 unique tree and bush meshes, each consisting of 500-2000 triangles. Without optimization, overdraw can exceed 10x: many trees are behind others but still rasterized and shaded. A depth prepass reduces shading cost but still processes all vertices. PVMs can cull up to 80% of objects per frame, depending on viewpoint. In testing, a team reported that with a 64×64 tile grid and union aggregation, the number of visible objects dropped from 20,000 to an average of 4,000 per frame. Vertex processing was reduced by 80%, and fragment shader invocations by 90%. The memory for compressed masks was about 2 MB per 10,000 objects (with RLE compression), negligible compared to texture memory. The preprocessing took 3 hours on a 32-core workstation. One challenge was handling thin branches: conservative rasterization caused some false positives (tiles marked visible even though only a few pixels were covered), but the union aggregation minimized culling errors. The team also implemented a two-level mask: a coarse 16×16 grid for quick rejection, then a fine 64×64 grid for detailed culling. This reduced lookup time by 40%. The forest scene ran at 90 FPS on a mid-range GPU, compared to 45 FPS with only depth prepass.

Real-World Scenario 2: High-Poly Character Close-Up in a Crowd Scene

In a crowd simulation with 500 unique characters, each with 50,000 triangles, overdraw is less of an issue because characters are often spaced apart. However, when the camera zooms in on one character, the others are still fully processed. PVMs can cull characters that are off-screen or occluded by the main character. For this scenario, the team used a per-character mask with a 128×128 tile grid to capture fine occlusion. The results: for a close-up shot, only 20 characters were visible, down from 500. Vertex processing was reduced by 96%, and fill rate was nearly halved. The memory cost was higher: each character’s mask was 2 KB uncompressed (128×128 bits = 2048 bytes), compressed to about 200 bytes with RLE. Total memory for 500 characters was 100 KB, acceptable. The preprocessing sampled camera positions in a sphere around each character at 2-meter intervals. One issue was that characters with similar poses had similar masks, so the team deduplicated masks to save memory. This scenario highlights that PVMs are not just for small objects; they also work for large, detailed objects when the camera is close.

Common Pitfalls and How to Avoid Them

Implementing PVMs comes with several pitfalls that can degrade performance or visual quality. The most common is mask aliasing: when the sampling density is too low, an object that should be visible may be culled. This appears as popping—objects suddenly appear as the camera moves. To avoid this, use conservative rasterization and union aggregation with a small bias (e.g., mark a tile visible if any sample within a small radius shows the object). Another pitfall is memory bloat. Without compression, masks for a large scene can exceed available memory. Compress aggressively using RLE or hierarchical bitmaps, and consider storing masks in a GPU buffer for fast lookup. A third pitfall is dynamic object interaction. If a dynamic object moves behind a static object that was culled, the dynamic object might be rendered incorrectly because the static object’s mask doesn’t account for the dynamic object. The solution is to always render dynamic objects with a depth prepass, and for static objects, use the mask only as a culling hint, not a hard rule. Finally, preprocessing time can be prohibitive for large scenes. Optimize by using multiple machines or sampling only from likely camera positions (e.g., based on player path data). If preprocessing takes more than 24 hours, consider reducing sample count or using a coarser grid.

When NOT to Use Precomputed Visibility Masks

PVMs are not a universal solution. Avoid them in scenarios with highly dynamic geometry (e.g., destructible environments, moving characters). The cost of regenerating masks for dynamic objects outweighs the benefits. Also, avoid them if your scene has very few overlapping objects—the overhead of mask lookup may exceed the savings. For scenes with large, simple objects (e.g., a single building), a depth prepass is more efficient. Another case is when camera movement is unpredictable and covers many angles; the mask may become too conservative (culling nothing) or too aggressive (culling visible objects). In such cases, hardware occlusion queries may be a better fit. Finally, if your target hardware is very low-end (e.g., mobile devices with limited memory bandwidth), the mask lookup overhead might be significant. Profile carefully: measure the time to decompress and check masks vs. the time saved by not processing culled objects. If the ratio is less than 2:1, PVMs may not be worthwhile.

Integration with Existing Pipelines: Hybrid Approaches

Most teams integrate PVMs as part of a hybrid culling system. The typical architecture has three tiers: frustum culling (fastest, always applied), PVM culling (for static objects), and hardware occlusion queries (for dynamic objects or as a fallback). The PVM culling step runs after frustum culling: for each object that passes frustum culling, check its mask. If invisible, skip. If visible, add to the render list. Then, for dynamic objects, issue hardware occlusion queries against the already-rendered static geometry. This hybrid approach maximizes culling efficiency without sacrificing dynamic object support. Another integration point is in the shader: if an object is partially visible (mask indicates some tiles visible, some not), you can use the mask to discard fragments in invisible tiles early in the shader. This requires passing the tile coordinates to the shader and computing the mask bit. However, this adds ALU cost and is only beneficial if the object spans many tiles. A simpler approach is to accept that some overdraw remains for partially visible objects. Many teams find that the hybrid approach reduces total GPU load by 30-50% in static-heavy scenes, with minimal integration effort.

Future Directions and Emerging Techniques

PVMs are an active area of research. One emerging direction is dynamic mask generation using neural networks: instead of precomputing masks, a lightweight network predicts visibility per object in real-time. Early experiments show promising results, but the overhead of running the network may offset gains. Another direction is adaptive mask resolution: use a coarse grid for objects far from the camera and a fine grid for nearby objects. This reduces memory while maintaining accuracy. Some engines are exploring combined PVMs with virtual texturing: the mask doubles as a page table for texture streaming, ensuring that only visible textures are loaded. Finally, hardware vendors are adding support for conservative rasterization and hierarchical culling in fixed-function units, which could make PVMs a standard feature in future APIs. As of May 2026, these techniques are still experimental; most production pipelines rely on the static approach described in this guide. We recommend prototyping PVMs in a side branch and measuring gains before committing to a full integration.

Frequently Asked Questions

How much memory do PVMs require?

Memory depends on tile resolution and compression. For a 64×64 grid (4096 bits = 512 bytes uncompressed), RLE compression typically reduces to 50-100 bytes per object. For 100,000 objects, that’s 5-10 MB. Fine grids (128×128) increase to 20-40 MB. This is acceptable for most PC and console games.

Can PVMs be used for transparent objects?

Yes, but with care. Transparent objects often require sorting, and PVMs only handle visibility, not order. You can use masks to cull transparent objects that are completely occluded by opaque geometry, but you must still sort visible transparent objects. The mask is still useful for reducing the number of transparent objects processed.

What about skinned meshes or animated objects?

For skinned meshes, PVMs are not directly applicable because the shape changes per frame. However, you can precompute masks for the binding pose and use them as a conservative estimate (the animated mesh will never be smaller than the binding pose, so if the binding pose is invisible, the animated version is also invisible). This works for characters that don’t deform drastically.

How do I validate that PVMs are not causing visual errors?

Render a debug view that highlights culled objects. Walk through the scene and ensure no objects pop in or out. Also, compare frame-by-frame with and without PVMs to verify identical output. Automated regression tests with known camera paths can catch errors.

Conclusion

Precomputed visibility masks offer a powerful way to eliminate overdraw in static scenes by shifting culling decisions to a preprocessing step. They are not a replacement for all other techniques but rather a specialized tool that excels in specific scenarios—dense vegetation, crowded environments, and any scene with many overlapping static objects. The implementation requires careful planning: tile resolution, sampling density, compression, and hybrid integration with dynamic object support. When done right, PVMs can reduce GPU workload by 50-80% in the right conditions, freeing up resources for higher-quality shading or more complex scenes. We encourage you to prototype PVMs in your pipeline and measure the gains. Start with a simple scene, benchmark, and iterate. Remember that PVMs are a static optimization; they complement, not replace, dynamic culling techniques. With the guidance in this article, you should be able to make an informed decision and implement a robust PVM system. The future of rendering is increasingly about doing less work, not more, and PVMs are a step in that direction.