Update Grid Rendering Pipeline

John McCardle 2025-12-02 03:10:19 +00:00
parent fe7b626c14
commit 0816bb4b98
1 changed files with 373 additions and 373 deletions

@ -1,374 +1,374 @@
# Grid Rendering Pipeline
## Overview
The Grid rendering pipeline handles multi-layer tilemap rendering with chunk-based caching for optimal performance. It supports arbitrary numbers of rendering layers, viewport culling, zoom, and entity rendering.
**Parent Page:** [[Grid-System]]
**Related Pages:**
- [[Performance-and-Profiling]] - Metrics and profiling tools
- [[Entity-Management]] - Entity rendering within grids
- [[Grid-TCOD-Integration]] - FOV and pathfinding (partially in transition)
**Key Files:**
- `src/UIGrid.cpp::render()` - Main rendering orchestration
- `src/GridLayers.cpp` - ColorLayer and TileLayer rendering
- `src/UIGrid.cpp::renderChunk()` - Per-chunk RenderTexture management
**Related Issues:**
- [#123](../../issues/123) - Chunk-based Grid Rendering (Closed - Implemented)
- [#148](../../issues/148) - Dirty Flag RenderTexture Caching (Closed - Implemented)
- [#147](../../issues/147) - Dynamic Layer System (Closed - Implemented)
- [#113](../../issues/113) - Batch Operations API (Open - includes FOV access discussion)
- [#115](../../issues/115) - SpatialHash for entity culling (Open)
---
## Architecture Overview
### Layer-Based Rendering
Grids render content through **layers** rather than per-cell properties. Each layer is either a `ColorLayer` (solid colors) or `TileLayer` (texture sprites).
**Render order is determined by z_index:**
- Layers with `z_index < 0` render **below** entities
- Entities render at the z=0 boundary
- Layers with `z_index >= 0` render **above** entities (overlays)
Within each group, lower z_index values render first (behind higher values).
```
z_index: -3 -2 -1 0 +1 +2
↓ ↓ ↓ ↓ ↓ ↓
[background] [tiles] [ENTITIES] [fog] [UI overlay]
```
**Implementation:** See `UIGrid::render()` which iterates layers in z_index order, inserting entity rendering at the 0 boundary.
### Chunk-Based Caching
Large grids are divided into **chunks** (regions of cells). Each chunk maintains its own `sf::RenderTexture` that caches the rendered result.
**Key concepts:**
- Chunk size is implementation-defined (currently ~256 cells per dimension)
- Only chunks intersecting the viewport are considered for rendering
- Each chunk tracks whether its content is "dirty" (needs redraw)
- Static content renders once, then the cached texture is reused
**Implementation:** See `UIGrid::renderChunk()` for chunk texture management.
### Dirty Flag Propagation
When layer content changes, only affected chunks are marked dirty:
1. `layer.set(x, y, value)` marks the containing chunk as dirty
2. On next render, dirty chunks redraw to their RenderTexture
3. Clean chunks simply blit their cached texture
This means a 1000x1000 grid with one changing cell redraws only ~1 chunk, not 1,000,000 cells.
**Implementation:** Dirty flags propagate through `UIGrid::markDirty()` and are checked in `UIGrid::render()`.
---
## Render Pipeline Stages
### Stage 1: Viewport Calculation
Calculate which chunks and cells are visible based on camera position, zoom, and grid dimensions.
**Key properties:**
- `center` / `center_x`, `center_y` - Camera position in pixel coordinates (within grid space)
- `zoom` - Scale factor (1.0 = normal, 2.0 = 2x magnification)
- `size` - Viewport dimensions in screen pixels
**Implementation:** Viewport bounds calculated at start of `UIGrid::render()`.
### Stage 2: Below-Entity Layers
For each layer with `z_index < 0`, sorted by z_index:
1. Determine which chunks intersect viewport
2. For each visible chunk:
- If dirty: redraw layer content to chunk's RenderTexture
- Draw chunk's cached texture to output
### Stage 3: Entity Rendering
Entities render at the z=0 boundary:
1. Iterate entity collection
2. Cull entities outside viewport bounds
3. Draw visible entity sprites at interpolated positions
**Note:** Entity culling is currently O(n). SpatialHash optimization planned in [#115](../../issues/115).
### Stage 4: Above-Entity Layers
For each layer with `z_index >= 0`, sorted by z_index:
- Same chunk-based rendering as Stage 2
- These layers appear as overlays (fog, highlights, UI elements)
### Stage 5: Final Compositing
All rendered content exists in the grid's output RenderTexture, which is drawn to the window in a single operation.
---
## Performance Characteristics
**Static Grids:**
- Near-zero CPU cost after initial render
- Cached chunk textures reused frame-to-frame
- Only viewport calculation and texture blitting
**Dynamic Grids:**
- Cost proportional to number of dirty chunks
- Single-cell changes affect only one chunk
- Bulk operations should batch changes before render
**Large Grids:**
- 1000x1000+ grids render efficiently
- Only visible chunks processed
- Memory scales with grid size (chunk textures)
**Profiling:** Use `mcrfpy.getMetrics()` or the F3 overlay to see render times. See [[Performance-and-Profiling]].
---
## FOV and Perspective System (In Transition)
**Current Status:** The FOV computation (`compute_fov()`, `is_in_fov()`) works correctly for pathfinding and visibility queries. However, the automatic fog-of-war overlay system is **not currently connected** to the new layer architecture.
**What works:**
- `grid.compute_fov(x, y, radius)` - Computes which cells are visible
- `grid.is_in_fov(x, y)` - Queries visibility of a specific cell
- Pathfinding uses walkable/transparent properties correctly
**What's in transition:**
- Per-entity perspective (`UIGridPointState`) not yet exposed to Python
- Automatic fog overlay rendering disconnected from layer system
- Batch FOV data access being designed ([#113 discussion](../../issues/113))
**Current workaround - Manual fog layer:**
```python
# Create a fog overlay layer (positive z_index = above entities)
fog_layer = grid.add_layer("color", z_index=1)
# After computing FOV, update fog colors
grid.compute_fov(player.grid_x, player.grid_y, radius=10)
for x in range(grid.grid_x):
for y in range(grid.grid_y):
if not grid.is_in_fov(x, y):
# Dim color for non-visible cells
fog_layer.set(x, y, mcrfpy.Color(0, 0, 0, 192))
else:
# Transparent for visible cells
fog_layer.set(x, y, mcrfpy.Color(0, 0, 0, 0))
```
**Limitation:** This O(n²) iteration is inefficient for large grids. Batch operations ([#113](../../issues/113)) will address this with patterns like `cells_in_radius()` iterators.
---
## Layer Techniques
### Multiple Overlay Layers
You can pre-create multiple overlay layers and toggle between them:
```python
# Create several overlay options
highlight_layer = grid.add_layer("color", z_index=1)
danger_layer = grid.add_layer("color", z_index=2)
fog_layer = grid.add_layer("color", z_index=3)
# Populate each with different data...
# highlight_layer shows selected cells
# danger_layer shows enemy threat zones
# fog_layer shows visibility
# Toggle visibility to switch which overlay shows
def show_danger_zones():
highlight_layer.visible = False
danger_layer.visible = True
fog_layer.visible = False
def show_fog_of_war():
highlight_layer.visible = False
danger_layer.visible = False
fog_layer.visible = True
```
### Z-Index Reordering
Change layer order at runtime by modifying z_index:
```python
# Bring a layer to front
important_layer.z_index = 100
# Send a layer behind entities
background_layer.z_index = -10
```
**Note:** Changing z_index marks the layer dirty, triggering re-render of affected chunks.
### Constructor `layers={}` Limitations
The `layers={}` constructor argument is convenient but limited:
```python
# This creates layers, but ALL get negative z_index (below entities)
grid = mcrfpy.Grid(
grid_size=(50, 50),
layers={"ground": "color", "terrain": "tile", "overlay": "color"}
)
# Result: ground=-3, terrain=-2, overlay=-1 (all below entities!)
```
**For overlays above entities, use `add_layer()` explicitly:**
```python
grid = mcrfpy.Grid(grid_size=(50, 50), layers={})
ground = grid.add_layer("color", z_index=-2)
terrain = grid.add_layer("tile", z_index=-1)
overlay = grid.add_layer("color", z_index=1) # Above entities!
```
---
## Migration from Legacy API
Prior to the layer system, grids had built-in per-cell rendering via `UIGridPoint` properties. Here's how to recreate that behavior:
### Legacy Pattern (Pre-November 2025)
```python
# OLD API (no longer works):
grid = mcrfpy.Grid(50, 50, 16, 16)
cell = grid.at(x, y)
cell.tilesprite = 42 # Sprite index
cell.color = (255, 0, 0) # Background color
```
### Equivalent Modern Pattern
```python
# NEW API - explicit layers:
grid = mcrfpy.Grid(grid_size=(50, 50), pos=(0, 0), size=(400, 400))
# Grid creates a default TileLayer; access it:
tile_layer = grid.layers[0]
tile_layer.set(x, y, 42) # Sprite index
# For background colors, add a ColorLayer behind tiles:
color_layer = grid.add_layer("color", z_index=-2) # Behind default tile layer
color_layer.set(x, y, mcrfpy.Color(255, 0, 0))
```
### Recreating Old Three-Layer Rendering
The legacy system rendered: background color → tile sprite → FOV overlay.
```python
# Create grid with no default layers
grid = mcrfpy.Grid(
grid_size=(50, 50),
pos=(100, 100),
size=(400, 400),
texture=tileset,
layers={}
)
# Layer 1: Background colors (z=-2, behind everything)
background = grid.add_layer("color", z_index=-2)
# Layer 2: Tile sprites (z=-1, above background, below entities)
tiles = grid.add_layer("tile", z_index=-1, texture=tileset)
# Layer 3: FOV overlay (z=+1, above entities)
fog = grid.add_layer("color", z_index=1)
# Now populate layers as needed
background.fill(mcrfpy.Color(20, 20, 30)) # Dark blue background
for x in range(50):
for y in range(50):
tiles.set(x, y, calculate_tile_index(x, y))
# FOV overlay updated after compute_fov() calls
```
---
## Common Issues
### Issue: Layer Changes Don't Appear
**Cause:** Layer content changed but chunk not marked dirty.
**Fix:** Use layer methods (`set()`, `fill()`) which automatically mark dirty. Direct property manipulation may bypass dirty flagging.
### Issue: Overlay Appears Behind Entities
**Cause:** Layer z_index is negative.
**Fix:** Use `z_index >= 0` for overlays:
```python
overlay = grid.add_layer("color", z_index=1) # Not z_index=-1
```
### Issue: Performance Degrades with Many Changes
**Cause:** Each `set()` call can mark a chunk dirty; many scattered changes = many chunk redraws.
**Fix:** Batch logically-related changes. For bulk updates, consider:
```python
# Less efficient: 10,000 individual calls
for x in range(100):
for y in range(100):
layer.set(x, y, value)
# More efficient when available: bulk fill
layer.fill(mcrfpy.Color(0, 0, 0)) # Single operation
```
Batch operations API ([#113](../../issues/113)) will provide more patterns.
---
## API Quick Reference
**Grid Properties:**
- `center`, `center_x`, `center_y` - Camera position (pixels in grid space)
- `zoom` - Scale factor
- `layers` - List of layer objects, sorted by z_index
- `fill_color` - Grid background (behind all layers)
**Grid Methods:**
- `add_layer(type, z_index, texture)` - Create new layer
- `remove_layer(layer)` - Remove a layer
**Layer Properties:**
- `z_index` - Render order
- `visible` - Show/hide layer
- `grid_size` - Dimensions (read-only)
**Layer Methods:**
- `set(x, y, value)` - Set cell (color or sprite index)
- `at(x, y)` - Get cell value
- `fill(value)` - Fill all cells
---
**Navigation:**
- [[Grid-System]] - Parent page, layer concepts
- [[Grid-TCOD-Integration]] - FOV computation details
- [[Performance-and-Profiling]] - Metrics and optimization
- [[Entity-Management]] - Entity rendering
---
# Grid Rendering Pipeline
## Overview
The Grid rendering pipeline handles multi-layer tilemap rendering with chunk-based caching for optimal performance. It supports arbitrary numbers of rendering layers, viewport culling, zoom, and entity rendering.
**Parent Page:** [[Grid-System]]
**Related Pages:**
- [[Performance-and-Profiling]] - Metrics and profiling tools
- [[Entity-Management]] - Entity rendering within grids
- [[Grid-TCOD-Integration]] - FOV and pathfinding (partially in transition)
**Key Files:**
- `src/UIGrid.cpp::render()` - Main rendering orchestration
- `src/GridLayers.cpp` - ColorLayer and TileLayer rendering
- `src/UIGrid.cpp::renderChunk()` - Per-chunk RenderTexture management
**Related Issues:**
- [#123](../../issues/123) - Chunk-based Grid Rendering (Closed - Implemented)
- [#148](../../issues/148) - Dirty Flag RenderTexture Caching (Closed - Implemented)
- [#147](../../issues/147) - Dynamic Layer System (Closed - Implemented)
- [#113](../../issues/113) - Batch Operations API (Open - includes FOV access discussion)
- [#115](../../issues/115) - SpatialHash for entity culling (Open)
---
## Architecture Overview
### Layer-Based Rendering
Grids render content through **layers** rather than per-cell properties. Each layer is either a `ColorLayer` (solid colors) or `TileLayer` (texture sprites).
**Render order is determined by z_index:**
- Layers with `z_index < 0` render **below** entities
- Entities render at the z=0 boundary
- Layers with `z_index >= 0` render **above** entities (overlays)
Within each group, lower z_index values render first (behind higher values).
```
z_index: -3 -2 -1 0 +1 +2
↓ ↓ ↓ ↓ ↓ ↓
[background] [tiles] [ENTITIES] [fog] [UI overlay]
```
**Implementation:** See `UIGrid::render()` which iterates layers in z_index order, inserting entity rendering at the 0 boundary.
### Chunk-Based Caching
Large grids are divided into **chunks** (regions of cells). Each chunk maintains its own `sf::RenderTexture` that caches the rendered result.
**Key concepts:**
- Chunk size is implementation-defined (currently ~256 cells per dimension)
- Only chunks intersecting the viewport are considered for rendering
- Each chunk tracks whether its content is "dirty" (needs redraw)
- Static content renders once, then the cached texture is reused
**Implementation:** See `UIGrid::renderChunk()` for chunk texture management.
### Dirty Flag Propagation
When layer content changes, only affected chunks are marked dirty:
1. `layer.set(x, y, value)` marks the containing chunk as dirty
2. On next render, dirty chunks redraw to their RenderTexture
3. Clean chunks simply blit their cached texture
This means a 1000x1000 grid with one changing cell redraws only ~1 chunk, not 1,000,000 cells.
**Implementation:** Dirty flags propagate through `UIGrid::markDirty()` and are checked in `UIGrid::render()`.
---
## Render Pipeline Stages
### Stage 1: Viewport Calculation
Calculate which chunks and cells are visible based on camera position, zoom, and grid dimensions.
**Key properties:**
- `center` / `center_x`, `center_y` - Camera position in pixel coordinates (within grid space)
- `zoom` - Scale factor (1.0 = normal, 2.0 = 2x magnification)
- `size` - Viewport dimensions in screen pixels
**Implementation:** Viewport bounds calculated at start of `UIGrid::render()`.
### Stage 2: Below-Entity Layers
For each layer with `z_index < 0`, sorted by z_index:
1. Determine which chunks intersect viewport
2. For each visible chunk:
- If dirty: redraw layer content to chunk's RenderTexture
- Draw chunk's cached texture to output
### Stage 3: Entity Rendering
Entities render at the z=0 boundary:
1. Iterate entity collection
2. Cull entities outside viewport bounds
3. Draw visible entity sprites at interpolated positions
**Note:** Entity culling is currently O(n). SpatialHash optimization planned in [#115](../../issues/115).
### Stage 4: Above-Entity Layers
For each layer with `z_index >= 0`, sorted by z_index:
- Same chunk-based rendering as Stage 2
- These layers appear as overlays (fog, highlights, UI elements)
### Stage 5: Final Compositing
All rendered content exists in the grid's output RenderTexture, which is drawn to the window in a single operation.
---
## Performance Characteristics
**Static Grids:**
- Near-zero CPU cost after initial render
- Cached chunk textures reused frame-to-frame
- Only viewport calculation and texture blitting
**Dynamic Grids:**
- Cost proportional to number of dirty chunks
- Single-cell changes affect only one chunk
- Bulk operations should batch changes before render
**Large Grids:**
- 1000x1000+ grids render efficiently
- Only visible chunks processed
- Memory scales with grid size (chunk textures)
**Profiling:** Use `mcrfpy.getMetrics()` or the F3 overlay to see render times. See [[Performance-and-Profiling]].
---
## FOV and Perspective System (In Transition)
**Current Status:** The FOV computation (`compute_fov()`, `is_in_fov()`) works correctly for pathfinding and visibility queries. However, the automatic fog-of-war overlay system is **not currently connected** to the new layer architecture.
**What works:**
- `grid.compute_fov(x, y, radius)` - Computes which cells are visible
- `grid.is_in_fov(x, y)` - Queries visibility of a specific cell
- Pathfinding uses walkable/transparent properties correctly
**What's in transition:**
- Per-entity perspective (`UIGridPointState`) not yet exposed to Python
- Automatic fog overlay rendering disconnected from layer system
- Batch FOV data access being designed ([#113 discussion](../../issues/113))
**Current workaround - Manual fog layer:**
```python
# Create a fog overlay layer (positive z_index = above entities)
fog_layer = grid.add_layer("color", z_index=1)
# After computing FOV, update fog colors
grid.compute_fov(player.grid_x, player.grid_y, radius=10)
for x in range(grid.grid_x):
for y in range(grid.grid_y):
if not grid.is_in_fov(x, y):
# Dim color for non-visible cells
fog_layer.set(x, y, mcrfpy.Color(0, 0, 0, 192))
else:
# Transparent for visible cells
fog_layer.set(x, y, mcrfpy.Color(0, 0, 0, 0))
```
**Limitation:** This O(n²) iteration is inefficient for large grids. Batch operations ([#113](../../issues/113)) will address this with patterns like `cells_in_radius()` iterators.
---
## Layer Techniques
### Multiple Overlay Layers
You can pre-create multiple overlay layers and toggle between them:
```python
# Create several overlay options
highlight_layer = grid.add_layer("color", z_index=1)
danger_layer = grid.add_layer("color", z_index=2)
fog_layer = grid.add_layer("color", z_index=3)
# Populate each with different data...
# highlight_layer shows selected cells
# danger_layer shows enemy threat zones
# fog_layer shows visibility
# Toggle visibility to switch which overlay shows
def show_danger_zones():
highlight_layer.visible = False
danger_layer.visible = True
fog_layer.visible = False
def show_fog_of_war():
highlight_layer.visible = False
danger_layer.visible = False
fog_layer.visible = True
```
### Z-Index Reordering
Change layer order at runtime by modifying z_index:
```python
# Bring a layer to front
important_layer.z_index = 100
# Send a layer behind entities
background_layer.z_index = -10
```
**Note:** Changing z_index marks the layer dirty, triggering re-render of affected chunks.
### Constructor `layers={}` Limitations
The `layers={}` constructor argument is convenient but limited:
```python
# This creates layers, but ALL get negative z_index (below entities)
grid = mcrfpy.Grid(
grid_size=(50, 50),
layers={"ground": "color", "terrain": "tile", "overlay": "color"}
)
# Result: ground=-3, terrain=-2, overlay=-1 (all below entities!)
```
**For overlays above entities, use `add_layer()` explicitly:**
```python
grid = mcrfpy.Grid(grid_size=(50, 50), layers={})
ground = grid.add_layer("color", z_index=-2)
terrain = grid.add_layer("tile", z_index=-1)
overlay = grid.add_layer("color", z_index=1) # Above entities!
```
---
## Migration from Legacy API
Prior to the layer system, grids had built-in per-cell rendering via `UIGridPoint` properties. Here's how to recreate that behavior:
### Legacy Pattern (Pre-November 2025)
```python
# OLD API (no longer works):
grid = mcrfpy.Grid(50, 50, 16, 16)
cell = grid.at(x, y)
cell.tilesprite = 42 # Sprite index
cell.color = (255, 0, 0) # Background color
```
### Equivalent Modern Pattern
```python
# NEW API - explicit layers:
grid = mcrfpy.Grid(grid_size=(50, 50), pos=(0, 0), size=(400, 400))
# Grid creates a default TileLayer; access it:
tile_layer = grid.layers[0]
tile_layer.set(x, y, 42) # Sprite index
# For background colors, add a ColorLayer behind tiles:
color_layer = grid.add_layer("color", z_index=-2) # Behind default tile layer
color_layer.set(x, y, mcrfpy.Color(255, 0, 0))
```
### Recreating Old Three-Layer Rendering
The legacy system rendered: background color → tile sprite → FOV overlay.
```python
# Create grid with no default layers
grid = mcrfpy.Grid(
grid_size=(50, 50),
pos=(100, 100),
size=(400, 400),
texture=tileset,
layers={}
)
# Layer 1: Background colors (z=-2, behind everything)
background = grid.add_layer("color", z_index=-2)
# Layer 2: Tile sprites (z=-1, above background, below entities)
tiles = grid.add_layer("tile", z_index=-1, texture=tileset)
# Layer 3: FOV overlay (z=+1, above entities)
fog = grid.add_layer("color", z_index=1)
# Now populate layers as needed
background.fill(mcrfpy.Color(20, 20, 30)) # Dark blue background
for x in range(50):
for y in range(50):
tiles.set(x, y, calculate_tile_index(x, y))
# FOV overlay updated after compute_fov() calls
```
---
## Common Issues
### Issue: Layer Changes Don't Appear
**Cause:** Layer content changed but chunk not marked dirty.
**Fix:** Use layer methods (`set()`, `fill()`) which automatically mark dirty. Direct property manipulation may bypass dirty flagging.
### Issue: Overlay Appears Behind Entities
**Cause:** Layer z_index is negative.
**Fix:** Use `z_index >= 0` for overlays:
```python
overlay = grid.add_layer("color", z_index=1) # Not z_index=-1
```
### Issue: Performance Degrades with Many Changes
**Cause:** Each `set()` call can mark a chunk dirty; many scattered changes = many chunk redraws.
**Fix:** Batch logically-related changes. For bulk updates, consider:
```python
# Less efficient: 10,000 individual calls
for x in range(100):
for y in range(100):
layer.set(x, y, value)
# More efficient when available: bulk fill
layer.fill(mcrfpy.Color(0, 0, 0)) # Single operation
```
Batch operations API ([#113](../../issues/113)) will provide more patterns.
---
## API Quick Reference
**Grid Properties:**
- `center`, `center_x`, `center_y` - Camera position (pixels in grid space)
- `zoom` - Scale factor
- `layers` - List of layer objects, sorted by z_index
- `fill_color` - Grid background (behind all layers)
**Grid Methods:**
- `add_layer(type, z_index, texture)` - Create new layer
- `remove_layer(layer)` - Remove a layer
**Layer Properties:**
- `z_index` - Render order
- `visible` - Show/hide layer
- `grid_size` - Dimensions (read-only)
**Layer Methods:**
- `set(x, y, value)` - Set cell (color or sprite index)
- `at(x, y)` - Get cell value
- `fill(value)` - Fill all cells
---
**Navigation:**
- [[Grid-System]] - Parent page, layer concepts
- [[Grid-TCOD-Integration]] - FOV computation details
- [[Performance-and-Profiling]] - Metrics and optimization
- [[Entity-Management]] - Entity rendering
---
*Last updated: 2025-11-29*