In this worked example, an undersea scene is populated with swimming creatures from a single gaming source. Multiple sources can be used; the principle is the same: all depths are compared, stencils are applied and the final composited result is displayed.
What the illustrations show
•Playback and real-time asset frames are shown correctly in colour as they will be seen. They are the only colour frames you see.
•Depth frames are greyscale (white is nearest). Areas with all-zero values for RGB are shown here as black.
•Stencil frames have RGB values per pixel only as required. Here they are shown in a black frame for clarity and contrast. These frames are never seen in colour, but are used in Delta as the stencil data format.
Playback
The playback media (as we move around the scene) comprises a (colour) media layer and a (greyscale) depth layer:
Image sequence RGB layer |
Image sequence depth layer |
|
|
Real-time captured assets
Over this, the captured real-time assets are (colour) creatures media also has a (greyscale) depth layer:
Real-time sequence RGB layer |
Real-time sequence depth layer |
|
|
Finally, a stencil (data) layer comprising: highlights (red channel), shadow (green channel), and alpha transparency (blue channel) is added. First, here are the separate channels:
Highlights |
Shadows |
Alpha |
|
|
|
In practice, these are combined in a single RGB stencil image sequence, which is rendered in Delta with the data format: Stencil R(+) G(×) B(Alpha):
|
|
Composited show
Putting it together, the display becomes an interactive show, combining the playback environment with the real-time sea creatures:
Click to play this 1-minute video