The application of a real-time stencil feed provides values for illumination (e.g. blooming, fire, light sources), for semi-transparency (e.g. mist, smoke, cloud), and for opacity (solid objects). The stencil defines the mathematical operations required, per pixel, to composite the real-time colour pixel and playback colour pixel.
After the depth comparison between playback and real-time pixels, the stencil feed is applied if required. As above, the following formulae are applied per pixel where:
PB = Playback image value
RT = Realtime input value
V = temporary calculated value
O = Output value
Real-time assets (e.g. from Unity or Unreal gaming engines) have their own colour media (RGB), depth buffer values and a stencil (per pixel mask) overlay. Together, we have depth value and colour value for every pixel in the playback and real-time input frames.
Considerations
•Depth information is mainly useful if playback (pre-rendered) assets are opaque.
•For semi-transparent pre-rendered assets, the stencil plane generation needs to take depth into account. This requires the real-time engine to have loaded all scene models / geometry.