Project Setup
Add a plane and a sphere in your Unity scene.
Apply a surface shader on sphere, and colorful texture.
and unlit shader on plane.
In this tutorial we will understand the Stencil Effect in Unity.
Add a plane and a sphere in your Unity scene.
Apply a surface shader on sphere, and colorful texture.
and unlit shader on plane.
The plane & sphere covers some pixels of the screen, and unlit shader on the plane & surface shader on sphere is responsible to set/define the color of these pixels.
All the pixels that the plane cover's, has color white, and all the pixels the sphere cover's has random color.
These grey pixels are large, but consider them as pixels, just for understanding.
Besides this, the pixels before getting drawn on the screen have to pass a few Unity internal tests/checks, and one of these tests/checks is stencil tests/checks.
For all the pixels on the screen unity has stored a stencil value in the stencil buffer, by default the stencil value of all pixels is 0.
The shader's on plane & sphere also have a stencil value, which is also 0 by default.
In this way one stencil value the shader holds and one stencil value the Unity's stencil buffer holds, for comparison.
Each pixel before getting drawn on the screen compares its shader stencil value with the stencil value stored in the Unity's stencil buffer.
If both stencil values of (shader & Unity's stencil buffer) are same the pixel will be drawn on screen.
Currently these values are same on both sides so pixel are visible on screen.
We can change the stencil value on the shader, and also on Unity's stencil buffer.
First we will change the stencil value of shader's.
Write a stencil BLOCK in both shaders. (mentioned with orange cell)
Ref 0, is how you define stencil value on shader, you could give it any number from 0 - 255.
Comp Equal, its means if shader stencil value is equal to the stencil value stored in Unity's stencil buffer, then draw the pixel, otherwise don't draw it on screen.
There are many Comp types like, Equal, Always, etc.
We will cover that in later blog's.
If we set the Ref value to 1, then no pixel will draw on screen, because shader stencil value for the plane & sphere pixels, is equal to 1, and the other one stored on Unity's stencil buffer is 0 for all pixels.
Next we will change the stencil value stored on Unity's stencil buffer.
There could be any other way to change stencil value on Unity's stencil buffer.
But We will do that with the unlit shader of plane.
Apply the changes in your unlit shader, highlighted with orange cells.
The plane will not draw itself, intead it will only change the stencil buffer value.
Ref 1
Comp Always , means shader will not compare the stencil value with stencil buffer.
Pass Replace, this line will change the stencil value in Unity's stencil buffer, of the pixels that the plane covers.
Now the unlit shader of the plane is changing the Unity's stencil buffer values for the pixels that it covers.
If we move the plane to sphere position then it will change the stencil buffer value of those pixels to 1, and stencil value of sphere's surface shader is also 1, so pixels will draw on screen.
Or you could say the sphere is visible inside the plane area, or plane is behaving like a 3D mask.
Changing the stencil value From inspector so we could create many, 3D mask pairs.
Capsule & Capsule Mask, both materials have stencil value 1.
Sphere & Sphere Mask, both materials have stencil value 2.
