Raphaele G.

#graphics#shader#hlsl#unity#csharp#experiment#solo

How to: Mask Feathered Holes in Images, Unity (Part 1)

How to: Mask Feathered Holes in Images, Unity (Part 1)

A huge mechanic of Luminth was the uncovering of the maze through lights. However, Unity doesn't have a built-in function to create transparent circles, needing technical expertise.

Primitive Solution

Within 72 hours, the best solution was using the stencil buffer, to reject pixels on the cloud texture based on a circle image attached with the light.

timeline

This worked using two stencil buffer materials. The first one passing a ref of 1 with no color, and the second checks if the stencil buffer is not equal to 1 to draw. This was initially done with regular materials since it was a 2D image, but it can be replicated with ShaderLab.

Mask.shader
...
Stencil {
	Ref 1
	Comp Always
	Pass Replace
}
ColorMask 0
...
Always write 1 for the entirety of this object
Obj.shader
...
Stencil {
	Ref 1
	Comp NotEqual
	Pass Keep
}
...
If the stencil buffer is not equal to 1, keep the original value

Benefits:

  • Easy to implement
  • Easy to use with any GameObject

Disadvantages:

  • The edges are too harsh for the intended effect.

We can't get any further with stencil / masking as it only holds integers (not floating points). It literally accept or reject pixels.

Polished Solution

There exist many solutions based on seeing players through walls, but the applications are mainly 3D, and uses ShaderGraph from URP. Let's look at how we can write a solution based in HLSL.

timeline

1. Draw a transparent, soft circle on texture.

Clouds.shader
Tags { RenderType = Transparent, Queue = Transparent }
Blend SrcAlpha OneMinusAlpha
1.1. SubShader Characteristics
Clouds.shader
void drawCircle(in float2 uv, in float2 center, in float radius, in float smooth, out float output) {
	// Using Signed Distance from the center
	float sqrDistance = (uv.x - center.x) * (uv.x - center.x) + (uv.y - center.y) * (uv.y - center.y);
	float sqrRadius = radius * radius;
	if (sqrDistance < radius) {
		output = smoothstep(sqrRadius, sqrRadius-smoothValue, sqrDistance);
	} else { output = 0; }
}
1.2. Define a circle function
Clouds.shader
fixed4 frag(v2f i) : SV_Target {
	fixed4 col = tex2D(_MainTex, i.uv);
	drawCircle(i.uv, center, radius, smoothstepValue, outputAlpha);	// Defined properties
	return col * (1,1,1,1-outputAlpha);
}
1.3. Integrate the circle function with the frag Function

2. Dynamically draw a circle

Since it's planned to have multiple circles, the data structure will be an array.

Clouds.shader
sampler2D _MainTex;
float4 _MainTex_ST;
float4 _Points[100];	// The variable being modified. Capped at 100.
2.1. Define a data structure that reads points to make holes
Clouds.cs
public class Clouds : MonoBehaviour {
	public Material m;
	public Texture2D t;
	Vector4[] lights;

	void NewLight(Vector4 v) {
		lights[0] = v;
		m.SetVectorArray("_Points", lights);	// Updates the data the shader will use
	}

	void Start() {
		Vector4 v = new(x, y, radius, 0);	// The defined data structure
		NewLight(v);
	}
}
2.2. Use a C# script to populate the variable with data
Clouds.shader
fixed4 frag(v2f i) : SV_Target {
	fixed4 col = tex2D(_MainTex, i.uv);
	float center = _Points[0].xy;
	float radius = _Points[0].z;
	float smoothValue = radius / 20;	// Feathering based on radius
	drawCircle(i.uv, center, radius, smoothValue, outputAlpha);	// Defined properties
	return col * (1,1,1,1-outputAlpha);
}
2.3. Use the variables to make a circle

Once it's possible to pass one variable of points, it's easy to pass an array of points using a counter and a for-loop. I won't go through that here.

3. World Coordinates to UV Coordinates

The last challenge is properly covering the world coordinates to the correct UV coordinates.

  • In World Space, the coordinates align on an axis
  • In Texture Space, the coordinates starts at the bottom-left, and range from (0 ~ 1).

To translate world to texture:

  1. Get point xworld, centerworld (the offset from (0,0)world), and centertexture (by dividing the total texture size by 2)
  2. Find how far xworld is from centerworld as distanceworld
  3. xtexture = centertexture + distanceworld
  4. To translate to uv Space, normalise xtexture to sizetexture so it's within 0 ~ 1 values
    • Make sure to use spriteRenderer.size instead of Texture2D.width or height
Vector2 size = sprRend.size;
Vector2 centerT = size/2;

Vector3 centerW = gameObject.transform.localPosition;

Vector2 distance = x - centerW;

Vector2 xT = distance + centerT;
Vector2 nXT = xT / size;

lights[j] = new(nXT.x, nXT.y, x.z, 0);	// push to _Points array!

This is how Gradient Holes are made!

Be careful, since we're using a C# script to send data to the GPU, a lot of data from CPU will be passed down to GPU, potentially becoming a bottleneck if there's a large amount of data to send. (Specifically, O(max_lights, 4 floats) = O(10044) = O(1600 bytes))

If I were to continue improving this, I would also look into simplifying the calculations by manipulating the vert function in the shader file.

Source Code: Clouds.shader Source Code: Clouds.cs

Thank you for reading! Check out the game that used this, or the project details about it here!

Part 2: Player Silhouette Luminth Details