Frustrum Culling of Shadows

Introduction

When performing frustum culling on vegetation, sometimes shadows disappear from the camera even when they should be visible. Those shadows could be far behind the camera, up on a mountain, but since they are not being looked at, they could disappear.

You can circumvent this by not applying frustum culling to shadows, only the main mesh, but that means that shadows will be rendered for all meshes, regardless of their position. Another solution is using some kind of occlusion space to make sure a shadow is visible or not.

However, this last method can be quite hard to achieve when working in a fully procedural world.

My solution

When doing Frustum Culling, I perform another separate check for the shadows. The idea is:

Get the light direction.
Create a ray from the object following the light direction
Get the corners of the camera Frustum Planes
Verify if that ray goes through the Frustum Planes by doing a triangle-intersection check.

If it does, draw it, otherwise, discard it. All of this is done inside a compute shader. To my surprise, it works well! You can add more precision by taking the boundary of the mesh into account, instead of just one point.

Code

To better understand my implementation, I attach the code related to it.

Getting the Frustrum Corners

Vector3[] GetFrustumCorners(Camera cam)
{
    Vector3[] frustumCornersNear = new Vector3[4];

    // Near plane corners
    cam.CalculateFrustumCorners(new Rect(0, 0, 1, 1), cam.nearClipPlane, 
        Camera.MonoOrStereoscopicEye.Mono, frustumCornersNear);

    // Near plane corners in world space
    for (int i = 0; i < 4; i++)
    {
        frustumCornersNear[i] = cam.transform.TransformPoint(frustumCornersNear[i]);
    }

    Vector3[] frustumCornersFar = new Vector3[4];

    // Far plane corners
    cam.CalculateFrustumCorners(new Rect(0, 0, 1, 1), cam.farClipPlane, 
        Camera.MonoOrStereoscopicEye.Mono, frustumCornersFar);

    // Far plane corners in world space
    for (int i = 0; i < 4; i++)
    {
        frustumCornersFar[i] = cam.transform.TransformPoint(frustumCornersFar[i]);
    }

    Vector3[] concat = frustumCornersNear.Concat(frustumCornersFar).ToArray();
    return concat;
}

Checking if the light ray intersect a triangle

bool DoesRayIntersectFrustum(Ray ray, Vector3[] frustumCorners)
{
    // Define frustum triangles for each side
    Vector3[][] frustumTriangles = {
        // Left side
        new[] { frustumCorners[0], frustumCorners[1], frustumCorners[5] },
        new[] { frustumCorners[0], frustumCorners[5], frustumCorners[4] },
        // Right side
        new[] { frustumCorners[2], frustumCorners[3], frustumCorners[7] },
        new[] { frustumCorners[2], frustumCorners[7], frustumCorners[6] },
        // Top side
        new[] { frustumCorners[1], frustumCorners[2], frustumCorners[6] },
        new[] { frustumCorners[1], frustumCorners[6], frustumCorners[5] },
        // Bottom side
        new[] { frustumCorners[0], frustumCorners[3], frustumCorners[7] },
        new[] { frustumCorners[0], frustumCorners[7], frustumCorners[4] },
        // Near plane
        new[] { frustumCorners[0], frustumCorners[1], frustumCorners[2] },
        new[] { frustumCorners[2], frustumCorners[3], frustumCorners[0] },
        // Far plane
        new[] { frustumCorners[4], frustumCorners[5], frustumCorners[6] },
        new[] { frustumCorners[6], frustumCorners[7], frustumCorners[4] }
    };
    // Check for intersection with any triangle
    foreach (var triangle in frustumTriangles)
    {
        if (RayIntersectsTriangle(ray, triangle[0], triangle[1], triangle[2]))
            return true;
    }
   
    return false;
}

Ray to triangle intersection code

bool RayIntersectsTriangle(Ray ray, Vector3 v0, Vector3 v1, Vector3 v2)
{
    Vector3 edge1 = v1 - v0;
    Vector3 edge2 = v2 - v0;

    Vector3 h = Vector3.Cross(ray.direction, edge2);
    float a = Vector3.Dot(edge1, h);

    if (a > -0.00001f && a < 0.00001f) return false;

    float f = 1.0f / a;
    Vector3 s = ray.origin - v0;
    float u = f * Vector3.Dot(s, h);

    if (u < 0.0f || u > 1.0f) return false;

    Vector3 q = Vector3.Cross(s, edge1);
    float v = f * Vector3.Dot(ray.direction, q);

    if (v < 0.0f || u + v > 1.0f) return false;

    float t = f * Vector3.Dot(edge2, q);
    return t > 0.00001f;
}