Project 5: Sarah Forcier

README.md

@@ -3,26 +3,70 @@ WebGL Clustered Deferred and Forward+ Shading
 
 **University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 5**
 
-* (TODO) YOUR NAME HERE
-* Tested on: (TODO) **Google Chrome 222.2** on
-  Windows 22, i7-2222 @ 2.22GHz 22GB, GTX 222 222MB (Moore 2222 Lab)
+Sarah Forcier
 
-### Live Online
+Tested on GeForce GTX 1070
 
-[![](img/thumb.png)](http://TODO.github.io/Project5B-WebGL-Deferred-Shading)
+### [Live Demo](http://sarahforcier.github.io/Project5B-WebGL-Deferred-Shading)
 
-### Demo Video/GIF
+[![](img/final.png)](https://youtu.be/QgfXGOQ58Ss)
 
-[![](img/video.png)](TODO)
+### Overview
 
-### (TODO: Your README)
+This project implements clustered forward+ and deferred shading. These methods improve performance for scenes with many dynamic lights. Typical forward shading is slow in this case because each fragments is shaded for each light in the scene even if the light does not affect the fragment or the fragment is occluded. Deferred shading solves the latter problem by only shading non-occluded fragments. Clustered forward+ shading solves the other problem by computing which lights effect each fragment and only shading with those lights.   
 
-*DO NOT* leave the README to the last minute! It is a crucial part of the
-project, and we will not be able to grade you without a good README.
+### Clustered Forward +
 
-This assignment has a considerable amount of performance analysis compared
-to implementation work. Complete the implementation early to leave time!
+The method divides the viewing frustum into clusters and computes the number of lights that overlap each cluster and their indices. Then during shading, each fragment in every cluster can compute shading on just the lights that influence the cluster. Because the clusters are divided in view space, the clusters in the x and y plan are evenly distributed across screen space, as seen below. 
 
+| X Clusters | Y Clusters | Z Clusters | 
+| ----------- | ----------- | ----------- |
+| ![](img/clusterX.png) | ![](img/clusterY.png) | ![](img/clusterZ.png) | 
+
+On the CPU, the minimum and maximum cluster bounds are found for each light. The cluster bounds are defined by planes intersecting the eye position and evenly divide the frustum in each direction. First the cluster is found that contains the light. Then the distance between the light and each plane of the cluster bounds is calculated starting from the closest planes to the light and moving out. The boundary is found when the distance becomes greater than the light's radius. Every time a light is found to influence a cluster, the light count for that cluster is increased and the light's index is added to a texture. 
+
+| Number of Lights per Cluster |
+| ----------- |
+| ![](img/numLights.png) |
+
+#### Blinn-Phong Shading
+
+Blinn-Phong Shading was added without any effect on performance since the implemented shading model does not require additional textures. 
+
+![](img/blinn.png)
+
+### Clustered Deferred
+
+| Albedo | Normal | Position |
+| ----------- | ----------- | ----------- |
+| ![](img/albedo.png) | ![](img/normal.png) | ![](img/position.png) |
+
+Deferred Shading takes a 2-pass approach to fragment shading. In the first pass, all the visible fragments write shading properties to a g-buffer. In the second pass, these properties are used to calculate the final color and output to the framebuffer. This second pass only allows shading computation to be computed for non-occluded fragments. For a simple scene, the properties stored in the g buffer are color, position, and normal, as shown below.  
+
+| X | Y | Z | A |
+| ----------- | ----------- | ----------- | ----------- |
+| color.r | color.g | color.b | color.a |
+| position.x | position.y | position.z | 1.0 |
+| normal.x | normal.y | normal.z | 0.0 |
+
+#### 2-component normals
+
+This g-buffer layout is not optimal because often the color does not have an alpha value (in fact rendering with alpha in deferred shading is very difficult because the method relies on only having to shade on non-occluded fragments, but with alpha < 1.0, some of the occluded fragments would need to be shaded) and the fourth channels for the position and normal are not used. Instead, 2 components of the normal can be packed into these channels, and the last component can be calculated during shading based on normalization.
+
+| X | Y | Z | A |
+| ----------- | ----------- | ----------- | ----------- |
+| color.r | color.g | color.b | norm.x |
+| position.x | position.y | position.z | norm.y |
+
+### Performance
+
+Clustered Forward+ performs much better for scenes with multiple lights because the implementation focuses on reducing the number of required shading computations. Adding Deferred Shading to the clustered forward+ implementation adds a small performance improvement.   
+
+![](img/performance.png)
+
+More clusters to divide up the lights in the scene performs better, but when there are too many clusters, the light texture gets larger and can slow down the shader and calculating the minimum and maximum clusters on the CPU takes longer when more iterations are required. 
+
+![](img/numclusters.png)
 
 ### Credits
 

src/init.js

@@ -1,5 +1,5 @@
 // TODO: Change this to enable / disable debug mode
-export const DEBUG = true && process.env.NODE_ENV === 'development';
+export const DEBUG = false && process.env.NODE_ENV === 'development';
 
 import DAT from 'dat-gui';
 import WebGLDebug from 'webgl-debug';
@@ -60,7 +60,7 @@ stats.domElement.style.top = '0px';
 document.body.appendChild(stats.domElement);
 
 // Initialize camera
-export const camera = new PerspectiveCamera(75, canvas.clientWidth / canvas.clientHeight, 0.1, 1000);
+export const camera = new PerspectiveCamera(75, canvas.clientWidth / canvas.clientHeight, 0.1, 50);
 
 // Initialize camera controls
 export const cameraControls = new OrbitControls(camera, canvas);

src/main.js

@@ -21,10 +21,10 @@ function setRenderer(renderer) {
       params._renderer = new ForwardRenderer();
       break;
     case CLUSTERED_FORWARD_PLUS:
-      params._renderer = new ClusteredForwardPlusRenderer(15, 15, 15);
+      params._renderer = new ClusteredForwardPlusRenderer(15,15,15);
       break;
     case CLUSTERED_DEFFERED:
-      params._renderer = new ClusteredDeferredRenderer(15, 15, 15);
+      params._renderer = new ClusteredDeferredRenderer(15,15,15);
       break;
   }
 }

src/renderers/clustered.js

@@ -1,28 +1,195 @@
+// 
+// http://gamedevs.org/uploads/fast-extraction-viewing-frustum-planes-from-world-view-projection-matrix.pdf
+
 import { mat4, vec4, vec3 } from 'gl-matrix';
 import { NUM_LIGHTS } from '../scene';
 import TextureBuffer from './textureBuffer';
 
-export const MAX_LIGHTS_PER_CLUSTER = 100;
+export const MAX_LIGHTS_PER_CLUSTER = 150;
+
+function sinatan(d) {
+  return d / Math.sqrt(1 + d * d);
+}
+
+function cosatan(d) {
+  return 1.0 / Math.sqrt(1 + d * d);
+}
 
 export default class ClusteredRenderer {
   constructor(xSlices, ySlices, zSlices) {
     // Create a texture to store cluster data. Each cluster stores the number of lights followed by the light indices
+
     this._clusterTexture = new TextureBuffer(xSlices * ySlices * zSlices, MAX_LIGHTS_PER_CLUSTER + 1);
     this._xSlices = xSlices;
     this._ySlices = ySlices;
     this._zSlices = zSlices;
   }
 
   updateClusters(camera, viewMatrix, scene) {
-    // TODO: Update the cluster texture with the count and indices of the lights in each cluster
-    // This will take some time. The math is nontrivial...
 
     for (let z = 0; z < this._zSlices; ++z) {
       for (let y = 0; y < this._ySlices; ++y) {
         for (let x = 0; x < this._xSlices; ++x) {
           let i = x + y * this._xSlices + z * this._xSlices * this._ySlices;
+
           // Reset the light count to 0 for every cluster
           this._clusterTexture.buffer[this._clusterTexture.bufferIndex(i, 0)] = 0;
+
+        }
+      }
+    }
+
+    // Perspective Camera attributes (reference: https://threejs.org/docs/index.html#api/cameras/PerspectiveCamera)
+    // fov: Camera frustum vertical field of view, from bottom to top of view, in degrees
+    // aspect: Camera frustum aspect ratio, usually the canvas width / canvas height.
+    // near: Camera frustum near plane
+    // far: Camera frustum far plane
+    var stride = 2.0 * Math.tan(camera.fov * 0.5 * Math.PI / 180);
+    var strideX = stride * camera.aspect / this._xSlices;
+    var strideY = stride / this._ySlices;
+    var strideZ = (camera.far - camera.near) / this._zSlices;
+
+    var lightPos = vec4.create();
+    var lightDir = vec3.create();
+
+    for (var li = 0; li < NUM_LIGHTS; ++li) {
+      // light variables
+      var lightRadius = scene.lights[li].radius;
+      lightPos[0] = scene.lights[li].position[0];
+      lightPos[1] = scene.lights[li].position[1];
+      lightPos[2] = scene.lights[li].position[2];
+      lightPos[3] = 1;
+
+      // transform from World space to eye space
+      vec4.transformMat4(lightPos, lightPos, viewMatrix); 
+
+      // determine which cluster contains light center
+      // and if it lies within camera frustum
+      vec3.set(lightDir, lightPos[0], lightPos[1], -lightPos[2]);
+      vec3.normalize(lightDir, lightDir);
+      var t = 1.0 / lightDir[2];
+      var clusterX = Math.floor((t * lightDir[0] + strideX * (this._xSlices / 2.0)) / strideX);
+      // if (clusterX < 0 || clusterX >= this._xSlices) continue;
+      var clusterY = Math.floor((t * lightDir[1] + strideY * (this._ySlices / 2.0)) / strideY);
+      // if (clusterY < 0 || clusterY >= this._ySlices) continue;
+      var clusterZ = Math.floor((-lightPos[2] - camera.near) / strideZ);
+      // if (clusterZ < 0 || clusterZ >= this._zSlices) continue;
+
+      var normal = vec4.create();
+      normal[3] = 0;
+      var d;
+      var distance;
+      // ----------------------
+      // XZ planes (vertical)
+      // ----------------------
+      normal[1] = 0.0;
+      var minX_cluster;
+      d = (clusterX - this._xSlices / 2.0) * strideX;
+      for (minX_cluster = clusterX; minX_cluster > 0; --minX_cluster, d -= strideX) {
+        normal[0] = cosatan(d);
+        normal[2] = sinatan(d);
+        vec4.normalize(normal, normal);
+        distance = Math.abs(vec4.dot(normal, lightPos));
+        if (distance > lightRadius) {
+          break;
+        }
+      }
+      if (minX_cluster >= this._xSlices) continue;
+      minX_cluster = Math.max(minX_cluster, 0);
+
+      var maxX_cluster;
+      d = (clusterX + 1 - this._xSlices / 2.0) * strideX;
+      for (maxX_cluster = clusterX + 1; maxX_cluster < this._xSlices; ++maxX_cluster, d += strideX) {
+        normal[0] = cosatan(d);
+        normal[2] = sinatan(d);
+        vec4.normalize(normal, normal);
+        distance = Math.abs(vec4.dot(normal, lightPos));
+        if (distance > lightRadius) {
+          break;
+        }
+      }
+      if (maxX_cluster < 0) continue;
+      maxX_cluster = Math.min(maxX_cluster, this._xSlices);
+
+      // ----------------------
+      // YZ planes (horizontal)
+      // ----------------------
+      normal[0] = 0.0;
+      var minY_cluster;
+      d = (clusterY - this._ySlices / 2.0) * strideY;
+      for (minY_cluster = clusterY; minY_cluster > 0; --minY_cluster, d -= strideY) {
+        normal[1] = cosatan(d);
+        normal[2] = sinatan(d);
+        vec4.normalize(normal, normal);
+        distance = Math.abs(vec4.dot(normal, lightPos));
+        if (distance > lightRadius) {
+          break;
+        }
+      }
+      if (minY_cluster >= this._ySlices) continue;
+      minY_cluster = Math.max(minY_cluster, 0);
+
+      var maxY_cluster;
+      d = (clusterY + 1 - this._ySlices / 2.0) * strideY;
+      for (maxY_cluster = clusterY + 1; maxY_cluster < this._ySlices; ++maxY_cluster, d += strideY) {
+        normal[1] = cosatan(d);
+        normal[2] = sinatan(d);
+        vec4.normalize(normal, normal);
+        distance = Math.abs(vec4.dot(normal, lightPos));
+        if (distance > lightRadius) {
+          break;
+        }
+      }
+      if (maxY_cluster < 0) continue;
+      maxY_cluster = Math.min(maxY_cluster, this._ySlices);
+
+      // ----------------------
+      // XY planes (perpendicular to camera)
+      // ----------------------
+      var minZ_cluster;
+      d = - clusterZ * strideZ;
+      for (minZ_cluster = clusterZ; minZ_cluster > 0; --minZ_cluster, d += strideZ) {
+        distance = - lightPos[2] - camera.near + d;
+        if (distance > lightRadius) {
+          break;
+        }
+      }
+      if (minZ_cluster >= this._zSlices) continue;
+      minZ_cluster = Math.max(minZ_cluster, 0);
+
+      var maxZ_cluster;
+      d = (clusterZ + 1) * strideZ;
+      for (maxZ_cluster = clusterZ + 1; maxZ_cluster < this._zSlices; ++maxZ_cluster, d += strideZ) {
+        distance = d + lightPos[2];
+        if (distance > lightRadius) {
+          break;
+        }
+      }
+      if (maxZ_cluster < 0) continue;
+      maxZ_cluster = Math.min(maxZ_cluster, this._zSlices);
+
+      // update cluster lights
+      // ranges partial inclusive [min, max)
+      for (let c_z = minZ_cluster; c_z < maxZ_cluster; ++c_z) {
+        for (let c_y = minY_cluster; c_y < maxY_cluster; ++c_y) {
+          for (let c_x = minX_cluster; c_x < maxX_cluster; ++c_x) {
+            let j = c_x + c_y * this._xSlices + c_z * this._xSlices * this._ySlices;
+
+            //     cluster0    |     cluster1    |    cluster 2
+            // num, i0, i1, i2 | num, i0, i1, i2 | num, i0, i1, i2
+            // i3, i4, i5, i6  | i3, i4, i5, i6  | i3, i4, i5, i6
+            //        ...      |       ...       |     ... 
+            let lightCountIndex = this._clusterTexture.bufferIndex(j, 0);
+            let numLights = this._clusterTexture.buffer[lightCountIndex];
+            let nextLightIndex = numLights;
+            numLights++;
+            if (numLights <= MAX_LIGHTS_PER_CLUSTER) {
+              var pixelRow = Math.floor((nextLightIndex + 1) * 0.25);
+              var pixelComponent = (nextLightIndex + 1) - 4 * pixelRow;
+              this._clusterTexture.buffer[this._clusterTexture.bufferIndex(j, pixelRow) + pixelComponent] = li;
+              this._clusterTexture.buffer[lightCountIndex] = numLights;
+            }
+          }
         }
       }
     }

src/renderers/clusteredDeferred.js

@@ -7,9 +7,9 @@ import toTextureFrag from '../shaders/deferredToTexture.frag.glsl';
 import QuadVertSource from '../shaders/quad.vert.glsl';
 import fsSource from '../shaders/deferred.frag.glsl.js';
 import TextureBuffer from './textureBuffer';
-import ClusteredRenderer from './clustered';
+import ClusteredRenderer, {MAX_LIGHTS_PER_CLUSTER} from './clustered';
 
-export const NUM_GBUFFERS = 4;
+export const NUM_GBUFFERS = 2;
 
 export default class ClusteredDeferredRenderer extends ClusteredRenderer {
   constructor(xSlices, ySlices, zSlices) {
@@ -21,15 +21,16 @@ export default class ClusteredDeferredRenderer extends ClusteredRenderer {
     this._lightTexture = new TextureBuffer(NUM_LIGHTS, 8);
 
     this._progCopy = loadShaderProgram(toTextureVert, toTextureFrag, {
-      uniforms: ['u_viewProjectionMatrix', 'u_colmap', 'u_normap'],
+      uniforms: ['u_viewProjectionMatrix', 'u_viewMatrix', 'u_colmap', 'u_normap'],
       attribs: ['a_position', 'a_normal', 'a_uv'],
     });
 
     this._progShade = loadShaderProgram(QuadVertSource, fsSource({
       numLights: NUM_LIGHTS,
+      numLightsPerCluster: MAX_LIGHTS_PER_CLUSTER,
       numGBuffers: NUM_GBUFFERS,
     }), {
-      uniforms: ['u_gbuffers[0]', 'u_gbuffers[1]', 'u_gbuffers[2]', 'u_gbuffers[3]'],
+      uniforms: ['u_gbuffers[0]', 'u_gbuffers[1]', 'u_viewMatrix', 'u_InvViewMatrix', 'u_lightbuffer', 'u_clusterbuffer', 'u_slices', 'u_cameranear', 'u_camerafar', 'u_camerapos', 'u_res'],
       attribs: ['a_uv'],
     });
 
@@ -123,6 +124,7 @@ export default class ClusteredDeferredRenderer extends ClusteredRenderer {
 
     // Upload the camera matrix
     gl.uniformMatrix4fv(this._progCopy.u_viewProjectionMatrix, false, this._viewProjectionMatrix);
+    gl.uniformMatrix4fv(this._progCopy.u_viewMatrix, false, this._viewMatrix);
 
     // Draw the scene. This function takes the shader program so that the model's textures can be bound to the right inputs
     scene.draw(this._progCopy);
@@ -153,7 +155,28 @@ export default class ClusteredDeferredRenderer extends ClusteredRenderer {
     // Use this shader program
     gl.useProgram(this._progShade.glShaderProgram);
 
+     // Upload the camera matrix
+    gl.uniformMatrix4fv(this._progShade.u_viewMatrix, false, this._viewMatrix);
+    var inverse = mat4.create();
+    mat4.invert(inverse, this._viewMatrix);
+    gl.uniformMatrix4fv(this._progShade.u_InvViewMatrix, false, inverse);
+
+    // Set the light texture as a uniform input to the shader
+    gl.activeTexture(gl.TEXTURE2);
+    gl.bindTexture(gl.TEXTURE_2D, this._lightTexture.glTexture);
+    gl.uniform1i(this._progShade.u_lightbuffer, 2);
+
+    // Set the cluster texture as a uniform input to the shader
+    gl.activeTexture(gl.TEXTURE3);
+    gl.bindTexture(gl.TEXTURE_2D, this._clusterTexture.glTexture);
+    gl.uniform1i(this._progShade.u_clusterbuffer, 3);
+
     // TODO: Bind any other shader inputs
+    gl.uniform1f(this._progShade.u_cameranear, camera.near); 
+    gl.uniform1f(this._progShade.u_camerafar, camera.far);
+    gl.uniform3f(this._progShade.u_camerapos, camera.position.x, camera.position.y, camera.position.z);
+    gl.uniform2f(this._progShade.u_res, canvas.width, canvas.height); 
+    gl.uniform3f(this._progShade.u_slices, this._xSlices, this._ySlices, this._zSlices);
 
     // Bind g-buffers
     const firstGBufferBinding = 0; // You may have to change this if you use other texture slots