Cesium Tools
When I joined the immersive media team at HNTB, one of the most time-consuming tasks was modeling the environment surrounding the main roadway or building we were visualizing.
As a result, we were an early adopter of the Cesium plugin for Unity, which allowed us to stream in Google satellite imagery as 3D tiles instead of having to model everything by hand.
Originally, the idea was to only use Cesium for far-background items and still model nearby objects by hand, but the quality of the Google satellite imagery was so good, we ended up wanting to use it for pretty much everything that wasn't the main model.
However it had a couple of limitiations. First, small objects like trees and light poles did not look nearly as good as large objects like buildings. And second, several of our projects featured large bodies of water and Cesium lacked a built-in water shader.
So I began adding these and other features to the Cesium shader to allow us to use Cesium more widely across our projects.
The first challenge was adding the ability to clip out small objects from the satellite imagery that did not look good up close. I had previously written a tool to clip out objects by drawing a polygon around it (Cesium eventually added this as a built-in feature), but this was mostly useful for large objects since it was quite time-consuming masking out dozens or hundreds of smaller objects by hand. The other issue was that drawing a Cesium polygon would remove the ground under the object as well, leaving holes in the environment. The solution we came up with was to mask out parts of the satellite imagery using Unity box colliders. This was not only faster than drawing out the shapes by hand, but also let us mask out objects while leaving the ground under it intact.
I implemented this by adding a custom box check function in the shader, which tested to see if a given pixel was inside any of the boxes in the scene. As time went on, however, we ran into an issue on larger projects where the framerate would slow down as the shader had to test against hundreds of boxes. One potential solution was to have people working in the scene use fewer boxes -- perhaps combining multiple small boxes into one larger one. However rather than put that extra burden on anyone who worked in the scene, I figured out a way to add spatial partitioning to the shader, meaning pixels only tested against nearby boxes and allowed us to support hundreds of boxes without affecting performance.
The other major issue was that most AEC visualizations had animated, reflective water and the satellite imagery only provided a static, flat color. Cesium supported the use of image masks, so I created a simple water shader and used the Cesium mask to define where to apply it.
This introduced another challenge in that, like the clipping polygons, the image masks were 2D and would apply the water material to objects above the water like bridges. The solution I came up with was creating multiple box types -- clipping, inverse clipping, water masking -- and creating a virtual texture that used RGB values to determine which boxes should be used for what. Not only did this allow us to remove the water effect from bridges, but also allowed us to perform boolean geometry like clipping an object with a polygon but retaining a small section of it using a box.
While it was a lot of work adding these features, Cesium was so ubiquitous in our projects that the time was more than worth it. By removing the limitations of Cesium like low quality trees and water, we were able to use Cesium more freely, replacing virtually all hand-modeling for background elements.
The first challenge was adding the ability to clip out small objects from the satellite imagery that did not look good up close. I had previously written a tool to clip out objects by drawing a polygon around it (Cesium eventually added this as a built-in feature), but this was mostly useful for large objects since it was quite time-consuming masking out dozens or hundreds of smaller objects by hand. The other issue was that drawing a Cesium polygon would remove the ground under the object as well, leaving holes in the environment. The solution we came up with was to mask out parts of the satellite imagery using Unity box colliders. This was not only faster than drawing out the shapes by hand, but also let us mask out objects while leaving the ground under it intact.
I implemented this by adding a custom box check function in the shader, which tested to see if a given pixel was inside any of the boxes in the scene. As time went on, however, we ran into an issue on larger projects where the framerate would slow down as the shader had to test against hundreds of boxes. One potential solution was to have people working in the scene use fewer boxes -- perhaps combining multiple small boxes into one larger one. However rather than put that extra burden on anyone who worked in the scene, I figured out a way to add spatial partitioning to the shader, meaning pixels only tested against nearby boxes and allowed us to support hundreds of boxes without affecting performance.
The other major issue was that most AEC visualizations had animated, reflective water and the satellite imagery only provided a static, flat color. Cesium supported the use of image masks, so I created a simple water shader and used the Cesium mask to define where to apply it.
This introduced another challenge in that, like the clipping polygons, the image masks were 2D and would apply the water material to objects above the water like bridges. The solution I came up with was creating multiple box types -- clipping, inverse clipping, water masking -- and creating a virtual texture that used RGB values to determine which boxes should be used for what. Not only did this allow us to remove the water effect from bridges, but also allowed us to perform boolean geometry like clipping an object with a polygon but retaining a small section of it using a box.
While it was a lot of work adding these features, Cesium was so ubiquitous in our projects that the time was more than worth it. By removing the limitations of Cesium like low quality trees and water, we were able to use Cesium more freely, replacing virtually all hand-modeling for background elements.