RealityCapture and Fabric Engine in VR

RealityCapture and Fabric Engine was used to bring Saint Martin’s Church to Virtual Reality

Source: www.saint-martin-de-londres.fr

Saint Martin is an 11th century Romanesque church which can be found in Saint-Martin-De-Londres, a small town in France, not far from Montpellier. This beautiful church is the key sight in this lovely French village as it is built in the clover-leaf shape. The aim of the project was to capture the church and reconstruct it in a way that it would be VR ready. In order to achieve that, an automated procedural pipeline has been developed.

Altogether 2,500 images have been captured with 2 different cameras. Approximately 2,000 area pictures and 500 detail pictures have been taken. The processing has taken 4 days and 3 hours on 3.5Ghz I7 x4.

Render from Virtual Reality

Production pipeline

First of all, the images have been processed in RealityCapture (RC). This resulted in 770 million triangles, which have then been simplified in RC to 10 million triangles. Low and high resolution mesh have been exported in parts and used in Fabric Engine (FE) to process bounding boxes, cut the mesh into textured pieces, compute normals and tangent space, re-lay out UVs and compute overlaps. Furthermore, FE has been used to composite the texture, and xNormal was used for most of the texture generation. This process is completely automatic and provides a nicely laid out asset which can be further reduced within the game engine (using LOD techniques).

“The quality of RealityCapture has been absolutely convincing. We were mainly impressed by series of things. No clean-up of the points was required – all scans came out fine first time. We were using variety of lenses, doing a solve with wide angles photos first and then aligning second time with all photos. RC always aligned the photos 100%. Lastly, the simplify and unfold tools are great, very fast and reliable.” Helge Mathee, software engineer at Fabric Engine

Blending of light

It is a mix of procedural material and lighting scenarios – standard Unreal Engine features. When you turn down the light, you can see the emissivity of the real texture. Scans were done in the diffuse light after the rain. Then re-lightning is very important – using cavity maps to remove cavity darkness.

For large environments, with some very dark parts, images were done with fixed exposure, but it had to be adapted. Overall, 3 different fixed exposures were used. What future holds?

For the future, the plan is to automate the pipeline as much as possible, beyond what is currently available. This includes introducing Unfold3D as an automatic step for all pieces, better cutting of existing pieces using convex decomposition, introducing Simplygon to service lower-end targets such as WebGL and using RealityCapture, for the export steps.

Here you can watch making of and the final result :

What is Fabric Engine

Fabric Engine (http://fabricengine.com/) is a digital content creation platform that enables anyone to build powerful tools and applications for VFX, games, virtual reality and visualization. It provides a visual programming system and scripting language on top of an execution engine that handles complex tasks like multithreading and GPU computing. Fabric enables artists and developers to rapidly develop custom tools and applications that boost productivity and integrate easily into the existing production pipelines.