TOM and RealityCapture

TOM is an integrated dance-for-camera installation that will be presented as a projection-mapped 3D visual and sound experience.

The project will bring together cutting-edge technology in film, animation, projection mapping, and sound design to create an immersive choreographic world for the protagonist to inhabit and the audience to be absorbed within. Wilkie will perform in the work alongside a younger version of himself, played by his nephew.

Commissioned and co-Produced by Sadler’s Wells., co-commissioned by Pavilion Dance South West, supported using public funding by the National Lottery through Arts Council England with additional support from the Unicorn Theatre, Hive House London, Theatre Bristol and Keir Moffatt Web Design. Research and development supported by Sadler’s Wells New Wave Associate Programme and Lakeside Arts Centre, Nottingham. Wilkie Branson is a Sadler’s Wells New Wave Associate supported by Esmèe Fairbairn Foundation.

“In this emotive choreography and handcrafted digital world, we follow TOM on his daily commute as he struggles to remember what life was like as a boy and how he became the man he is now. Set in a hinterland between the civilized world and the wild, TOM tells the story of one man’s journey to rediscover who he really is.”

Idea

The project Tom has started as a commission from Sadler’s Wells in London where Wilkie is an associate choreographer. The idea was to make a live show with a performer in the playing space with three gauzes up/mid and down the stage, giving a sort of semi-3D world without the need for glasses or anything.

Underpinned by universal themes framed through Wilkie’s personal experiences, the work will question: what do we abandon, lose and hide as we journey into adulthood? What of the loss of innocence and self as these identities are replaced by the fabricated layers of a new character? And where do these parallel identities begin and cease to exist?

Model making

All the models that are presented were created manually by Wilkie. They are based on the storyboard for the film, with reference photos that have been collated to give him something to work from. The building of the actual models varies a lot depending on what it is. The rural landscapes were all made out of layered up polystyrene which was then sculpted with hot wire tools before being covered in a protective coating allowing paint and texture to be added. Finally, they can be painted and have scattered materials added, like small rocks and vegetation. Most of the landscapes took about one day to build and another to paint and finish.

By contrast, the more urban environments such as trains and metro stations are a lot more time-consuming to make. Each one is made out of approximate designs extrapolated from photos using Photoshop. They are then predominantly constructed from plastic (PVC) sheeting, balsa wood and other types of sheet materials. They are all glued and finished with modelling tools to craft the finer details on them. Owing to the more precise nature of the models and level of specific detail they were very slow to make. For example, the train carriage sections took about 2 weeks to complete.

Photogrammetry

As for the photogrammetry, a simple rig is used which essentially consists of a small automated turntable which the models sit on, five continuous photographic lights and a backlit white backdrop.

The camera is mounted on a jib so after each rotation of the turntable it can be quickly moved to a different height or angle to get the coverage of the model. The automated shutter of the camera allows him to continue to build new models whilst the scan is taking place, only having to return to the rig every few minutes to move the camera position. The greatest challenge with the process is trying to get enough light on the model to ensure minimal shadows but also to achieve a fast-enough shutter speed to avoid motion blur from the turntable whilst still maintaining an ISO as low as possible and an aperture of around 11 (to preserve sharpness and avoid distortion and better help capture reality reconstruct).

The number of images per model varies depending on jointly the complexity of the model itself but also the level of detail ultimately needed in the final 3D asset. A small, simple model which is only going to be seen as a background element will typically have about 400-600 images with the most complex models captured going all the way up to 2500 images per model.

Workflow

Once the models are reconstructed and textured a simplified low poly version of the models is generated and exported with the high poly mesh, all from within RealityCapture. Using Xnormal and Knald, normal, ambient occlusion and bump maps can be then baked from the high poly mesh so the final low poly asset can be used in 3D software still with a high level of detail preserved.

Why RealityCapture?

“The decision to use RealityCapture was obvious, given the level of detail and the speed with which I needed models to be turned around. Firstly, with over one hundred models to be scanned there really isn’t another program available that can process that quantity of models in the time frame I have available. In addition to that, the level of quality from the scans is also one of the best in any of the photogrammetry programs currently available, which is remarkable given the relative speed of the reconstructions. The whole point of making models by hand only to turn them into 3D assets is to try and capture the organic hand-crafted nature of them, something which would be difficult to achieve if they were to be modelled directly in 3D but also is dependent on a photogrammetry workflow that can capture the finest detail of this organic quality.” Wilkie Branson

The project will preview in Bournemouth UK in summer 2018 and premier as a part of a large event at Sadler’s wells in the autumn 2018.

About Wilkie Branson

Wilkie Branson (https://www.wilkiebranson.com/) is an interdisciplinary dance artist and film maker. Self-taught in both dance and film, which form the main focus of his work, the roots of his practice lie in B-boying. Wilkie’s dance style has developed into a unique fusion, with expression, accessibility and integrity at its heart. He received the Arts Foundation Choreographic Fellowship in 2012 and has recently been the recipient of several awards for his latest dance animation, Little Dreams. Wilkie is also a New Wave Associate Artist at Sadler's Wells.

TOM Workflow RnD from Wilkie Branson on Vimeo.