Spatial Computing Artwork

Hugues Bruyere is a creative technologist based in Montreal. He has been creating interactive new media art and experiences for the past 15 years, lately with the focus on Spatial Computing (VR, AR, MR). He is also a partner at Dpt, an award-winning studio specialized in immersive and narrative experience (working at the intersection of film and gaming, art and code, design and research.)

How did you get the idea to create such a specific artwork? Why AR?

My foundational process is about technological and creative exploration. I love this element of surprise! Building on this, I have always been interested in creating interactive experiences, ideally immersive and allowing full-body interaction. Experiences that blur the boundaries between the physical and digital world. Lately, I have really enjoyed capturing people and creating bite-size interactive AR experiments using their figure, as a means to explore 3D scanning, a relation between the physical and the digital world.

What inspires you in your artwork and creating different effects?

Some AR pieces came from pure exploration and others were inspired by different artists, like for example “Sliced Arnaud” was inspired by the work of Julian Voss-Andreae.

Subsequent work was driven by my desire to push the relationship between the position of the user in the physical world and the subject in the virtual space. I would ask myself, ‘How can I “break“ the physical world? How can I use occlusion effects in order to accentuate the sensation of immersion?’

How does photogrammetry help you in your work?

Photogrammetry allows me to easily bring the real world into interactive 3D environments, it can be objects, locations or people. I am able to interact with the scanned subject in ways that may not be possible in the physical world... In a way, photogrammetry gives me superpowers, haha.

I also really like the process of manually capturing the location or a person, it’s almost a meditative and relaxing process.

Arnaud [Credit: © Hugues Bruyere]

Why you like to work with RealityCapture as photogrammetry software?

I quickly understood RealityCapture is the best solution available. It’s fast and reliable and can be used on any machine. The fact that there is a floating license available on Steam was also a really attractive no brainer for me. All these allow me to easily work from several locations and from different computers.

What other tools do you usually use for your artwork?

The tools vary based on the project’s needs. For the capture process “manually” (ex. I do not have a rig with 50 photo cameras to do full-body captures), and I never have much time to do it. I also try to keep the cost down to a reasonable amount. I have experimented with both photo and video entry-level cameras.

When I work with a point cloud of big spaces, I tend to use both photo and video cameras simultaneously. I then do a reconstruction and simplification in RealityCapture, and sometimes I work with the point cloud in tools such as MeshLab or CloudCompare. Once the data has been cleaned and subsampled, I move to Unity in order to manipulate, treat, and render the scan in real-time.

Monastiraki, a continually shifting curiosity shop and art gallery established in 1998 and located in the heart of Mile End[Credit: © Hugues Bruyere]

When I work with face or full-body scans, I reconstruct the photos inside of RealityCapture. Then I use a mix of ZBrush, Blender, and Substance Painter to do the retopology, normal, and occlusion maps baking and texturing work. When the 3D model is ready, I use Unity again to build the custom interactive applications.

Also, I should say that I am lucky enough to collaborate with a team of experts and creative people at Dpt – 3D artists, Unity and Unreal developers, etc. all this feeds my passion and interests.

Could you tell us a little bit more about some of your projects you did?

I recently worked on two main explorative projects, one called Chasing Ghosts and the other being a series of small AR iterations.

Chasing Ghosts, is a series of discrete landmarks found in Montreal – these landmarks are capable of modifying their properties in order to be able to differentiate themselves from their surrounding environment. Chasing Ghosts is a work in progress driven by the stories these places tell. Ultimately, the project will be a series of real-time interactive projections and a VR experience.

Left: Presstube HQ, the studio of the amazing James Patterson. Right: A typical Montreal alleyway.[Credit: © Hugues Bruyere]

AR experiments, is a series of bite size AR explorations. In this series, I quite like the iteration where I am playing with transparency and wireframes. I want to demonstrate how there are several contradicting forces at play: by building a monolithic shape that contains an entity, a mass, by using a transparent material to create a sense of fragility of the object, and by applying wireframes to accentuate the shards and the edges. I want to express an interplay between soft and hard, heavy and light, visible and invisible.

At the studio, we are currently working on several amazing VR, AR and MR projects, including 3D volumetric scans, so stay tuned.