RealityCapture and Project JUA

Worldwide, 1.3 billion people lack access to electricity. Many people in Africa are reliant on dirty kerosene to power their households. This form of energy is expensive, unsafe, and can cause health problems.

PROJECT JUA is an interactive virtual reality experience that raises awareness of these issues and builds empathy through a new storytelling medium by taking the player to a rural farm in Malooni, Kenya where they can experience firsthand the challenges of living off the grid and discover an African solar power solution that is rapidly improving lives.

IDEA

The concept for the project came from the team behind “Hack for Africa” at Microsoft in Redmond, Washington. Their mission is to accelerate digital transformation in Africa while empowering people and organizations to achieve their full potential. The Hack for Africa team wanted to build empathy for the challenges and opportunities surrounding affordable solar power in Africa and knew instinctively that when compared to traditional media, interactive VR, when designed well, has the ability to leave a lasting and emotional impression on those that experience it.

We at Anotherworld VR were excited by the chance to combine awareness and innovative storytelling and humbled when we were given the green light to create the experience. We were chosen because of the combination of photogrammetry, cinematic lighting, intuitive programming and storytelling that we create for all of our VR projects in order to achieve the highest level of realism, immersion and emotional engagement. We took the challenge, knowing we would have to turn around the project in under 6 weeks.

We were briefed that the site would be disconnected from electricity so we packed portable battery-operated gear and kept the gadgetry to an absolute minimum. But nothing would prepare us for how remote the location really was. The winding red soil road that twisted through villages, acacia woodlands and grasslands took us deep into Machakos County and it was only when the road ended and we found ourselves driving through a dense forest of banana trees that our host family came to greet us and we knew we had arrived at our location.

PHOTOGRAMMETRY

The farmstead was built from mud bricks with only the occasional window allowing natural light to come in, so we lit the interior areas with battery powered LED lights, which we positioned to cast a flat, even light in order to avoid unwanted shadows creeping into our textures. When it came to the exterior areas we knew we would have to take as many pictures as we could before the sun went down, so we got the basic coverage from a UAV while the ground team focused on details by snapping high detail photos of every nook and cranny using light sensitive DSLR cameras and even smartphones on selfie sticks to get into those hard to reach places.

We had barely finished when the sunset and everything fell into total and complete darkness. Our batteries lasted just long enough for us to gather our gear, say our goodbyes and make our way back to the car. It was only while driving back that we noticed that our car headlights were the only source of light for the entire journey. The villages we had passed before were now shrouded in the deep, dark night with only the occasional light bulb flickering from a storefront. Even more striking was the sight of women returning from the market, often completely alone, walking down anonymous roads through total darkness.

That night, we loaded our photos from day one into RealityCapture running on a laptop and aligned images using the preview mode. By the next morning, we had a point cloud that clearly showed us which areas we had covered well and which areas we needed to take more photos of. Preview mode is incredibly useful in the field but what was particularly impressive was the sheer volume of photos that we aligned and how quickly RealityCapture gave us a result.

When we returned to the farmstead on day two we interviewed our host family about the solar panel they had installed on their roof. They told us that they had become customers of a Kenyan company called M-Kopa Solar, which offers a solar power kit that comes with a rechargeable flashlight and can power three household light bulbs, a television, a radio and a hub to charge 5 volt USB devices like mobile phones on a single day’s charge of sunlight. They told us how these basic amenities had changed their lives, how they felt more connected to the outside world, how their kids could study in the evening without inhaling toxic kerosene smoke and how the trip home from the market was less daunting with a flashlight.

When the Hack for Africa Team at Microsoft heard about these conversations, a clear idea about how the VR experience would unfold began to materialize. The idea: step into the shoes of a farmer on his way home from the market, when he receives a phone call from his wife, reminding him to close the chicken coop before nightfall and to place a kerosene lamp into the living room so the kids can do their homework when it gets dark. We knew that these simple tasks would be tricky to complete in VR while navigating through darkness and it was exactly this burden that we wanted to convey and contrast with the benefits of the M-Kopa Solar power kit and how it was empowering even the lowest income households in Africa to take solar power into their own hands.

BUILDING THE VR EXPERIENCE

By the time we got back to Berlin the deadline to complete the VR experience was three weeks away, so the reason for choosing RealityCapture was clear. Despite the powerful workstations we dedicated to the project, we knew from previous tests that no photogrammetry software other than RealityCapture would be able to deliver beautiful results from such a large amount of data in such a short amount of time.

So we aligned all of our images, over 27,000 in total, starting with the components of the interior locations, which we linked to the exterior components using control points until we had large pieces of the entire farm ready to generate in high detail. Using the reconstruction region tool we then separated the farm into pieces and divided these among a team of 3D artists who began retopologizing the meshes and optimizing them for performance in the Unreal 4 games engine.

Meanwhile, programmers and VR artists began building the VR mechanics, level design, interactions, and gameplay, racing against the clock to complete a fully functional demo version in time for the 2017 Microsoft //Oneweek Hackathon in Redmond, Washington.

Over the course of four days, hundreds of engineers, managers and executives took the digital trip to Kenya, stepping out of their high-tech lives to experience life on the other side of the planet, often responding emotionally to a topic they had not previously considered or even taken for granted.

As one respondent put it: “You know how people say a picture is worth a thousand words? I think virtual reality is worth millions of pictures. It’s just another level because you understand better, you get to live it, you don’t only see a two dimensional thing, you get to live it and feel it. It’s great!”

CREDIT INFORMATION

PROJECT JUA is a collaboration between Anotherworld VR GmbH, M-Kopa Solar Ltd. and the Microsoft Corporation and would go on to win 3rd Place in the Judson Althoff Challenge and Microsoft Employees Award

ANOTHERWORLD VR is a VR startup from Berlin made up of writers, producers, game developers, 3D artists and culture buffs that have a background in filmmaking and have taken the leap into virtual reality content production.. www.anotherworldvr.com

M-Kopa Solar is a Kenyan company whose mission is to upgrade lives by making high-quality solar power solutions affordable to the lowest income households in Kenya, Tanzania and Uganda. www.m-kopa.com

Microsoft works with M-KOPA Solar to provide kerosene-free lighting and productive assets through its solar and mobile payment system. With the use of the sun, mobile phones, and Microsoft Azure the lives of people in Africa are improving while connecting them to the world.

From CR Team

We would like to say special thanks to Mr Max Sacker and whole ANOTHERWORLD VR team, who have kindly provided us with the information and this brilliant article. We are very glad that RealityCapture was chosen for this amazing project.