A short Essay on Volumetric Filmmaking

May I share how I believe directing and shooting in a digital volume (360, light depth, VR etc) will begin to change how we make films and content, and ultimately tell visual stories.

I’d also like to invite other cinematographers, actors and directors to join me in embracing these new filmmaking techniques, today.

For I believe virtual production and volumetric filmmaking will give birth to a new age in artistic storytelling, but it will in the beginning at least, borrow heavily from seasons in the past, and take us back to another time in cinema history.

Firstly – If you agree, film is all about capturing a performance and managing how it is then retold, you’ll enjoy the future. For I believe virtual storytelling and volumetric capture will focus storytellers on the craft of performance, direction, and story. Here’s my thinking:

Examples of modern editing style in film and American cinema.

Todays modern films use fast paced, rapid cut sequences and a visual pace that bombards audiences with multiple images from multiple angles. This mode of shooting and storytelling excites modern audience senses, its almost like a drug that has become a staple visual diet of most modern audiences around the world.

What I am suggesting means not only filmmakers have to change, but audiences will have to change their expectations too. Here’s why.

360 Volumetric Studios

The New Wave of Volumetric Filmmaking

On the virtual stage, in the volumetric set, we digitise the space and the performance as a whole, into an exact pixel by pixel representation of what is happing (the action). Every ray of light is captured, every motion tracked, (or pre-programmed) performance information is instantly digitised and fed into an on-set computers and prepared for the post-production pipeline.

But for story tellers used to using a camera, here’s the big change, on tomorrows volumetric set there are no directional cameras.

HypeVR Tonaci Tran RED rig

The single lens will be eventually replaced by virtual cameras and on-set by rigs of 360º cameras that capture the entire scene, in one shot, these optical capture technologies are then supported by virtual movement trackers that help to refine the data.

Narrative cameras are operated in a VR engine in virtual space.

At it’s summit, the concept of virtual and volumetric capture allows us to access the capture data to  extract 2D narrative storytelling for exhibition (cinema, TV, phones etc) but it also allows us to create 3D scenes too, for VR applications.

Multiple storytelling formats for multiple devices, from one single performance capture. This is the ultimate dream of developers in volumetric capture.

Today, this concept of virtual filmmaking remains just a concept for most, as the cameras and computing power required to capture in high-fidelity is immense. Even with access to some of the best minds, equipment and computing power, to make anything useful today, we are limited to work with virtual sets that use deep compositing of live action plates to form what could be described as augmented production.

But as anyone familiar with Moore’s Law will know, what’s impossible in computing terms today will become mainstream tomorrow.

What this means for storytelling.

In the first wave of volumetric filmmaking we’ll see a focus on the actors performance, long takes and higher tension. This will be an inevitable result of entering this space now, we have to change how we direct and shoot to make best use of these new tools today.

I can hear you saying that art should never be driven or decided on by the tools. Personally I think thats rubbish, it almost counteracts it’s own argument. But lets look at what drives the opinions of some of the greatest, most awarded folk in cinematography. Roger Deakins has to be my all time favourite DP, he is brave, digital, sensitive, and all the things I want to work with. Here he is asked about what scenes impacted him and why, here’s his answer:

In the volumetric future of filmmaking using directional cameras on-set to capture images will become less common. Instead, in the volume we have the potential to capture environments (sets) and performances (Actors) in one go, then cut as we wish.

Multiple lens frames, multiple angles, multiple everything but with one limitation, data. This will be the driving constraint of artistic choice as we move into volumetric production.

Army of Shadows

But this may not be a bad thing creatively for storytelling. As Deakin’s said, its the performance and the way the director holds you in that room while the other actors decide on how to kill the man on the chair. Its the performance and the tension of the situation, the writing.

So to go forwards we have to go backwards, to a simpler time, a time of tension and story that could be more akin to the stages of the round house, the simple sets of early TV and film, 50’s style of editing, even though these sets will become virtually rendered.

In time, digital doubles will seamlessly switch between optical captured faces, until one day resolutions and computing power will make way for digitally captured pixel performance, where audiences will not be able to tell the difference, and won’t even care if an actor is real of not.

Lets face it, non of us have ever seen a real person on a screen ever anyway, it’s just that light capture makes us believe we have. Soon volumetric capture will do the same.