On-Set Facilities
Unreal Engine Virtual Production

First Look at Virtual Production Features in Unreal Engine 5.1.0

Unreal Engine Virtual Production
Unreal Engine Virtual Production 5.1.0

With Unreal Engine 5.1.0 to be released sometime during the last quarter this year, the OSF team took a first look at the UE5.1.0 development roadmap, to bring you this round up of the upcoming UE5 features coming to OSF’s Unreal Engine virtual production pipelines.

(In this article we’ll be covering some of the key developments for virtual production. See the full upcoming feature documentation here)

On-Set Facilities are an industry leading Unreal Engine Virtual Production Studio that deploy and manage UE VP pipelines for some of the worlds top content studios and production companies.

nDisplay Clustered Rendering Enhancements

SMPTE 2110 – Unreal Engine 5.1.0 will support SMPTE 2110, enabling different hardware configuration for nDisplay systems where each inner frustum can be rendered by a dedicated LED node. Providing stages and volumes with improved performance, scalability and reliable content.

Quick Local Launch – nDisplay will now offer a simplified quick launch ability for local configurations. The solution will behave similar to the switchboard utility with set variables and settings but greatly simplifying the local machine workflow experience.

Input and Output Mapping System – An improved workflow enabling users to specify arbitrary time-synchronous pixel sources and destinations to ICVFX cameras, viewports, and the application windows. Now in addition to the native UE 3D scene render, users can now also register IP video sources such as SMPTE 2110, media textures, or live-generated data from hardware capture devices to nDisplay viewports.

Export of nDisplay Renders via Movie Render Queue – The nDisplay configuration file reference is now available to Movie Render Queue so that users can render selected nDisplay viewports to disk for high-fidelity offline or backup content playback. The recorded nDisplay viewports support recompression to arbitrary codecs or exporting formats with necessary bit depth for plate playback.

GODBOX Virtual Production Computer

VP Stage Operator UX & Workflows

2D Light Card Placement and Management UI – Light cards are a essential tool to a majority of LED volumes, Epic has optimized stage operators workflows with the creation of dedicated panel for the creation, placement, and management of all light cards that uses a simple click-and-drag UX over the top of a 2D render of the LED walls. Designed around the key goals of intuitive use and fast operation.

Viewport and Region-Based Color Correction UI – Unreal Engine 5.1.0 adds a dedicated panel for a larger, more intuitive color correction experience. Enabling both finer control and faster access, answering the need for both accuracy and efficiency on-set.

ICVFX iOS Stage App

iOS Stage App – The details of what the ICVFX iOS stage app will specifically do are still misty but it’s confirmed that it we provide UX controls for color corrections, light cards, and nDiplay management tasks. Whilst working in conjunction with the existing web remote control system.

Lumen for In-Camera VFX

Lumen for ICVFXLumen will now be fully integrated and available within In-Camera VFX workflows. Providing Cinematographers with Interactive lighting and ability to animate light sources with GI contributions, in real-time without the need of pre-baked lighting.

Video & EXR Playback Improvements

EXR Tiling and MIPs Support – it’s essential within LED virtual productions to show playback plates at the highest possible resolutions and colour depths. In order to balance the rendering load across nodes and server bandwidth limitations Epic has added EXR tiling and mipmaps support.

Supported EXR Tiling and MIPs Workflows:

  • In-engine, multi-threaded EXR tiling and MIPs generation tool. This also enables users to colour the different MIPs levels for debugging.
  • Ability to stream and decode only visible tiles per rendered viewport.
  • Tiling and mipmap support for custom EXR image file formats.

Dedicated Media Plate Actor for Media Management – The new Media Plate Actor provides users with a streamed lined playback workflow for both cluster and non-cluster rendering. The Media plate actor consolidates both 2D plate and 360 latlong playback formats with EXR tiles and MIPs support.

The new media Architecture brings a multitude of benefits to users, that include:

  • Greatly simplified video import process
  • Support for Media Playlists
  • Drag and drop support for both the suggested assets and Actor
  • Updated Info and Preview windows
  • Sequencer integration
  • View frustum culling for improved performance
  • Media Player and Texture Manager

 

Unreal Engine Virtual Camera, shooting fully virtual real-time 3D.

Virtual Camera Enhancements

VCam UX – The VCam app UI has been updated to a more camera-focused paradigm with extra usabilty design for IOS and Android devices. Built on top of a customizable system, it opens the possibilities to support other workflows in the future like light placement, set dressing, and 2D scouting.

Vcam Performance – The VCam system has been updated to use VP9 streaming with WebRTC for lower latency and higher fidelity. This will improve both the transmission of images as well as the ARKit tracking data.

VCam Operator Workflows – The VCam updates allow for operator improvements on both the Unreal Engine UX and the integration of hardware controls. Operators will be able to easily:

  • Control Near Clipping Plane on CineCameraActors
  • Perform time-scaled recording through Take Recorder with full-speed playback
  • Maintain references to spawnable
  • Perform layered records for dollies, splines, camera passes, and Actor tracking

Virtual Scouting Enhancements

Virtual Scouting Update – The Virtual Scouting update in 5.1.0 works more of a pre-update for future waves of development. Moving away from the previous platform specific code, onto a new foundation built around OpenXR API.

Future updates to virtual scouting will focus on the development of role-specific toolsets, production design, lighting, animation review and the integration of Lumen and Nanite.

Full Unreal Engine 5.1.0 Feature List

Unreal Engine has heaps of new features being released in 5.1.0 spanning across multiply industries from automotive design to real-time animation. Check out all the latest features coming to Unreal Engine 5.1.0 here. Contact On-Set Facilities who deploy, manage and operate the full spectrum of Unreal Engine Virtual Production pipelines and solutions, on-set and in the cloud, at https://onsetfacilities.com.