How To Create A Virtual Set

How to create the nearly realistic virtual sets using a 360 camera assortment

Virtual Sets & Virtual Studios – Revolutionizing Film Production

If y'all've seen the Mandalorian and you've wondered why it looked Then REAL, then you have come up to the correct spot. Learn how the Mandalorian utilized a breakout engineering – an in-photographic camera virtual prepare – to bring the nigh life-like experience to viewers effectually the globe.

The Evolution of Filming and Use of Screens

To fully appreciate how far we've come since the invention of film and the flick industry (just over a century old), let'due south have a look at the evolution of film, in item – how directors created backgrounds when they were non able to movie the scene live.

Single screen backside an actor

Remember those horribly fake looking backgrounds in EVERY driving scene in movies? The actors always moved the steering cycle in such a jerky way while the background stayed in the same viewpoint? Who drives like that anyway?!
Problems with unmarried screen: If the director wanted to move the camera angle during the scene the projected footage couldn't move with the photographic camera, then directors were forced to work with 1, and simply one, bending when filming a chase scene.

Example of a driver in an old movie with a single screen behind him (James Bond, 007) - the beginning of filming. — Image credit: Den of Geek

Green screens or Chroma Keying

A literal green screen is positioned behind the role player(s). Due to the nature of the color green and the way it is captured on picture, it is possible to film a bare background and subsequently digitally remove or "central out" that colour, dropping that scene onto the background of choice in post-production.

The evolution of screens in film making. The green screen or chroma keying is when an actor is filmed in front of a green screen and later image details are added to the screen. — image credit: Pond 5

Problem with Dark-green Screens: actors have to imagine their environs. Just recall about how difficult information technology must accept been to film a whole fight scene from the Avengers movies in which the alien invaders were falling from the heaven. The actors had to visual all of that in their heads and react accordingly. (I know action films often get a lot of heat for the less than stellar acting – but possibly they're better than we recall if they tin can brand us believe they are fighting armies of invaders when in fact they are surrounded past brilliant neon walls.)

An example of green screen or chroma key in the filming of Avengers: Endgame in a battle sceen.

LED screens/wall technology

Digitally created or real-live footage is displayed on a large drove of LED screens, normally in a semi-circle germination, creating a wall and ceiling
Benefits of LED screens: actors can see their environs on the screens in the appropriate fourth dimension and react accordingly

On set with the latest in-camera VFX feature set from Epic Games' new film — Image credit: Befores and Afters

No, that is not your eyes deceiving you – that is a virtual prepare. The walls and ceiling are dozens upon dozens of LED screens fitted together and displaying an incredibly clear, true-to-life environs for the actors to 'play' in front of.

With the touch on of a few buttons, the managing director tin change that gorgeous twilight sky into a fiercely bright midday blaze.

What is the difference between a set and a virtual set (or virtual studio)?

A virtual prepare is a drove of several extremely novel technologies which project a life-like environment onto LED screens located backside actors while filming a scene for a movie or Television show (these are the two biggest uses at present, but theoretically whatever sort of picture production – including commercials, news, etc will nigh probable benefit from this technology in the not to afar future).

A virtual studio allows for existent objects and real people (actors) to seamlessly interact in existent-time with computer-generated environments and objects.

What is virtual production?

The central element of a virtual prepare comes in the way that the real-life camera that is on set can move in 3D space, while at the verbal same time, the image coming from the virtual camera is being rendered from the same perspective. The different camera settings – pan, angle, zoom, traveling – can adapt at any time to be in tune and in line with the action of the actors.

Dissimilar the other technologies used until now, virtual production is done in existent-time and does non require mail-product work.

How Virtual Sets Will Dominate Previous Filming Solution

Virtual sets are already becoming the norm in many loftier-end film productions due to the many benefits and very few drawbacks.

The benefits of using a virtual prepare

Motion-tracking creates hyper-realistic scenes
- Using Unreal Engine (a 3D video game "engine"), artists can create a photorealistic 3D background that moves strictly with the photographic camera'due south field of view (aka: frustum), so if the photographic camera swings around and changes angles, the groundwork shifts in precisely the same way. This allows motion-tracked cameras to execute traditional cinematography techniques within the virtual set, achieving cinematic movements like parallax effect, (object in the foreground moves at a different speed than the background). This amplifies the illusion of filming at an bodily location.
Solves the problem of lighting
- Light coming from LEDs provide realistic colors and reflections on the actors and props (non possible with green screen)
Removes greenish spill
- This refers to the green light that is reflected onto the actors and props because of all of the green screens, giving everything a greenish glow.
Big impact on fourth dimension and budgets
- Since almost of the virtual set can exist bars to a detail area, less space is needed overall, thereby limiting the amount of stage space necessary in filming/shooting
- Developing and building the LED screens is definitely expensive the start time, simply the initial investment will soon make upward for information technology past the savings on traveling to locations, building new sets, and costly mail service-production
Customization of the features
- Artists and directors can make changes and control the world on the day of shooting
- Exposure, color, animation playback, and fill lighting are bachelor
- Can move a mountain with the printing of a finger
Nevertheless able to use light-green screens within an LED wall
- It is nonetheless possible to use a specific portion of the LED and transform into a light-green screen.
  - Ex: Can remove people and add additional action behind them
- This is due in part to the fact that some special effects are still best kept at traditional methods (prevents damage on the LED screens)
Speed of setup
- Rather than traveling to different locations or setting up in-studio sets with existent objects, setting upwardly a virtual gear up is much quicker. As well, when working inside a normal set up, changing the lighting tin can add together meaning time to each shot as cranes and lights have to be moved effectually and adjusted oftentimes.

How to create virtual sets

Let's wait at the technologies every bit well equally the steps and the people needed to create a virtual fix.

Technologies necessary for creating a virtual set

There be many technical solutions for creating virtual sets and studios, only well-nigh of them include the following components:

Optical or mechanical measurements are used in photographic camera tracking to create a alive stream of data describing the exact perspective of the camera.

Real time rendering software uses the camera tracking data and generates a synthetic image of a idiot box studio.

A video mixer combines the video from the camera with the video from the existent fourth dimension rendering software to produce a concluding video output. Replacing a chroma fundamental background is one of the most common ways to mix the video.

Steps to creating and using virtual sets

While the actual steps and processes involved are much more complicated then what can be covered here, nosotros volition focus on why this affects Mosaic. For a more thorough agreement of the ins and outs of creating a virtual prepare, check out this incredible article here. We will stick to what we know – data capture – and why nosotros're so excited for the applications of our Mosaic cameras and mobile mapping services for something equally exciting and innovative as in-camera VFX on virtual sets.

Hopefully at this point we all understand that a virtual set is basically the way in which actors can film in front of a apartment backdrop but feel as if they are in an actual, real scene. The imagery which is displayed on those LED screens has to come from somewhere though, right?

Who is behind creating virtual sets?

There are several methods of achieving the images to exist displayed on virtual set up LED screens.

Option A: 3D graphic artists and 3D computer graphics software
Selection B: capture real-life images (and generate complete 3D models of big scale scenes) with the highest resolution photographic camera array on the market today.

This is where we go involved and why we're and so amped upward! The images portrayed on these screens have for now been created through the hard piece of work of 3D graphic artists and 3D reckoner graphics software who create virtual backgrounds and all of the bells and whistles that get along with making a scene look real.

The Mandalorian in particular used images and environments sourced from Unreal Engine – which is basically a library for super avant-garde, super real-fourth dimension 3D creations. Up until at present, this applied science has been used generally in the gaming manufacture for those nearly-too realistic first player shooter games.

But Mosaic cameras, with the highest resolution photographic camera array in the world, are capable of collecting 3D environments at a much quicker and efficient manner than 3D graphic artists tin create them.

With image capture abilities from a camera array which is expertly synchronized to collect a full 360-degree view of the surrounding area, film production has a brand new tool at their disposal. They can effortlessly gather real-life information in nearly no time at all, thereby reducing fourth dimension and resources significantly.

Check out some of these videos captured by the Mosaic Viking – the earth'southward highest resolution camera.

Check out this footage from Mosaic extreme data capture.

The future of film production

Certainly, 3D artists, VFX specialists and complementary software will be necessary when creating the content for films or shows which are fix in unreal places. For situations such as those, the virtual environments will need to be carefully crafted to run across the demands of the directors, producers and writers.

Nonetheless, for scenes which are set in real-life places – downtown Hong Kong, the middle of Santiago Bernabéu Stadium (Real Madrid), or the depths of the Amazon woods – in that location will be a much faster, easier and cost-efficient manner in which the epitome data is collected. If y'all are interested in more information about the Mosaic line of cameras or our data collection services, feel free to drop united states a line below.