TL;DR
- Virtual production has evolved from a proof-of-concept technology to an industry standard, revolutionizing film and TV production.
- NAB Show New York will feature a panel discussion, “The Virtual Production Revolution — A Real-Time Love Affair,” led by Jim Rider, Virtual Production Supervisor at Pier59Studio, with KéexFrame founder & CEO Arturo Brena, VP Toolkit founder & CEO Ian Fursa, and ASHER XR founder & CEO Christina Lee Storm.
- Core technologies like camera tracking, LED walls, and Unreal Engine have matured, making virtual production a reliable tool for filmmakers.
- The technology is not just for high-end productions; it’s becoming more accessible and is also being employed in various industries from fashion to automotive.
- The future of virtual production looks promising, with advancements in AI and other technologies set to further integrate and standardize workflows.
Virtual production has rapidly evolved from a nascent technology into a mainstay that’s fundamentally altering the film and television production landscape. But VP isn’t just about the use of LED walls and securing a really great stage; it’s an ever-evolving set of technology solutions and practices.
This month, NAB Show New York will gather a panel of experts who are leveraging new tools to make virtual production their go-to production method. Held October 26, “The Virtual Production Revolution — A Real-Time Love Affair” promises an in-depth exploration of the latest trends and techniques.
Leading the panel is Jim Rider, virtual production supervisor at Pier59Studio, whose years of hands-on experience make him an authority on core VP technologies. He will be joined by KéexFrame founder & CEO Arturo Brena, who previously won an Emmy Award for innovative live-camera tracking and real-time graphics, and has helped conceptualize and lead projects that define the new cutting edge in graphics on-air technology. Rounding out the lineup of industry pros is Ian Fursa, developer and CEO of the VP Toolkit plugin for Unreal Engine and educational workflow series at educational institutions around the world, and ASHER XR founder & CEO Christina Lee Storm, who specializes in the creative development of emerging technologies for linear and multi-platform storytelling.
Ahead of NAB Show New York, the panelists provided a taste of what promises to be a dynamic and insightful session in an exclusive Q&A for NAB Amplify. Watch the full discussion in the video at the top of the page.
A Rapid Evolution
Until recently, virtual production was largely thought to be still in its experimental phase, intriguing but not yet fully integrated into mainstream workflows. “A year ago, two years ago, this was a proof-of-concept technology,” Fursa recalls. “A lot of people didn’t really have a grasp on the technical side of things yet, or how to quite integrate it into their already existing workflows.”
Fast-forward to today, and the landscape has dramatically changed. Rider emphasizes that the core virtual production technologies — camera tracking, LED walls, and real-time graphics engines like Unreal Engine — have matured to the point where they are reliable production tools. “Most of it has been figured out in terms of [being] a production tool.” Alongside color workflows, he says, “the biggest challenge these days is adoption.”
Virtual production, says Rider, has “gone from a, you know, Mandalorian, Disney multimillion-dollar workflow to something that you’re seeing in a lot more stages and a lot more productions that are able to tap into it as a production technique.”
“The genie’s out of the bottle,” says Storm, noting that VP workflows are more accessible than ever. Currently she’s seeing virtual production employed primarily for pre-production, “whether that’d be real-time previs, cinematics, or real-time animation,” although she expects that to open up even more, “and that’s pretty exciting.”
But it’s not just about the shiny new tools; it’s also about how they fit into existing pipelines. “Companies are trying to really leverage real-time pipelines,” she says.
The shift from experimental to essential has been rapid, but there is still much ground to cover, as Fursa notes. “All of the different industries have now started building their own pipelines to utilize this technology. So now we learn how to tell stories with it better, we learn how to refine how those stories are told and what tools are built,” he says. The current push for standardization is another sign that virtual production is maturing. “Now is the time where everybody’s building things and creating what everybody wants: an industry standard.”
Advances in Image-Based Lighting and LED Technology
When it comes to the tech that’s currently driving the virtual production revolution, Fursa and Rider are particularly excited about advancements in image-based lighting and LED technology.
Image-based lighting “really goes hand in hand with virtual production,” Rider explains. “One of the things we always tout is that we can put up multiple environments on the wall, multiple locations in one day. But the ability to actually relate those by doing pixel mapping onto full-spectrum LED light fixtures on set — not only are we changing the contents on the LED wall, but we’re changing the lighting at the same time because that lighting is being mapped to these LED fixtures.”
Other advancements in LED technology include physical chips designed to enhance performance and image quality. “Before [now] LED walls did not render accurate lighting for talent and for real props,” which became a real problem during filming, says Fursa. “It makes it so that we can’t have an explosion that is the accurate color on our subjects. So there’s been a lot of advancements in the development of the physical chips and technology for us to be able to start using LED walls as an actual film production light source.”
What’s Next for Virtual Production?
In the near future, Fursa predicts, virtual production will be employed not just by “high-end series and film productions, but smaller-budget commercial and music videos.”
And as workflows begin to integrate with artificial intelligence, toolsets will become “very powerful,” he says, allowing things like background replacements to become routine automated tasks. Studios, he says, will continue to focus on refinement and standardization in the near-term. Yet “there’s still a lot of benefit” to separating production from in-camera VFX, “because there’s physics there that we still haven’t figured out how to emulate,” he adds.
Brena also sees “a massive grand-scale adoption from the entertainment industry” in the short term, leading to extensive pipeline and workflow modifications and even new ways of storytelling.
“So either from interactive storytelling, from broadcasts, for example, the idea of having things that have been, right now, just eye-candy, they can now be editorial, right? Because you can change our concept of the content in real time.”
As virtual production technologies become more sophisticated, they’re also opening up new avenues for monetization. Looking further out, Brena anticipates that in-device rendering, which will allow consumers to render content on their local devices, “will allow for new ways of not only storytelling, but also new multiplying streams of revenue for advertisers,” he says, a development that’s “going to be very attractive for advertisers.”
Virtual production’s influence isn’t limited to Media & Entertainment, and as VP technologies continue to evolve, their application across various industries will likely broaden, creating new opportunities and challenges. “This is a cross-industry effort,” says Brena. “We see people from fashion, automotive, film, and broadcast collaborating together with these new technologies.”
Generative AI adds a new layer of potential efficiencies for collaboration, Storm adds. “With generative AI, creators are able to see, at least visually, their concepts and ideas,” she says. “You could do that in a quick iteration that cuts out some of that process… that gives a little bit more time in the front end to just work through the kinks, work through, like, ‘Are we on the on the same page creatively?’”
The production industry will continue to combine technologies like GenAI or even volumetric motion capture with virtual production, she says. “More and more, every year, we’re going to see that evolution as well as the new iterations on workflows.”
New at NAB Show New York: Photo+Video Lab
