Hollywood Co-Stars: oneAPI and Open Source Technology

Before-and-After imagery shows how Intel^® Open Image Denoise, an open source library that is part of the Intel® oneAPI Rendering Toolkit, removed noise from scenes in The Addams Family 2 for an attractive motion picture experience. The Open Image Denoise API is part of the open oneAPI industry specification.  Image courtesy: Cinesite.

The motion picture industry then:  At the start, Thomas Edison controlled the patents on everything needed to make and show movies: cameras, film stock, projectors.  To produce a movie, or project one in a theater, companies had to pay Edison fees to use his proprietary technology.  Those that refused would be taken to court.  Some even received an on-set visit by mob enforcers sent from Edison’s Motion Picture Patents Company in New Jersey.

The motion picture industry now:  Today, a spirit of cooperation continues to transform movie-making in Hollywood.  Among the factors enabling this revolution include the open oneAPI initiative – which streamlines the production of special effects – along with a philosophic change among movie producers, technicians and artists, as well as the technology companies that serve them.  The industry has reinvented itself from the one that Edison had started.

oneAPI is an open, unified and cross-architecture programming model for CPUs and accelerator architectures.  Its standards-based model simplifies software development, enables legacy code integration, and delivers uncompromised performance for accelerated compute without proprietary lock-in.  With oneAPI, developers can choose the best architecture for their solution without rewriting software for the next architecture and platform.

The Plot – Reduce Resources, Increase Efficiency, Create Opportunity for Art

Digital animation and special effects production for movies and television shows is compute intensive, requiring groups of networked computers known as render farms to crank out scenes.  The time and electricity needed to produce them can strain even big budget Hollywood blockbusters.  Therefore, creating ways to lower the number of resources and production time needed to deliver special effects has driven Hollywood producers toward open source tools where possible to enable studios and supporting commercial applications to focus energy on their software’s unique differentiation.  Open source software that solves cross-company, cross-industry needs is on the rise.

According to Jim Jeffers, Sr. Principal Engineer, Sr. Director, Advanced Rendering and Visualisation at Intel Corporation, as well as, a board member of the Academy Software Foundation (aswf.io), “Open source enables the film community to jointly and efficiently participate in ensuring key software tools are always available, can be updated quickly to deliver new feature functionality without waiting for the necessary release cycle of high quality commercial software.”

Building an infrastructure of advanced tools that everyone can use modernizes the motion picture industry.  “At Intel, our open source tools act as an accelerator that provides efficiency to make things better, faster and higher fidelity,” said Jeffers, “while allowing individual studios to maintain their style.  That’s why a Pixar movie, a DreamWorks TV show or film, a Cinesite production and anything from Illumination Studios has its own unique look and feel,” Jeffers added.

To use a metaphor: open source animation tools are like a power saw that can accept different, tunable blades.  “The ability to customize is what makes this ‘power saw’ so amazing,” said Jeffers.  “The result is unheard-of efficiency that was not possible ten years ago.”

Let the Processors be Co-Stars

Developing a standard that enables CPUs and GPUs to run rendering special effects software is what enables oneAPI to provide value.  “The movie industry really gets the vision and value of oneAPI,” said Jeffers.  “By allowing for a heterogenous, cross-platform and cross-vendor environment, it provides exactly what the market wants, more choices and efficiency.”

That’s why Intel and the oneAPI community are working together to create an integrated open platform that supports the use of both CPUs and GPUs for computer animation and to get what’s best out of each processor.  “This increases creativity and reduces the amount of rendering, so compute becomes an accelerator, not a burden,” said Jeffers.

The Need to Eliminate Visual Noise from Computer Animation

Even films that don’t look like special effects extravaganzas often use computer animation and physically based rendering via a technique known as path tracing to create scenes that would be too expensive or impossible to stage in front of the camera.  The ray tracing process is multi-pass and cumulatively improves the image during which telltale signs of noise, or the appearance of grain and artifacts that distract from the motion picture experience, are visible in the frames being rendered.

The traditional method is to simply wait, sometimes for many hours of accumulated processing, for the ideal, high-fidelity pixels to be rendered to a point known as ‘convergence’.  A denoiser is used to reduce the number of passes over the image, by intelligently “filling in the blanks’ to create a very close approximation of a fully converged image that is not distinguishable by a typical viewer.  Therefore, a ‘high fidelity” denoiser saves computation, time and money.  This enables the filmmakers to focus on critical aspects of the film process, like the story.

To help creators, scientists, engineers, and developers develop amazing studio animation and visual effects or to create scientific and industrial visualizations, Intel created a set of premier ray tracing and rendering open source libraries.  These components are available as part of a bundled product called Intel^® oneAPI Rendering Toolkit.

The libraries include:  Intel^® Open Image Denoise, which provides high-performance, high-quality denoising filters. The academy technical award winning Intel^® Embree, a collection of geometric ray tracing kernels that helps rendering engineers improve the performance of their photorealistic applications, Intel^® Open Volume Kernel Library, a collection of 3D volumetric kernels for fog, clouds, fire and other volume rendered effects, and Intel^® OSPRay, which provides an easy to use middleware API that packages all the capabilities of the libraries supporting visualization performance and fidelity scalably across CPU and GPU computational units, including networked computing clusters.

The photo below is a raw image from the Disney computer-animated film, Moana, and part of a dataset made publicly available by Walt Disney Animation Studios for testing purposes.

Note the roughness in the pixels, particularly in the water and vegetation on the left side.

As you move the cursor over the image right to left, you’ll see the “after”, now rendered at eight samples per pixel with Intel OSPRay and denoised with Intel Open Image Denoise, using prefiltered albedo and normal buffers.  Areas in the water and vegetation that previously looked rough and noisy have taken on a clean and more photorealistic appearance.  Intel Open Image Denoise uses machine learning algorithms to refine and clean the images.

Intel Open Image Denoise is AI-based and runs on CPUs and GPUs. Prior generation, image filtering based denoisers often were slower and less accurate.  AI denoising can occur nearly instantaneously with higher fidelity, enabling artists to fast-track the workload and accelerate production.  “Without a high quality denoiser, it can take 8 to 24 hours to render one frame of a movie,” said Jeffers.  “Open Image Denoise lets the computer artist choose how long they want to spend on render passes versus denoising, so they can get the right balance between image quality and rendering time, enabling more film refinement iterations to the director so the vision of the movie can be better met within budget” he added.

How Hollywood Professionals, Intel and Autodesk Worked Together

Cinesite, the company that created The Addams Family 2, was using Arnold, a popular render software made by Autodesk, for special effects and animation, along with Gaffer, an open-source, node-based VFX application used to build, modify and render scenes.

Arnold, which has its own denoise functionality, had not incorporated Intel Open Image Denoise.  However, the filmmakers wanted to use the Intel product on Arnold, due to its ability to produce more brilliant, vivid images that supported the look and feel of the film.  To make that possible, Intel worked with Autodesk to integrate Intel Open Image Denoise into Arnold.  This allowed The Addams Family 2 to be rendered and denoised in record time, without sacrificing quality or spending an inordinate amount in computer processing.

The Addams Family 2 Co-director, Laura Brousseau, and Head of Lighting and Composition, Kenny Chang, explain about how they used Intel oneAPI libraries and other tools at Cinesite to render and denoise the images in The Addams Family 2.  According to Chang, they used Intel Open Image Denoise on every shot in the movie, delivering a 10-25% increase in rendering efficiency. “We saved thousands of hours in rendering production,” said Chang.

The creepy and kooky come to life in The Addams Family 2 with brilliant imagery and photorealism powered by Intel^® Xeon^® processors and multiple rendering tools including Gaffer, Autodesk Arnold, and Intel^® Open Image Denoise.

With its machine-learning algorithms and efficient deep-learning-based denoising filters, Intel Open Image Denoise removed noise from the images in The Addams Family 2.  Through machine learning training, it learned to differentiate the good (geometric shapes, textural detail and patterns) from the bad (render and pixel noise) and remove the latter.

The Intel Open Image Denoise process produced brilliant scenes, while significantly reducing the rendering time on every shot.  At an average of 175 shots per week, run in parallel and 24/7 on the render farm, machine-power time was reduced by more than 500 hours per week throughout the production.

The Market’s Never Wrong, and Hollywood is a Marketplace

The idea that Hollywood studios would cooperate around open source code and standards comes down to business sense.  The Academy of Motion Picture Arts and Sciences (AMPAS) – the same organization behind the Oscars^® – conducted a two-year survey into the use of open source software across the motion picture industry.

Nearly 84% of the industry uses open source software the survey showed, predominantly for animation and visual effects, but it also uncovered challenges.  These included siloed development, managing multiple versions of open source libraries and varying governance and licensing models that must be overcome to create a vigorous open source community.

The survey led to AMPAS forming the Academy Software Foundation (ASWF) to help overcome obstacles to production around open source software.  Since then, the ASWF has built a solid community and a set of projects that align with its mission of increasing the quality and quantity of contributions to the industry’s open source software base, while providing a:

• Neutral forum to coordinate cross-project efforts
• Common build and test infrastructure
• Clear path to participation in advancing our open source ecosystem

The Academy Software Foundation show reel demonstrates how the open source movement has transformed movie making, enabling studios to produce more exciting animation and special effects with fewer resources.

A Real Hollywood Ending

In the early days of the motion picture industry, technology played a major role.  Today, perhaps, even more so.  The greatest change between the birth of Hollywood and the present may be how studios and organizations now work together.

By committing themselves to open source software, Hollywood producers can increase their investment in the artistic side of movie making.  In this way, they’re reducing costs and enabling the telling of new stories in ways that were once technically and economically impossible.

Get Involved and Review the oneAPI Specification

Learn about the latest oneAPI updates, industry initiative and news.  Check out our videos and podcasts.  Visit our GitHub repo – review the spec and give feedback or join the conversation happening now on our Discord channel.  Then get inspired, network with peers and participate in oneAPI events.

Legal Disclosures

• Testing Date: Results are based on data conducted by Cinesite 2020-21.
• 10% to up to 25% rendering efficiency/thousands of hours saved in rendering production time/15 hrs per frame per shot to 12-13 hrs.
• Cinesite Configuration: 18-core Intel^® Xeon^® Scalable processors (W-2295) used in render farm, 2nd gen Intel Xeon processor-based workstations (W-2135 and -2195) used. Rendering tools: Gaffer, Arnold, along with optimizations by Intel^® Open Image Denoise.
• Performance varies by use, configuration, and other factors. Learn more at www.Intel.com/PerformanceIndex​.
• Your costs and results may vary.
• Intel technologies may require enabled hardware, software, or service activation.
• Intel does not control or audit third-party data. You should consult other sources to evaluate accuracy.

© 2022 Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.