Buzz Hayes wants his Hollywood colleagues to understand the pros and cons of generative AI, especially model tweaking.
And a good starting point for explaining this controversial topic is the work Google Cloud’s global entertainment industry solutions leader Hayes and his team did to bring Warner Bros. Discovery’s The Wizard of Oz to life at the Sphere event in Las Vegas in August.
“What led us to this project is a sometimes confusing process. I’ve heard people use the term incorrectly, but there’s a process called model fine-tuning,” Hayes told Variety. “You could say, “Give me the output of the Tin Woodman doing something, without the model understanding anything about the actual Tin Woodman. So you get a fuzzy approximation. But when you do something like this, that’s not enough. So we actually fine-tuned the model. It’s basically a base that any IP owner has trained on and uses all the sources that people have opted in to contribute. It’s a process that allows us to take a model. So this is a legitimate model, but doesn’t understand the Tin Woodman, so what we can do is tweak the Tin Woodman and say, “Now we know how humans move. So when we refer to the Tin Woodman, use this version of the Tin Woodman.”
“In that case, we had a fine-tuned model for each of the main characters in the movie, so no matter where he was facing, how close he was to the camera, no matter how he turned around, we were able to faithfully represent that actor. For example, what does the back of the Tin Man’s head look like? There’s not a lot in the movie, but we had a few shots in there that we could make adjustments to in scenes where he had to turn his face,” Hayes continued. Head, we knew exactly what he looked like. ”
“All the data used to do that is owned and maintained by the customer,” said Hayes, who worked as a Hollywood producer for more than 30 years before moving to his current role at the tech giant. “It’s never incorporated into the model.” Google Cloud Entertainment leaders want to continue to articulate this concept, as they see this technology becoming increasingly integral to the production process across the film and television industry.
“That’s where the power of AI comes in. I think especially for a franchise title, you want a certain similarity that persists throughout the series, so it makes sense for these tweaked models to exist for that specific reason,” Hayes said. “So I think it kind of runs the gamut in terms of how these things are leveraged in terms of what will produce successful outcomes and what will not produce successful outcomes.”
Hayes stressed that while the technology is here to stay, Google Cloud has put in place “significant guardrails” to protect IP and publicity rights as it continues to expand across entertainment.
“We don’t allow our models to produce likenesses of known celebrities or anyone. We have very careful protections in place for these models,” Hayes said. “We put guardrails against things like violence. We put guardrails against various things, like procreating children. So all these guardrails contribute to this idea of reparations policy where we protect you but don’t do bad things. That’s basically what we do, but we don’t even allow you to do bad things because those guardrails protect you.”
Hayes points out how these principles were used in the Sphere project’s “Wizard of Oz,” and says the Google Cloud team is “evaluating” the limits placed on what generative AI can do.
“It was really about how can we honor this original film,” Hayes said. “There were a lot of constraints from the studio, like you couldn’t replace Dorothy’s performance with an actress, you couldn’t replace her voice, you couldn’t use motion capture to do new performances that weren’t there in the original, things like that. I love constraints in filmmaking. I think the best art comes from constraints, not from unlimited possibilities. So this was a really interesting way to approach it.”
