Using it on video is almost exactly like creating a set extension.
I recently used the AI Generative Fill in Photoshop to generate some unique material for a pitch deck I’m working on. So far, results have been mixed but also impressive. I found it has worked best when filing in spaces on images you already have.
But what about using it for video?
Kinda, Maybe?
Long story short, no.
Even if you break out the individual frames from the video and use the AI Generative Fill on each one, you won’t be able to get any cohesion between the different frames. While that might create a cool effect, it’s not really what we want.
We want actual elements that stay static between our frames. That’s the keyword here: static.
Creator Karl Ndieli took a page from the movie world and used the Generative Fill as a set extension when cropping his videos for social media.
He did this by zooming in on the 16:9 YouTube video but leaving room at the top or bottom. Then he would take a single frame, and use the AI Generative Fill to fill in the gaps. Then he would take just that generated section and layer it on his video, much like films used to do with matte painting.
You can check out Ndieli’s approach in this YouTube Short.
Matte Painting in 2023
Lizzie Peirce piggybacked off this technique and also made her own version of the “set extension” in her own video talking about the different ways you can utilize this new tool from Adobe.
But here are some obstacles that you’ll have to work with if you use this technique. Much like matte paintings of yesteryear (the ones painted on glass), creatives may only be able to utilize this approach with static footage.
We say “may” because some tech-savvy creatives could use things like camera mapping in Blender to animate the extended section using a program like Blender. But that’s a lot of extra work for just a simple extension.
It’s not a perfect solution, but it’s definitely a unique tool to have in your bag when you need that extra space around your frame, especially if your composition is static.
But what do you think? How would you utilize this technique?
