LORE - AI Labs
Where VFX & AI meet
Hybrid workflow for asset builds.
Is it possible to take a 3d model and let AI handle grooming & lookdev?
It is. But why would you when you can prompt whatever in minutes? You do this because you want to be able to direct performances.
Based off of our carefully crafted 3d model we let AI handle grooming & lookdev, two tasks that takes quite some time to do properly with standard VFX tools. With AI it still takes a bit of time getting things right but it’s definitely a faster process. It’s not possible to tweak and finesse as you would with a regular VFX asset and the final image is not a 1:1 match, but close enough.
Hybrid workflow for shot production.
Is it possible to animate the hybrid asset, light it and comp it into an AI generated plate?
It is. For the first few tasks we did everything as usual:
- Rig the asset.
- Animate the asset as usual.
OK
We then use the 3d animation to drive the performance of the hybrid asset in our AI tools. As you can see here it starts in the classical “bind-pose” since that’s how the image of the asset was generated, this results in a wee bit of an offset in the actual animation but once it’s walking it reflects the animation dailie pretty well. Again, not a 1:1 match but AI smooths out minor kinks and also does the CFX pass (muscles, skin and fur simulations) at the same time. Another step that takes quite some time to setup and run on a per shot basis, so this is quite a time saver.
All in all doing things like this is a bit of a trade-off, you get what you get but it’s a lot faster than doing it with traditional tools.
Can we integrate it in a live action plate?
Yes we can!
- We undistort and track the plate as usual so we can animate within that 3d space.
- To get a start frame corresponding closer to the actual 3d output we generate a new start frame based off of the 3d.
- Based off of the plate and carefully crafted prompts lighting, CFX and comp are all handled by the AI.
- Again, not a 1:1 match on the plate but close enough.
Can we the use plate camera and generate an AI plate?
We sure do. So now we have a live action camera, a hand animated creature and AI generated backplate, fur, lookdev, lighting and comp. To do this we need to let the AI dream more freely hence some of the animation is lost in translation.