Creating Styleframes with LeonardoAI and InvokeAI

By -


I decided to take screenshots of one of my favourite animated movies, The Road to El Dorado, and create some live action styleframes out of them.


---

Process:

To get an initial image out, I'm using Leonardo AI. It's workflow helps me quickly establish the style and composition I'm looking for. In this case, I have a screenshot to match to:



Quick definition before continuing:

Control Nets

These are AI tools that guide image generation by providing additional structural input, such as sketches, depth maps, or edge detection. They help maintain consistency in composition, pose, or perspective while still allowing for creative freedom in generating an output image.

Since I want to match my image as closely as possible to the screenshot, I'm using a combination of ControlNets with the screenshot as my input image.

The Edge to image ensures my generated images follow the specific structure.
The Depth to image acts as my depth map.

Image Prompt Adapter

IP Adapters extract specific visual elements from reference images and apply them to new AI-generated artwork. It's an add-on that allows you to use images as prompts.


The Style to image uses the colours and textures from my input image.


Each control has its own strength and finding the right value for each while having them all work together is key to obtaining the look you're after. 


My output image:



I'm pretty happy with my output, but there are some specific areas that I wanted to change to match it more accurately to the input frame, for example, change the trees to hanging vines and add some woven fabric in the doorway.

I'm taking this image into InvokeAI, where I can mask the specific region I want to work on.

Note: The denoising strength in Invoke AI drives how much you want your output image to deviate from your input image. 
Lower value = Closer to the image, for subtle changes.
Higher value = Closer to your prompts/inputs, more noise added, more creative freedom for changes to your image.



My final output image:





Here are some more screenshots I turned to styleframes using the leonardo AI and Invoke AI workflow.

Chel character:






One of the Eldorado temples:






X marks the spot:


























Comments

Post a Comment

Popular posts from this blog

DMP plate rework with GenAI

By -
Image
I have been reworking plates with GenAI to complete DMP tasks such as season changes and time of day changes. --- Process: I start with my plate, making sure it's cleaned up and ready to work with. My plate will be the starting raster layer in InvokeAI: I would like to change the overall overcast lighting to a brighter sky and warm sunlight with long cast shadows. I am using an image to image approach, together with ControlNets and IP adapters, which allows me to preserve the plate as much as possible while controlling the changes I want to make to the plate.  For example, using a canny edge detection and depth map will retain the structure. After some tweaking and finding a good balance between the denoising strength value, the weight of each ControlNet and my prompt, I was happy with my result: A season change to autumn: A season change to winter: Quick note that the style referencing takes into account the mood and texture of the image reference. You can add more than one refere...
Read more

ComfyUI IC-Light relighting exploration

By -
Image
I've started using  ComfyUI  to gain more control over my workflow.  Its  node-based system  is similar to InvokeAI. I'm exploring IC-Light (image relighting) and it's a great tool in visual development for composition, lighting, mood and style. Since it runs on an SD1.5 model, it's not quite ready for a final DMP output yet, but I see lots of potential for visual development. For example, it is quick at giving you a location with different lighting scenarios. I was also able to exactly control the direction of the lighting, the colour and intensity. Using my own photos, I'm able to quickly create multiple different moods for art direction. For my workflow, I mixed different colour gradients with my image, controlling the light direction and tone of the scene. Original photo: My IC-Light generations: Original photo: My IC-Light generations: Example of light direction control with a spline: Adding some colour to the gradient: Output example: Original photo: My IC...
Read more