The Generative Canvas: A Matte Painter's Perspective

I've started using  ComfyUI  to gain more control over my workflow.  Its  node-based system  is similar to InvokeAI. I'm exploring IC-Light (image relighting) and it's a great tool in visual development for composition, lighting, mood and style. Since it runs on an SD1.5 model, it's not quite ready for a final DMP output yet, but I see lots of potential for visual development. For example, it is quick at giving you a location with different lighting scenarios. I was also able to exactly control the direction of the lighting, the colour and intensity. Using my own photos, I'm able to quickly create multiple different moods for art direction. For my workflow, I mixed different colour gradients with my image, controlling the light direction and tone of the scene. Original photo: My IC-Light generations: Original photo: My IC-Light generations: Example of light direction control with a spline: Adding some colour to the gradient: Output example: Original photo: My IC...

I have been reworking plates with GenAI to complete DMP tasks such as season changes and time of day changes. --- Process: I start with my plate, making sure it's cleaned up and ready to work with. My plate will be the starting raster layer in InvokeAI: I would like to change the overall overcast lighting to a brighter sky and warm sunlight with long cast shadows. I am using an image to image approach, together with ControlNets and IP adapters, which allows me to preserve the plate as much as possible while controlling the changes I want to make to the plate.  For example, using a canny edge detection and depth map will retain the structure. After some tweaking and finding a good balance between the denoising strength value, the weight of each ControlNet and my prompt, I was happy with my result: A season change to autumn: A season change to winter: Quick note that the style referencing takes into account the mood and texture of the image reference. You can add more than one refere...