sd-forge-layerdiffuse

Transparent Image Layer Diffusion using Latent Transparency

This is a WIP extension for SD WebUI (via Forge) to generate transparent images and layers.

Updates

img2img is finished! See also here

Before You Start

Because many people may be curious about how the latent preview looks like during a transparent diffusion process, I recorded a video so that you can see it before you download the models and extensions:

https://github.com/layerdiffusion/sd-forge-layerdiffusion/assets/161511761/e93b71d1-3560-48e2-a970-0b8efbfebb42

You can see that the native transparent diffusion can process transparent glass, semi-transparent glowing effects, etc, that are not possible with simple background removal methods. Native transparent diffusion also gives you detailed fur, hair, whiskers, and detailed structure like that skeleton.

Model Notes

Note that in this extension, all model downloads/selections are fully automatic. In fact most users can just skip this section.

Below models are released:

layer_xl_transparent_attn.safetensors This is a rank-256 LoRA to turn a SDXL into a transparent image generator. It will change the latent distribution of the model to a "transparent latent space" that can be decoded by the special VAE pipeline.
layer_xl_transparent_conv.safetensors This is an alternative model to turn your SDXL into a transparent image generator. This safetensors file includes an offset of all conv layers (and actually, all layers that are not q,k,v of any attention layers). These offsets can be merged to any XL model to change the latent distribution to transparent images. Because we excluded the offset training of any q,k,v layers, the prompt understanding of SDXL should be perfectly preserved. However, in practice, I find the layer_xl_transparent_attn.safetensors will lead to better results. This layer_xl_transparent_conv.safetensors is still included for some special use cases that needs special prompt understanding. Also, this model may introduce a strong style influence to the base model.
layer_xl_fg2ble.safetensors This is a safetensors file includes offsets to turn a SDXL into a layer generating model, that is conditioned on foregrounds, and generates blended compositions.
layer_xl_fgble2bg.safetensors This is a safetensors file includes offsets to turn a SDXL into a layer generating model, that is conditioned on foregrounds and blended compositions, and generates backgrounds.
layer_xl_bg2ble.safetensors This is a safetensors file includes offsets to turn a SDXL into a layer generating model, that is conditioned on backgrounds, and generates blended compositions.
layer_xl_bgble2fg.safetensors This is a safetensors file includes offsets to turn a SDXL into a layer generating model, that is conditioned on backgrounds and blended compositions, and generates foregrounds.
vae_transparent_encoder.safetensors This is an image encoder to extract a latent offset from pixel space. The offset can be added to latent images to help the diffusion of transparency. Note that in the paper we used a relatively heavy model with exactly same amount of parameters as the SD VAE. The released model is more light weighted, requires much less vram, and does not influence result quality in my tests.
vae_transparent_decoder.safetensors This is an image decoder that takes SD VAE outputs and latent image as inputs, and outputs a real PNG image. The model architecture is also more lightweight than the paper version to reduce VRAM requirement. I have made sure that the reduced parameters does not influence result quality.
layer_sd15_vae_transparent_encoder.safetensors Same as above VAE encoder, but fine-tuned for SD1.5.
layer_sd15_vae_transparent_decoder.safetensors Same as above VAE decoder, but fine-tuned for SD1.5.
layer_sd15_transparent_attn.safetensors This is a rank-256 LoRA to turn a SD1.5 into a transparent image generator. It will change the latent distribution of the model to a "transparent latent space" that can be decoded by the special VAE pipeline.
layer_sd15_joint.safetensors This model file allows for generating all layers together with SD1.5. It includes two rank-256 loras (foreground lora and background lora), and an attention sharing module to share attention between multiple diffusion processes on par. Note that different from paper, this model file includes an additional "blended lora", and it actually can generate three images together (fg, bg, and blended image). Generating blended images together with fg and bg is helpful for structural understanding in our very recent tests.
layer_sd15_fg2bg.safetensors This model file allows for generating background from foreground with SD1.5. It includes a rank-256 lora and an attention sharing module to share attention between multiple diffusion processes on par. This model file includes an additional "blended lora", and it actually can generate two images together (bg and blended image). Generating blended images together with bg is helpful for structural understanding in our very recent tests. Besides, to save VRAM, the fg is directly feed into all attention layers as control signal, rather than creating another diffusion pass.
layer_sd15_bg2fg.safetensors This model file allows for generating foreground from background with SD1.5. It includes a rank-256 lora and an attention sharing module to share attention between multiple diffusion processes on par. This model file includes an additional "blended lora", and it actually can generate two images together (fg and blended image). Generating blended images together with fg is helpful for structural understanding in our very recent tests. Besides, to save VRAM, the bg is directly feed into all attention layers as control signal, rather than creating another diffusion pass.

Below models may be released soon (if necessary):

SDXL models that can generate foreground and background together and SDXL's one step conditional model. (Note that all joint models for SD1.5 are already released) I put this model on hold because of these reasons: (1) the other released models can already achieve all functionalities and this model does not bring more functionalities. (2) the inference speed of this model is 3x slower than others and requires 4x more VRAM than other released model, and I am working on reducing the VRAM of this model and speed up the inference. (3) This model will involve more hyperparameters and if demanded, I will investigate the best practice for inference/training before release it.
The current background-conditioned foreground model for SDXL may be a bit too lightweight. I will probably release a heavier one with more parameters and different behaviors (see also the discussions later).
Because the difference between diffusers training and k-diffusion inference, I can observe some mystical problems like sometimes DPM++ will give artifacts but Euler A will fix it. I am looking into it and may provide some revised model that works better with all A1111 samplers.
Two-step foreground and background conditional models for SD1.5. (Note that one-step conditional/joint models are already released.)

Sanity Check

SDXL

We highly encourage you to go through the sanity check and get exactly same results (so that if any problem occurs, we will know if the problem is on our side).

The two used models are:

https://civitai.com/models/133005?modelVersionId=198530 Juggernaut XL V6 (note that the used one is V6, not v7 or v8 or V9)
https://civitai.com/models/261336?modelVersionId=295158 anima_pencil-XL 1.0.0 (note that the used one is 1.0.0, not 1.5.0)

We will first test transparent image generating. Set your extension to this:

an apple, high quality

Negative prompt: bad, ugly

Steps: 20, Sampler: DPM++ 2M SDE Karras, CFG scale: 5, Seed: 12345, Size: 1024x1024, Model hash: 1fe6c7ec54, Model: juggernautXL_version6Rundiffusion, layerdiffusion_enabled: True, layerdiffusion_method: Only Generate Transparent Image (Attention Injection), layerdiffusion_weight: 1, layerdiffusion_ending_step: 1, layerdiffusion_fg_image: False, layerdiffusion_bg_image: False, layerdiffusion_blend_image: False, layerdiffusion_resize_mode: Crop and Resize, Version: f0.0.17v1.8.0rc-latest-269-gef35383b

Make sure that you get this apple

woman, messy hair, high quality

Negative prompt: bad, ugly

Make sure that you get the woman with hair as messy as this

a cup made of glass, high quality

Negative prompt: bad, ugly

Make sure that you get this cup

glowing effect, book of magic, high quality

Negative prompt: bad, ugly

Steps: 20, Sampler: DPM++ 2M SDE Karras, CFG scale: 7, Seed: 12345, Size: 1024x1024, Model hash: 1fe6c7ec54, Model: juggernautXL_version6Rundiffusion, layerdiffusion_enabled: True, layerdiffusion_method: Only Generate Transparent Image (Attention Injection), layerdiffusion_weight: 1, layerdiffusion_ending_step: 1, layerdiffusion_fg_image: True, layerdiffusion_bg_image: False, layerdiffusion_blend_image: True, layerdiffusion_resize_mode: Crop and Resize, Version: f0.0.17v1.8.0rc-latest-269-gef35383b

make sure that you get this glowing book

OK then lets move on to a bit longer prompt:

(this prompt is from https://civitai.com/images/3160575)

photograph close up portrait of Female boxer training, serious, stoic cinematic 4k epic detailed 4k epic detailed photograph shot on kodak detailed bokeh cinematic hbo dark moody

Negative prompt: (worst quality, low quality, normal quality, lowres, low details, oversaturated, undersaturated, overexposed, underexposed, grayscale, bw, bad photo, bad photography, bad art:1.4), (watermark, signature, text font, username, error, logo, words, letters, digits, autograph, trademark, name:1.2), (blur, blurry, grainy), morbid, ugly, asymmetrical, mutated malformed, mutilated, poorly lit, bad shadow, draft, cropped, out of frame, cut off, censored, jpeg artifacts, out of focus, glitch, duplicate, (airbrushed, cartoon, anime, semi-realistic, cgi, render, blender, digital art, manga, amateur:1.3), (3D ,3D Game, 3D Game Scene, 3D Character:1.1), (bad hands, bad anatomy, bad body, bad face, bad teeth, bad arms, bad legs, deformities:1.3)

Steps: 20, Sampler: DPM++ 2M SDE Karras, CFG scale: 7, Seed: 12345, Size: 896x1152, Model hash: 1fe6c7ec54, Model: juggernautXL_version6Rundiffusion, layerdiffusion_enabled: True, layerdiffusion_method: Only Generate Transparent Image (Attention Injection), layerdiffusion_weight: 1, layerdiffusion_ending_step: 1, layerdiffusion_fg_image: False, layerdiffusion_bg_image: False, layerdiffusion_blend_image: False, layerdiffusion_resize_mode: Crop and Resize, Version: f0.0.17v1.8.0rc-latest-269-gef35383b

Anime model test:

girl in dress, high quality

Negative prompt: nsfw, bad, ugly, text, watermark

Steps: 20, Sampler: DPM++ 2M SDE Karras, CFG scale: 7, Seed: 12345, Size: 896x1152, Model hash: 7ed8da12d9, Model: animaPencilXL_v100, layerdiffusion_enabled: True, layerdiffusion_method: Only Generate Transparent Image (Attention Injection), layerdiffusion_weight: 1, layerdiffusion_ending_step: 1, layerdiffusion_fg_image: False, layerdiffusion_bg_image: False, layerdiffusion_blend_image: False, layerdiffusion_resize_mode: Crop and Resize, Version: f0.0.17v1.8.0rc-latest-269-gef35383b

(I am not very good at writing prompts in the AnimagineXL format, and perhaps you can get better results with better prompts)

SD1.5

The tested model is realisticVisionV51_v51VAE. We highly encourage you to go through the sanity check and get exactly same results (so that if any problem occurs, we will know if the problem is on our side).

an apple, 4k,