Diffusers から AnimateDiff が使えるようになりました

github.com

動作環境

動作確認できた環境です。必ずしもこれでなければいけないということはありません。

Windows 11

Windows 11
CUDA 11.7
Python 3.10

WSL2

Ubuntu 22.04 on WSL2
CUDA 11.8
Python 3.10

環境構築

CUDA 11.8の場合

pip install torch==2.0.1+cu118 torchvision==0.15.2+cu118 --index-url https://download.pytorch.org/whl/cu118
pip install diffusers[torch]
pip install transformers omegaconf

Pythonスクリプト

import torch
from diffusers import MotionAdapter, AnimateDiffPipeline, DDIMScheduler, AutoencoderKL
from diffusers.utils import export_to_gif

# Load the motion adapter
adapter = MotionAdapter.from_pretrained("animatediff-motion-adapter-v1-5-2")
#adapter = MotionAdapter.from_pretrained("guoyww/animatediff-motion-adapter-v1-5-2")
# load SD 1.5 based finetuned model
model_id = "model/yabalMixTrue25D_v4_ema"
pipe = AnimateDiffPipeline.from_pretrained(
    model_id, 
    motion_adapter=adapter,
    vae=AutoencoderKL.from_single_file("vae/vae-ft-mse-840000-ema-pruned.safetensors")
)
pipe.scheduler = DDIMScheduler.from_pretrained(
    model_id,
    subfolder="scheduler",
    beta_schedule="linear",
    clip_sample=False,
    timestep_spacing="linspace",
    steps_offset=1
)
pipe.load_lora_weights("lora/more_details.safetensors")

# enable memory savings
pipe.enable_vae_slicing()
pipe.enable_model_cpu_offload()

output = pipe(
    prompt="a girl, dancing, blue denim, white plain t-shirt, best quality, extremely detailed",
    negative_prompt="longbody, lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality",
    num_frames=16,
    guidance_scale=8.5,
    num_inference_steps=40,
    generator=torch.manual_seed(1000000000),
)
frames = output.frames[0]
export_to_gif(frames, "animation.gif")

結果

結果はGoogle Bloggerに載せています。
support-touchsp.blogspot.com