【Diffusers】MiniMax-H3で動画生成

使用したPC

こちらのPCを使っています。

Windows 11 Home
Core i7-12700H
GeForce RTX 3080 Laptop VRAM 16GB
RAM: 32GB
SSD: 1TB NVMe SSD

環境

Ubuntu 24.04 on WSL2

事前準備

Ubuntuに以下をインストールする必要がありました。

sudo apt install -y build-essential gcc g++

Python環境構築

uvを使用しています。pyproject.tomlを載せておきます。

[project]
name = "minimax"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.13, <3.14"
dependencies = [
    "accelerate==1.14.0",
    "diffusers @ git+https://github.com/huggingface/diffusers",
    "hf-xet==1.5.1",
    "torch==2.13.0+cu126",
    "torchvision==0.28.0+cu126",
    "transformers==5.14.1",
    "sdnq==0.2.4",
    "triton==3.7.1",
    "av==18.0.0",
]

[[tool.uv.index]]
name = "torch-cuda"
url = "https://download.pytorch.org/whl/cu126"
explicit = true

[tool.uv.sources]
torch = [{ index = "torch-cuda" }]
torchvision = [{ index = "torch-cuda" }]

Pythonスクリプト

import torch

# sdnq をインポートすることで transformers 側に量子化バックエンドが登録されます
import sdnq

from diffusers import ModularPipeline
from diffusers.hooks import apply_group_offloading
from diffusers.utils import encode_video


MODEL = "OzzyGT/MiniMax_H3_sdnq_dynamic_4bit"
PROMPT = (
    "A red fox trotting through a snowy pine forest at dawn, snow crunching underfoot, "
    "breath visible in the cold air, low winter sun through the trees"
)

pipe = ModularPipeline.from_pretrained(MODEL)
pipe.load_components(workflow="t2va", dtype=torch.bfloat16)

offload = {
    "onload_device": torch.device("cuda"),
    "offload_device": torch.device("cpu"),
    "use_stream": True,
    "low_cpu_mem_usage": True,
}
pipe.transformer.enable_group_offload(offload_type="block_level", num_blocks_per_group=1, **offload)
apply_group_offloading(pipe.text_encoder.model, offload_type="leaf_level", **offload)
apply_group_offloading(pipe.vae, offload_type="leaf_level", **offload)
pipe.audio_vae.to("cuda")

state = pipe(
    prompt=PROMPT,
    num_frames=124,
    height=544,
    width=960,
    num_inference_steps=30,
    generator=torch.Generator("cpu").manual_seed(42),
)

encode_video(
    state.get("videos")[0],
    fps=24,
    audio=state.get("audio")[0],
    audio_sample_rate=state.get("sampling_rate"),
    output_path="h3_768p_group_offload.mp4",
)

結果

生成動画はこちらの載せておきます。