ð Click on the language section to expand / èšèªãã¯ãªãã¯ããŠå±é
Wan 2.1
Overview / æŠèŠ
This is an unofficial training and inference script for Wan2.1. The features are as follows.
- fp8 support and memory reduction by block swap: Inference of a 720x1280x81frames videos with 24GB VRAM, training with 720x1280 images with 24GB VRAM
- Inference without installing Flash attention (using PyTorch's scaled dot product attention)
- Supports xformers and Sage attention
This feature is experimental.
æ¥æ¬èª
[Wan2.1](https://github.com/Wan-Video/Wan2.1) ã®éå ¬åŒã®åŠç¿ããã³æšè«ã¹ã¯ãªããã§ãã以äžã®ç¹åŸŽããããŸãã
- fp8察å¿ããã³block swapã«ããçã¡ã¢ãªåïŒ720x1280x81framesã®åç»ã24GB VRAMã§æšè«å¯èœã720x1280ã®ç»åã§ã®åŠç¿ã24GB VRAMã§å¯èœ
- Flash attentionã®ã€ã³ã¹ããŒã«ãªãã§ã®å®è¡ïŒPyTorchã®scaled dot product attentionã䜿çšïŒ
- xformersããã³Sage attention察å¿
ãã®æ©èœã¯å®éšçãªãã®ã§ãã
Download the model / ã¢ãã«ã®ããŠã³ããŒã
Download the T5 models_t5_umt5-xxl-enc-bf16.pth
and CLIP models_clip_open-clip-xlm-roberta-large-vit-huge-14.pth
from the following page: https://huggingface.co/Wan-AI/Wan2.1-I2V-14B-720P/tree/main
Download the VAE from the above page Wan2.1_VAE.pth
or download split_files/vae/wan_2.1_vae.safetensors
from the following page: https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/tree/main/split_files/vae
Download the DiT weights from the following page: https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/tree/main/split_files/diffusion_models
Wan2.1 Fun Control model weights can be downloaded from here. Navigate to each weight page and download. The Fun Control model seems to support not only T2V but also I2V tasks.
Please select the appropriate weights according to T2V, I2V, resolution, model size, etc.
fp16
and bf16
models can be used, and fp8_e4m3fn
models can be used if --fp8
(or --fp8_base
) is specified without specifying --fp8_scaled
. Please note that fp8_scaled
models are not supported even with --fp8_scaled
.
(Thanks to Comfy-Org for providing the repackaged weights.)
Model support matrix / ã¢ãã«ãµããŒããããªãã¯ã¹
- columns: training dtype (è¡ïŒåŠç¿æã®ããŒã¿å)
- rows: model dtype (åïŒã¢ãã«ã®ããŒã¿å)
model \ training | bf16 | fp16 | --fp8_base | --fp8base & --fp8_scaled |
---|---|---|---|---|
bf16 | â | -- | â | â |
fp16 | -- | â | â | â |
fp8_e4m3fn | -- | -- | â | -- |
fp8_scaled | -- | -- | -- | -- |
æ¥æ¬èª
T5 `models_t5_umt5-xxl-enc-bf16.pth` ããã³CLIP `models_clip_open-clip-xlm-roberta-large-vit-huge-14.pth` ããæ¬¡ã®ããŒãžããããŠã³ããŒãããŠãã ããïŒhttps://huggingface.co/Wan-AI/Wan2.1-I2V-14B-720P/tree/mainVAEã¯äžã®ããŒãžãã Wan2.1_VAE.pth
ãããŠã³ããŒãããããæ¬¡ã®ããŒãžãã split_files/vae/wan_2.1_vae.safetensors
ãããŠã³ããŒãããŠãã ããïŒhttps://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/tree/main/split_files/vae
DiTã®éã¿ã次ã®ããŒãžããããŠã³ããŒãããŠãã ããïŒhttps://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/tree/main/split_files/diffusion_models
Wan2.1 Fun Controlã¢ãã«ã®éã¿ã¯ããã¡ããããããããã®éã¿ã®ããŒãžã«é·ç§»ããããŠã³ããŒãããŠãã ãããFun Controlã¢ãã«ã¯T2Vã ãã§ãªãI2Vã¿ã¹ã¯ã«ã察å¿ããŠããããã§ãã
T2VãI2Vãè§£å床ãã¢ãã«ãµã€ãºãªã©ã«ããé©åãªéã¿ãéžæããŠãã ããã
fp16
ããã³ bf16
ã¢ãã«ã䜿çšã§ããŸãããŸãã--fp8
ïŒãŸãã¯--fp8_base
ïŒãæå®ã--fp8_scaled
ãæå®ãããªããšãã«ã¯ fp8_e4m3fn
ã¢ãã«ã䜿çšã§ããŸãã**fp8_scaled
ã¢ãã«ã¯ãããã®å ŽåããµããŒããããŠããŸããã®ã§ã泚æãã ããã**
ïŒrepackagedçã®éã¿ãæäŸããŠãã ãã£ãŠããComfy-Orgã«æè¬ããããŸããïŒ
Pre-caching / äºåãã£ãã·ã¥
Latent Pre-caching
Latent pre-caching is almost the same as in HunyuanVideo. Create the cache using the following command:
python wan_cache_latents.py --dataset_config path/to/toml --vae path/to/wan_2.1_vae.safetensors
If you train I2V models, add --clip path/to/models_clip_open-clip-xlm-roberta-large-vit-huge-14.pth
to specify the CLIP model. If not specified, the training will raise an error.
If you're running low on VRAM, specify --vae_cache_cpu
to use the CPU for the VAE internal cache, which will reduce VRAM usage somewhat.
The control video settings are required for training the Fun-Control model. Please refer to Dataset Settings for details.
æ¥æ¬èª
latentã®äºåãã£ãã·ã³ã°ã¯HunyuanVideoãšã»ãŒåãã§ããäžã®ã³ãã³ãäŸã䜿çšããŠãã£ãã·ã¥ãäœæããŠãã ãããI2Vã¢ãã«ãåŠç¿ããå Žåã¯ã--clip path/to/models_clip_open-clip-xlm-roberta-large-vit-huge-14.pth
ã远å ããŠCLIPã¢ãã«ãæå®ããŠãã ãããæå®ããªããšåŠç¿æã«ãšã©ãŒãçºçããŸãã
VRAMãäžè¶³ããŠããå Žåã¯ã--vae_cache_cpu
ãæå®ãããšVAEã®å
éšãã£ãã·ã¥ã«CPUã䜿ãããšã§ã䜿çšVRAMãå€å°åæžã§ããŸãã
Fun-Controlã¢ãã«ãåŠç¿ããå Žåã¯ãå¶åŸ¡çšåç»ã®èšå®ãå¿ èŠã§ããããŒã¿ã»ããèšå®ãåç §ããŠãã ããã
Text Encoder Output Pre-caching
Text encoder output pre-caching is also almost the same as in HunyuanVideo. Create the cache using the following command:
python wan_cache_text_encoder_outputs.py --dataset_config path/to/toml --t5 path/to/models_t5_umt5-xxl-enc-bf16.pth --batch_size 16
Adjust --batch_size
according to your available VRAM.
For systems with limited VRAM (less than ~16GB), use --fp8_t5
to run the T5 in fp8 mode.
æ¥æ¬èª
ããã¹ããšã³ã³ãŒãåºåã®äºåãã£ãã·ã³ã°ãHunyuanVideoãšã»ãŒåãã§ããäžã®ã³ãã³ãäŸã䜿çšããŠãã£ãã·ã¥ãäœæããŠãã ããã䜿çšå¯èœãªVRAMã«åãã㊠--batch_size
ã調æŽããŠãã ããã
VRAMãéãããŠããã·ã¹ãã ïŒçŽ16GBæªæºïŒã®å Žåã¯ãT5ãfp8ã¢ãŒãã§å®è¡ããããã« --fp8_t5
ã䜿çšããŠãã ããã
Training / åŠç¿
Training
Start training using the following command (input as a single line):
accelerate launch --num_cpu_threads_per_process 1 --mixed_precision bf16 wan_train_network.py
--task t2v-1.3B
--dit path/to/wan2.1_xxx_bf16.safetensors
--dataset_config path/to/toml --sdpa --mixed_precision bf16 --fp8_base
--optimizer_type adamw8bit --learning_rate 2e-4 --gradient_checkpointing
--max_data_loader_n_workers 2 --persistent_data_loader_workers
--network_module networks.lora_wan --network_dim 32
--timestep_sampling shift --discrete_flow_shift 3.0
--max_train_epochs 16 --save_every_n_epochs 1 --seed 42
--output_dir path/to/output_dir --output_name name-of-lora
The above is an example. The appropriate values for timestep_sampling
and discrete_flow_shift
need to be determined by experimentation.
For additional options, use python wan_train_network.py --help
(note that many options are unverified).
--task
is one of t2v-1.3B
, t2v-14B
, i2v-14B
, t2i-14B
(for Wan2.1 official models), t2v-1.3B-FC
, t2v-14B-FC
, and i2v-14B-FC
(for Wan2.1 Fun Control model). Specify the DiT weights for the task with --dit
.
Don't forget to specify --network_module networks.lora_wan
.
Other options are mostly the same as hv_train_network.py
.
Use convert_lora.py
for converting the LoRA weights after training, as in HunyuanVideo.
æ¥æ¬èª
`timestep_sampling`ã`discrete_flow_shift`ã¯äžäŸã§ããã©ã®ãããªå€ãé©åãã¯å®éšãå¿ èŠã§ãããã®ä»ã®ãªãã·ã§ã³ã«ã€ããŠã¯ python wan_train_network.py --help
ã䜿çšããŠãã ããïŒå€ãã®ãªãã·ã§ã³ã¯æªæ€èšŒã§ãïŒã
--task
ã«ã¯ t2v-1.3B
, t2v-14B
, i2v-14B
, t2i-14B
ïŒãããã¯Wan2.1å
¬åŒã¢ãã«ïŒãt2v-1.3B-FC
, t2v-14B-FC
, i2v-14B-FC
ïŒWan2.1-Fun Controlã¢ãã«ïŒãæå®ããŸãã--dit
ã«ãtaskã«å¿ããDiTã®éã¿ãæå®ããŠãã ããã
--network_module
ã« networks.lora_wan
ãæå®ããããšãå¿ããªãã§ãã ããã
ãã®ä»ã®ãªãã·ã§ã³ã¯ãã»ãŒhv_train_network.py
ãšåæ§ã§ãã
åŠç¿åŸã®LoRAã®éã¿ã®å€æã¯ãHunyuanVideoãšåæ§ã«convert_lora.py
ã䜿çšããŠãã ããã
Command line options for training with sampling / ãµã³ãã«ç»åçæã«é¢é£ããåŠç¿æã®ã³ãã³ãã©ã€ã³ãªãã·ã§ã³
Example of command line options for training with sampling / èšè¿°äŸ:
--vae path/to/wan_2.1_vae.safetensors
--t5 path/to/models_t5_umt5-xxl-enc-bf16.pth
--sample_prompts /path/to/prompt_file.txt
--sample_every_n_epochs 1 --sample_every_n_steps 1000 -- sample_at_first
Each option is the same as when generating images or as HunyuanVideo. Please refer to here for details.
If you train I2V models, add --clip path/to/models_clip_open-clip-xlm-roberta-large-vit-huge-14.pth
to specify the CLIP model.
You can specify the initial image, the negative prompt and the control video (for Wan2.1-Fun-Control) in the prompt file. Please refer to here.
æ¥æ¬èª
åãªãã·ã§ã³ã¯æšè«æãããã³HunyuanVideoã®å Žåãšåæ§ã§ãã[ãã¡ã](/docs/sampling_during_training.md)ãåç §ããŠãã ãããI2Vã¢ãã«ãåŠç¿ããå Žåã¯ã--clip path/to/models_clip_open-clip-xlm-roberta-large-vit-huge-14.pth
ã远å ããŠCLIPã¢ãã«ãæå®ããŠãã ããã
ããã³ãããã¡ã€ã«ã§ãåæç»åããã¬ãã£ãããã³ãããå¶åŸ¡åç»ïŒWan2.1-Fun-ControlçšïŒçãæå®ã§ããŸãããã¡ããåç §ããŠãã ããã
Inference / æšè«
Inference Options Comparison / æšè«ãªãã·ã§ã³æ¯èŒ
Speed Comparison (Faster â Slower) / é床æ¯èŒïŒéãâé ãïŒ
Note: Results may vary depending on GPU type
fp8_fast > bf16/fp16 (no block swap) > fp8 > fp8_scaled > bf16/fp16 (block swap)
Quality Comparison (Higher â Lower) / å質æ¯èŒïŒé«âäœïŒ
bf16/fp16 > fp8_scaled > fp8 >> fp8_fast
T2V Inference / T2Væšè«
The following is an example of T2V inference (input as a single line):
python wan_generate_video.py --fp8 --task t2v-1.3B --video_size 832 480 --video_length 81 --infer_steps 20
--prompt "prompt for the video" --save_path path/to/save.mp4 --output_type both
--dit path/to/wan2.1_t2v_1.3B_bf16_etc.safetensors --vae path/to/wan_2.1_vae.safetensors
--t5 path/to/models_t5_umt5-xxl-enc-bf16.pth
--attn_mode torch
--task
is one of t2v-1.3B
, t2v-14B
, i2v-14B
, t2i-14B
(these are Wan2.1 official models), t2v-1.3B-FC
, t2v-14B-FC
and i2v-14B-FC
(for Wan2.1-Fun Control model).
--attn_mode
is torch
, sdpa
(same as torch
), xformers
, sageattn
,flash2
, flash
(same as flash2
) or flash3
. torch
is the default. Other options require the corresponding library to be installed. flash3
(Flash attention 3) is not tested.
Specifying --fp8
runs DiT in fp8 mode. fp8 can significantly reduce memory consumption but may impact output quality.
--fp8_scaled
can be specified in addition to --fp8
to run the model in fp8 weights optimization. This increases memory consumption and speed slightly but improves output quality. See here for details.
--fp8_fast
option is also available for faster inference on RTX 40x0 GPUs. This option requires --fp8_scaled
option. This option seems to degrade the output quality.
--fp8_t5
can be used to specify the T5 model in fp8 format. This option reduces memory usage for the T5 model.
--negative_prompt
can be used to specify a negative prompt. If omitted, the default negative prompt is used.
--flow_shift
can be used to specify the flow shift (default 3.0 for I2V with 480p, 5.0 for others).
--guidance_scale
can be used to specify the guidance scale for classifier free guidance (default 5.0).
--blocks_to_swap
is the number of blocks to swap during inference. The default value is None (no block swap). The maximum value is 39 for 14B model and 29 for 1.3B model.
--vae_cache_cpu
enables VAE cache in main memory. This reduces VRAM usage slightly but processing is slower.
--compile
enables torch.compile. See here for details.
--trim_tail_frames
can be used to trim the tail frames when saving. The default is 0.
--cfg_skip_mode
specifies the mode for skipping CFG in different steps. The default is none
(all steps).--cfg_apply_ratio
specifies the ratio of steps where CFG is applied. See below for details.
--include_patterns
and --exclude_patterns
can be used to specify which LoRA modules to apply or exclude during training. If not specified, all modules are applied by default. These options accept regular expressions.
--include_patterns
specifies the modules to be applied, and --exclude_patterns
specifies the modules to be excluded. The regular expression is matched against the LoRA key name, and include takes precedence.
The key name to be searched is in sd-scripts format (lora_unet_<module_name with dot replaced by _>
). For example, lora_unet_blocks_9_cross_attn_k
.
For example, if you specify --exclude_patterns "blocks_[23]\d_"
, it will exclude modules containing blocks_20
to blocks_39
. If you specify --include_patterns "cross_attn" --exclude_patterns "blocks_(0|1|2|3|4)_"
, it will apply LoRA to modules containing cross_attn
and not containing blocks_0
to blocks_4
.
If you specify multiple LoRA weights, please specify them with multiple arguments. For example: --include_patterns "cross_attn" ".*" --exclude_patterns "dummy_do_not_exclude" "blocks_(0|1|2|3|4)"
. ".*"
is a regex that matches everything. dummy_do_not_exclude
is a dummy regex that does not match anything.
--cpu_noise
generates initial noise on the CPU. This may result in the same results as ComfyUI with the same seed (depending on other settings).
If you are using the Fun Control model, specify the control video with --control_path
. You can specify a video file or a folder containing multiple image files. The number of frames in the video file (or the number of images) should be at least the number specified in --video_length
(plus 1 frame if you specify --end_image_path
).
Please try to match the aspect ratio of the control video with the aspect ratio specified in --video_size
(there may be some deviation from the initial image of I2V due to the use of bucketing processing).
Other options are same as hv_generate_video.py
(some options are not supported, please check the help).
æ¥æ¬èª
`--task` ã«ã¯ `t2v-1.3B`, `t2v-14B`, `i2v-14B`, `t2i-14B` ïŒãããã¯Wan2.1å ¬åŒã¢ãã«ïŒã`t2v-1.3B-FC`, `t2v-14B-FC`, `i2v-14B-FC`ïŒWan2.1-Fun Controlã¢ãã«ïŒãæå®ããŸãã--attn_mode
ã«ã¯ torch
, sdpa
ïŒtorch
ãšåãïŒãxformers
, sageattn
, flash2
, flash
ïŒflash2
ãšåãïŒ, flash3
ã®ãããããæå®ããŸããããã©ã«ã㯠torch
ã§ãããã®ä»ã®ãªãã·ã§ã³ã䜿çšããå Žåã¯ã察å¿ããã©ã€ãã©ãªãã€ã³ã¹ããŒã«ããå¿
èŠããããŸããflash3
ïŒFlash attention 3ïŒã¯æªãã¹ãã§ãã
--fp8
ãæå®ãããšDiTã¢ãã«ãfp8圢åŒã§å®è¡ããŸããfp8ã¯ã¡ã¢ãªæ¶è²»ã倧å¹
ã«åæžã§ããŸãããåºåå質ã«åœ±é¿ãäžããå¯èœæ§ããããŸãã
--fp8_scaled
ã --fp8
ãšäœµçšãããšãfp8ãžã®éã¿éååãè¡ããŸããã¡ã¢ãªæ¶è²»ãšé床ã¯ãããã«æªåããŸãããåºåå質ãåäžããŸãã詳ããã¯ãã¡ããåç
§ããŠãã ããã
--fp8_fast
ãªãã·ã§ã³ã¯RTX 40x0 GPUã§ã®é«éæšè«ã«äœ¿çšããããªãã·ã§ã³ã§ãããã®ãªãã·ã§ã³ã¯ --fp8_scaled
ãªãã·ã§ã³ãå¿
èŠã§ããåºåå質ãå£åããããã§ãã
--fp8_t5
ãæå®ãããšT5ã¢ãã«ãfp8圢åŒã§å®è¡ããŸããT5ã¢ãã«åŒã³åºãæã®ã¡ã¢ãªäœ¿çšéãåæžããŸãã
--negative_prompt
ã§ãã¬ãã£ãããã³ãããæå®ã§ããŸããçç¥ããå Žåã¯ããã©ã«ãã®ãã¬ãã£ãããã³ããã䜿çšãããŸãã
--flow_shift
ã§flow shiftãæå®ã§ããŸãïŒ480pã®I2Vã®å Žåã¯ããã©ã«ã3.0ããã以å€ã¯5.0ïŒã
--guidance_scale
ã§classifier free guianceã®ã¬ã€ãã³ã¹ã¹ã±ãŒã«ãæå®ã§ããŸãïŒããã©ã«ã5.0ïŒã
--blocks_to_swap
ã¯æšè«æã®block swapã®æ°ã§ããããã©ã«ãå€ã¯NoneïŒblock swapãªãïŒã§ããæå€§å€ã¯14Bã¢ãã«ã®å Žå39ã1.3Bã¢ãã«ã®å Žå29ã§ãã
--vae_cache_cpu
ãæå¹ã«ãããšãVAEã®ãã£ãã·ã¥ãã¡ã€ã³ã¡ã¢ãªã«ä¿æããŸããVRAM䜿çšéãå€å°æžããŸãããåŠçã¯é
ããªããŸãã
--compile
ã§torch.compileãæå¹ã«ããŸãã詳现ã«ã€ããŠã¯ãã¡ããåç
§ããŠãã ããã
--trim_tail_frames
ã§ä¿åæã«æ«å°Ÿã®ãã¬ãŒã ãããªãã³ã°ã§ããŸããããã©ã«ãã¯0ã§ãã
--cfg_skip_mode
ã¯ç°ãªãã¹ãããã§CFGãã¹ãããããã¢ãŒããæå®ããŸããããã©ã«ã㯠none
ïŒå
šã¹ãããïŒã--cfg_apply_ratio
ã¯CFGãé©çšãããã¹ãããã®å²åãæå®ããŸãã詳现ã¯åŸè¿°ããŸãã
LoRAã®ã©ã®ã¢ãžã¥ãŒã«ãé©çšããããã--include_patterns
ãš--exclude_patterns
ã§æå®ã§ããŸãïŒæªæå®æã»ããã©ã«ãã¯å
šã¢ãžã¥ãŒã«é©çšãããŸã
ïŒããããã®ãªãã·ã§ã³ã«ã¯ãæ£èŠè¡šçŸãæå®ããŸãã--include_patterns
ã¯é©çšããã¢ãžã¥ãŒã«ã--exclude_patterns
ã¯é©çšããªãã¢ãžã¥ãŒã«ãæå®ããŸããæ£èŠè¡šçŸãLoRAã®ããŒåã«å«ãŸãããã©ããã§å€æãããincludeãåªå
ãããŸãã
æ€çŽ¢å¯Ÿè±¡ãšãªãããŒå㯠sd-scripts 圢åŒïŒlora_unet_<ã¢ãžã¥ãŒã«åã®ãããã_ã«çœ®æãããã®>
ïŒã§ããäŸïŒlora_unet_blocks_9_cross_attn_k
ããšãã° --exclude_patterns "blocks_[23]\d_"
ã®ã¿ãæå®ãããšãblocks_20
ããblocks_39
ãå«ãã¢ãžã¥ãŒã«ãé€å€ãããŸãã--include_patterns "cross_attn" --exclude_patterns "blocks_(0|1|2|3|4)_"
ã®ããã«includeãšexcludeãæå®ãããšãcross_attn
ãå«ãã¢ãžã¥ãŒã«ã§ããã€blocks_0
ããblocks_4
ãå«ãŸãªãã¢ãžã¥ãŒã«ã«LoRAãé©çšãããŸãã
è€æ°ã®LoRAã®éã¿ãæå®ããå Žåã¯ãè€æ°åã®åŒæ°ã§æå®ããŠãã ãããäŸïŒ--include_patterns "cross_attn" ".*" --exclude_patterns "dummy_do_not_exclude" "blocks_(0|1|2|3|4)"
".*"
ã¯å
šãŠã«ãããããæ£èŠè¡šçŸã§ããdummy_do_not_exclude
ã¯äœã«ããããããªããããŒã®æ£èŠè¡šçŸã§ãã
--cpu_noise
ãæå®ãããšåæãã€ãºãCPUã§çæããŸããããã«ããåäžseedæã®çµæãComfyUIãšåãã«ãªãå¯èœæ§ããããŸãïŒä»ã®èšå®ã«ããããŸãïŒã
Fun Controlã¢ãã«ã䜿çšããå Žåã¯ã--control_path
ã§å¶åŸ¡çšã®æ åãæå®ããŸããåç»ãã¡ã€ã«ããŸãã¯è€æ°æã®ç»åãã¡ã€ã«ãå«ãã ãã©ã«ããæå®ã§ããŸããåç»ãã¡ã€ã«ã®ãã¬ãŒã æ°ïŒãŸãã¯ç»åã®ææ°ïŒã¯ã--video_length
ã§æå®ãããã¬ãŒã æ°ä»¥äžã«ããŠãã ããïŒåŸè¿°ã®--end_image_path
ãæå®ããå Žåã¯ãããã«+1ãã¬ãŒã ïŒã
å¶åŸ¡çšã®æ åã®ã¢ã¹ãã¯ãæ¯ã¯ã--video_size
ã§æå®ããã¢ã¹ãã¯ãæ¯ãšã§ãããããåãããŠãã ããïŒbucketingã®åŠçãæµçšããŠããããI2Vã®åæç»åãšãºã¬ãå ŽåããããŸãïŒã
ãã®ä»ã®ãªãã·ã§ã³ã¯ hv_generate_video.py
ãšåãã§ãïŒäžéšã®ãªãã·ã§ã³ã¯ãµããŒããããŠããªãããããã«ãã確èªããŠãã ããïŒã
CFG Skip Mode / CFGã¹ãããã¢ãŒã
These options allow you to balance generation speed against prompt accuracy. More skipped steps results in faster generation with potential quality degradation.
Setting --cfg_apply_ratio
to 0.5 speeds up the denoising loop by up to 25%.
--cfg_skip_mode
specified one of the following modes:
early
: Skips CFG in early steps for faster generation, applying guidance mainly in later refinement stepslate
: Skips CFG in later steps, applying guidance during initial structure formationmiddle
: Skips CFG in middle steps, applying guidance in both early and later stepsearly_late
: Skips CFG in both early and late steps, applying only in middle stepsalternate
: Applies CFG in alternate steps based on the specified rationone
: Applies CFG at all steps (default)
--cfg_apply_ratio
specifies a value from 0.0 to 1.0 controlling the proportion of steps where CFG is applied. For example, setting 0.5 means CFG will be applied in only 50% of the steps.
If num_steps is 10, the following table shows the steps where CFG is applied based on the --cfg_skip_mode
option (A means CFG is applied, S means it is skipped, --cfg_apply_ratio
is 0.6):
skip mode | CFG apply pattern |
---|---|
early | SSSSAAAAAA |
late | AAAAAASSSS |
middle | AAASSSSAAA |
early_late | SSAAAAAASS |
alternate | SASASAASAS |
The appropriate settings are unknown, but you may want to try late
or early_late
mode with a ratio of around 0.3 to 0.5.
æ¥æ¬èª
ãããã®ãªãã·ã§ã³ã¯ãçæé床ãšããã³ããã®ç²ŸåºŠã®ãã©ã³ã¹ãåãããšãã§ããŸããã¹ããããããã¹ããããå€ãã»ã©ãçæé床ãéããªããŸãããå質ãäœäžããå¯èœæ§ããããŸããratioã«0.5ãæå®ããããšã§ãããã€ãžã³ã°ã®ã«ãŒããæå€§25%çšåºŠãé«éåãããŸãã
--cfg_skip_mode
ã¯æ¬¡ã®ã¢ãŒãã®ãããããæå®ããŸãïŒ
early
ïŒåæã®ã¹ãããã§CFGãã¹ãããããŠãäž»ã«çµç€ã®ç²Ÿçްåã®ã¹ãããã§é©çšããŸãlate
ïŒçµç€ã®ã¹ãããã§CFGãã¹ãããããåæã®æ§é ãæ±ºãŸã段éã§é©çšããŸãmiddle
ïŒäžéã®ã¹ãããã§CFGãã¹ãããããåæãšçµç€ã®ã¹ãããã®äž¡æ¹ã§é©çšããŸãearly_late
ïŒåæãšçµç€ã®ã¹ãããã®äž¡æ¹ã§CFGãã¹ãããããäžéã®ã¹ãããã®ã¿é©çšããŸãalternate
ïŒæå®ãããå²åã«åºã¥ããŠCFGãé©çšããŸã
--cfg_apply_ratio
ã¯ãCFGãé©çšãããã¹ãããã®å²åã0.0ãã1.0ã®å€ã§æå®ããŸããããšãã°ã0.5ã«èšå®ãããšãCFGã¯ã¹ãããã®50%ã®ã¿ã§é©çšãããŸãã
å ·äœçãªãã¿ãŒã³ã¯äžã®ããŒãã«ãåç §ããŠãã ããã
é©åãªèšå®ã¯äžæã§ãããã¢ãŒãã¯late
ãŸãã¯early_late
ãratioã¯0.3~0.5çšåºŠãã詊ããŠã¿ããšè¯ããããããŸããã
Skip Layer Guidance
Skip Layer Guidance is a feature that uses the output of a model with some blocks skipped as the unconditional output of classifier free guidance. It was originally proposed in SD 3.5 and first applied in Wan2GP in this PR. It may improve the quality of generated videos.
The implementation of SD 3.5 is here, and the implementation of Wan2GP (the PR mentioned above) has some different specifications. This inference script allows you to choose between the two methods.
The SD3.5 method applies slg output in addition to cond and uncond (slows down the speed). The Wan2GP method uses only cond and slg output.
The following arguments are available:
--slg_mode
: Specifies the SLG mode.original
for SD 3.5 method,uncond
for Wan2GP method. Default is None (no SLG).--slg_layers
: Specifies the indices of the blocks (layers) to skip in SLG, separated by commas. Example:--slg_layers 4,5,6
. Default is empty (no skip). If this option is not specified,--slg_mode
is ignored.--slg_scale
: Specifies the scale of SLG whenoriginal
. Default is 3.0.--slg_start
: Specifies the start step of SLG application in inference steps from 0.0 to 1.0. Default is 0.0 (applied from the beginning).--slg_end
: Specifies the end step of SLG application in inference steps from 0.0 to 1.0. Default is 0.3 (applied up to 30% from the beginning).
Appropriate settings are unknown, but you may want to try original
mode with a scale of around 3.0 and a start ratio of 0.0 and an end ratio of 0.5, with layers 4, 5, and 6 skipped.
æ¥æ¬èª
Skip Layer Guidanceã¯ãäžéšã®blockãã¹ãããããã¢ãã«åºåãclassifier free guidanceã®unconditionalåºåã«äœ¿çšããæ©èœã§ããå ã ã¯[SD 3.5](https://github.com/comfyanonymous/ComfyUI/pull/5404)ã§ææ¡ããããã®ã§ãWan2.1ã«ã¯[Wan2GPã®ãã¡ãã®PR](https://github.com/deepbeepmeep/Wan2GP/pull/61)ã§åããŠé©çšãããŸãããçæåç»ã®å質ãåäžããå¯èœæ§ããããŸããSD 3.5ã®å®è£ ã¯ãã¡ãã§ãWan2GPã®å®è£ ïŒåè¿°ã®PRïŒã¯äžéšä»æ§ãç°ãªããŸãããã®æšè«ã¹ã¯ãªããã§ã¯äž¡è ã®æ¹åŒãéžæã§ããããã«ãªã£ãŠããŸãã
â»SD3.5æ¹åŒã¯condãšuncondã«å ããŠslg outputãé©çšããŸãïŒé床ãäœäžããŸãïŒãWan2GPæ¹åŒã¯condãšslg outputã®ã¿ã䜿çšããŸãã
以äžã®åŒæ°ããããŸãã
--slg_mode
ïŒSLGã®ã¢ãŒããæå®ããŸããoriginal
ã§SD 3.5ã®æ¹åŒãuncond
ã§Wan2GPã®æ¹åŒã§ããããã©ã«ãã¯Noneã§ãSLGã䜿çšããŸããã--slg_layers
ïŒSLGã§ã¹ãããããblock (layer)ã®ã€ã³ãã¯ã¹ãã«ã³ãåºåãã§æå®ããŸããäŸïŒ--slg_layers 4,5,6
ãããã©ã«ãã¯ç©ºïŒã¹ãããããªãïŒã§ãããã®ãªãã·ã§ã³ãæå®ããªããš--slg_mode
ã¯ç¡èŠãããŸãã--slg_scale
ïŒoriginal
ã®ãšãã®SLGã®ã¹ã±ãŒã«ãæå®ããŸããããã©ã«ãã¯3.0ã§ãã--slg_start
ïŒæšè«ã¹ãããã®SLGé©çšéå§ã¹ãããã0.0ãã1.0ã®å²åã§æå®ããŸããããã©ã«ãã¯0.0ã§ãïŒæåããé©çšïŒã--slg_end
ïŒæšè«ã¹ãããã®SLGé©çšçµäºã¹ãããã0.0ãã1.0ã®å²åã§æå®ããŸããããã©ã«ãã¯0.3ã§ãïŒæåãã30%ãŸã§é©çšïŒã
é©åãªèšå®ã¯äžæã§ãããoriginal
ã¢ãŒãã§ã¹ã±ãŒã«ã3.0çšåºŠãéå§å²åã0.0ãçµäºå²åã0.5çšåºŠã«èšå®ãã4, 5, 6ã®layerãã¹ãããããèšå®ããå§ãããšè¯ããããããŸããã
I2V Inference / I2Væšè«
The following is an example of I2V inference (input as a single line):
python wan_generate_video.py --fp8 --task i2v-14B --video_size 832 480 --video_length 81 --infer_steps 20
--prompt "prompt for the video" --save_path path/to/save.mp4 --output_type both
--dit path/to/wan2.1_i2v_480p_14B_bf16_etc.safetensors --vae path/to/wan_2.1_vae.safetensors
--t5 path/to/models_t5_umt5-xxl-enc-bf16.pth --clip path/to/models_clip_open-clip-xlm-roberta-large-vit-huge-14.pth
--attn_mode torch --image_path path/to/image.jpg
Add --clip
to specify the CLIP model. --image_path
is the path to the image to be used as the initial frame.
--end_image_path
can be used to specify the end image. This option is experimental. When this option is specified, the saved video will be slightly longer than the specified number of frames and will have noise, so it is recommended to specify --trim_tail_frames 3
to trim the tail frames.
You can also use the Fun Control model for I2V inference. Specify the control video with --control_path
.
Other options are same as T2V inference.
æ¥æ¬èª
`--clip` ã远å ããŠCLIPã¢ãã«ãæå®ããŸãã`--image_path` ã¯åæãã¬ãŒã ãšããŠäœ¿çšããç»åã®ãã¹ã§ãã--end_image_path
ã§çµäºç»åãæå®ã§ããŸãããã®ãªãã·ã§ã³ã¯å®éšçãªãã®ã§ãããã®ãªãã·ã§ã³ãæå®ãããšãä¿åãããåç»ãæå®ãã¬ãŒã æ°ãããããå€ããªãããã€ãã€ãºãä¹ãããã--trim_tail_frames 3
ãªã©ãæå®ããŠæ«å°Ÿã®ãã¬ãŒã ãããªãã³ã°ããããšããå§ãããŸãã
I2Væšè«ã§ãFun Controlã¢ãã«ã䜿çšã§ããŸãã--control_path
ã§å¶åŸ¡çšã®æ åãæå®ããŸãã
ãã®ä»ã®ãªãã·ã§ã³ã¯T2Væšè«ãšåãã§ãã
New Batch and Interactive Modes / æ°ãããããã¢ãŒããšã€ã³ã¿ã©ã¯ãã£ãã¢ãŒã
In addition to single video generation, Wan 2.1 now supports batch generation from file and interactive prompt input:
Batch Mode from File / ãã¡ã€ã«ããã®ãããã¢ãŒã
Generate multiple videos from prompts stored in a text file:
python wan_generate_video.py --from_file prompts.txt --task t2v-14B
--dit path/to/model.safetensors --vae path/to/vae.safetensors
--t5 path/to/t5_model.pth --save_path output_directory
The prompts file format:
- One prompt per line
- Empty lines and lines starting with # are ignored (comments)
- Each line can include prompt-specific parameters using command-line style format:
A beautiful sunset over mountains --w 832 --h 480 --f 81 --d 42 --s 20
A busy city street at night --w 480 --h 832 --g 7.5 --n low quality, blurry
Supported inline parameters (if ommitted, default values from the command line are used):
--w
: Width--h
: Height--f
: Frame count--d
: Seed--s
: Inference steps--g
or--l
: Guidance scale--fs
: Flow shift--i
: Image path (for I2V)--cn
: Control path (for Fun Control)--n
: Negative prompt
In batch mode, models are loaded once and reused for all prompts, significantly improving overall generation time compared to multiple single runs.
Interactive Mode / ã€ã³ã¿ã©ã¯ãã£ãã¢ãŒã
Interactive command-line interface for entering prompts:
python wan_generate_video.py --interactive --task t2v-14B
--dit path/to/model.safetensors --vae path/to/vae.safetensors
--t5 path/to/t5_model.pth --save_path output_directory
In interactive mode:
- Enter prompts directly at the command line
- Use the same inline parameter format as batch mode
- Use Ctrl+D (or Ctrl+Z on Windows) to exit
- Models remain loaded between generations for efficiency
æ¥æ¬èª
åäžåç»ã®çæã«å ããŠãWan 2.1ã¯çŸåšããã¡ã€ã«ããã®ãããçæãšã€ã³ã¿ã©ã¯ãã£ããªããã³ããå ¥åããµããŒãããŠããŸãããã¡ã€ã«ããã®ãããã¢ãŒã
ããã¹ããã¡ã€ã«ã«ä¿åãããããã³ããããè€æ°ã®åç»ãçæããŸãïŒ
python wan_generate_video.py --from_file prompts.txt --task t2v-14B
--dit path/to/model.safetensors --vae path/to/vae.safetensors
--t5 path/to/t5_model.pth --save_path output_directory
ããã³ãããã¡ã€ã«ã®åœ¢åŒïŒ
- 1è¡ã«1ã€ã®ããã³ãã
- 空è¡ã#ã§å§ãŸãè¡ã¯ç¡èŠãããŸãïŒã³ã¡ã³ãïŒ
- åè¡ã«ã¯ã³ãã³ãã©ã€ã³åœ¢åŒã§ããã³ããåºæã®ãã©ã¡ãŒã¿ãå«ããããšãã§ããŸãïŒ
ãµããŒããããŠããã€ã³ã©ã€ã³ãã©ã¡ãŒã¿ïŒçç¥ããå Žåãã³ãã³ãã©ã€ã³ã®ããã©ã«ãå€ã䜿çšãããŸãïŒ
--w
: å¹--h
: é«ã--f
: ãã¬ãŒã æ°--d
: ã·ãŒã--s
: æšè«ã¹ããã--g
ãŸãã¯--l
: ã¬ã€ãã³ã¹ã¹ã±ãŒã«--fs
: ãããŒã·ãã--i
: ç»åãã¹ïŒI2VçšïŒ--cn
: ã³ã³ãããŒã«ãã¹ïŒFun ControlçšïŒ--n
: ãã¬ãã£ãããã³ãã
ãããã¢ãŒãã§ã¯ãã¢ãã«ã¯äžåºŠã ãããŒãããããã¹ãŠã®ããã³ããã§åå©çšããããããè€æ°åã®åäžå®è¡ãšæ¯èŒããŠå šäœçãªçææéãå€§å¹ ã«æ¹åãããŸãã
ã€ã³ã¿ã©ã¯ãã£ãã¢ãŒã
ããã³ãããå ¥åããããã®ã€ã³ã¿ã©ã¯ãã£ããªã³ãã³ãã©ã€ã³ã€ã³ã¿ãŒãã§ãŒã¹ïŒ
python wan_generate_video.py --interactive --task t2v-14B
--dit path/to/model.safetensors --vae path/to/vae.safetensors
--t5 path/to/t5_model.pth --save_path output_directory
ã€ã³ã¿ã©ã¯ãã£ãã¢ãŒãã§ã¯ïŒ
- ã³ãã³ãã©ã€ã³ã§çŽæ¥ããã³ãããå ¥å
- ãããã¢ãŒããšåãã€ã³ã©ã€ã³ãã©ã¡ãŒã¿åœ¢åŒã䜿çš
- çµäºããã«ã¯ Ctrl+D (Windowsã§ã¯ Ctrl+Z) ã䜿çš
- å¹çã®ãããã¢ãã«ã¯çæéã§èªã¿èŸŒãŸãããŸãŸã«ãªããŸã