File size: 2,339 Bytes
faa7d83 7d4f327 faa7d83 2277f75 3a31b44 c2131cf 3a31b44 dcf9560 be4c548 43cbd86 be4c548 dcf9560 3a31b44 e6fec9d 3a31b44 0de1736 3a31b44 f29e281 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 |
---
tags:
- text-to-image
library_name: diffusers
license: apache-2.0
---
# DMM: Building a Versatile Image Generation Model via Distillation-Based Model Merging
<div style="text-align: center;">
<a href="https://arxiv.org/abs/2504.12364"><img src="https://img.shields.io/badge/arXiv-2504.12364-b31b1b.svg" alt="arXiv"></a>
<a href="https://huggingface.co/papers/2504.12364"><img src="https://huggingface.co/datasets/huggingface/badges/resolve/main/paper-page-sm.svg" alt="Paper page"></a>
</div>
## Introduction
We propose a score distillation based model merging paradigm DMM, compressing multiple models into a single versatile T2I model.
This checkpoint merges pre-trained models from many different domains, including *realistic style, Asian portrait, anime style, illustration, etc*.
Specifically, the source models are listed below:
- [JuggernautReborn](https://civitai.com/models/46422)
- [MajicmixRealisticV7](https://civitai.com/models/43331)
- [EpicRealismV5](https://civitai.com/models/25694)
- [RealisticVisionV5](https://civitai.com/models/4201)
- [MajicmixFantasyV3](https://civitai.com/models/41865)
- [MinimalismV2](https://www.liblib.art/modelinfo/8b4b7eb6aa2c480bbe65ca3d4625632d?from=personal_page&versionUuid=4b8e98cc17fc49ed826af941060ffd0b)
- [RealCartoon3dV17](https://civitai.com/models/94809)
- [AWPaintingV1.4](https://civitai.com/models/84476)
## Visualization
### Results
### Results combined with charactor LoRA
### Results of interpolation between two styles
## Online Demo
https://huggingface.co/spaces/MCG-NJU/DMM .
## Usage
Please refer to https://github.com/MCG-NJU/DMM .
```python
import torch
from modeling.dmm_pipeline import StableDiffusionDMMPipeline
pipe = StableDiffusionDMMPipeline.from_pretrained("path/to/pipeline/checkpoint", torch_dtype=torch.float16, use_safetensors=True)
pipe = pipe.to("cuda")
# select model index
model_id = 5
output = pipe(
prompt="portrait photo of a girl, long golden hair, flowers, best quality",
negative_prompt="worst quality,low quality,normal quality,lowres,watermark,nsfw",
width=512,
height=512,
num_inference_steps=25,
guidance_scale=7,
model_id=model_id,
).images[0]
``` |