qwen image 2.1 lora/full 训出来的模型 出图全是噪声

[sjf18]

同一批数据 qwen image edit 2511上训练没问题，迁移到qwen image 2.1 无论是lora 还是full ,训出来的模型推理都是噪声，而且loss 非常不稳定，越来越大，都到几十了，lr调小了训也没用，换批数据训也没用，有人遇到同样问题吗，噪声图如下

[mi804]

感谢反馈！我们目前还未能复现这个问题，使用 Qwen-Image-2.1.sh 中的编辑训练脚本进行训练，没有遇到输出类似噪声的情况。

如果方便，您可以提供少量示例训练数据、训练和推理脚本，以及 Python 环境中主要依赖的具体版本，例如 transformers、PyTorch 等，我们可以帮助进一步排查问题。

另外，也可以一并说明在本地使用原始模型、不加载微调权重时，图像编辑推理是否正常，这有助于缩小排查范围。

[gobigrassland]

@sjf18 @mi804 我也遇到了这个问题。lora微调后都是噪音，没有微调推理都是正常的

[sjf18]

感谢反馈！我们目前还未能复现这个问题，使用 Qwen-Image-2.1.sh 中的编辑训练脚本进行训练，没有遇到输出类似噪声的情况。

如果方便，您可以提供少量示例训练数据、训练和推理脚本，以及 Python 环境中主要依赖的具体版本，例如 transformers、PyTorch 等，我们可以帮助进一步排查问题。

另外，也可以一并说明在本地使用原始模型、不加载微调权重时，图像编辑推理是否正常，这有助于缩小排查范围。

@mi804 不加训练的lora推理一切正常，1卡B200，cuda12.8, python3.10，torch 2.8, transformers 4.57.1, accelerate 1.10.1
样例数据，具体环境，训出来的权重在链接里，脚本用的就是Qwen-Image-2.1.sh，仅替换本地模型和数据，训练过程中loss 越来越大。
只有lr 非常小 <1e-7，loss才会稳定在一开始的水平，训出来的模型才不会出噪声，但太小了训练完也学不到什么，没啥效果
https://pan.quark.cn/s/2540f40cd1b9
@gobigrassland 一模一样的问题，你方便也分享一下你的配置和数据样例吗，看看有什么共通点

[gobigrassland]

@sjf18 我整理好后，发出来～

[gobigrassland]

@mi804 @sjf18 我已经复现了Lora训练后这个异常输出。

• https://drive.google.com/file/d/1DQyJJ6dUcnJYV3XK2K06GMCEXLYSEocY/view?usp=sharing 下载并解压，将测试数据与测试脚本放到根目录下
• 没有改变代码，commit=7539a33b16844e2ce7e06306a2576346aa00de2b，只是新增测试数据和脚步

scripts_inf
|-- Qwen-Image-2.1_debug1.py 基于qwen2511数据进行推理，没有问题
|-- Qwen-Image-2.1_debug_shoes_v1.py 基于鞋靴单条数据lora训练后推理，没有问题
`-- Qwen-Image-2.1_debug_shoes_v2.py 基于鞋靴10条数据lora训练后推理，就异常输出

scripts_train
|-- Qwen-Image-2.1_debug_1k.sh 基于qwen2511-单条样例数据进行lora训练
|-- Qwen-Image-2.1_debug_shoes_2k_one_sample.sh 基于鞋靴单条数据进行lora训练
`-- Qwen-Image-2.1_debug_shoes_2k_ten_samples.sh 基于鞋靴10条数据lora训练

data/diffsynth_example_dataset
|-- dataset_infos.json
|-- flux2
|-- qwen_image
|-- qwen_image_21
|-- README.md
`-- shoes

原权重推理	1个样本lora训练后推理	10个样本lora训练后推理

• 环境

点击展开/折叠查看依赖库及版本

``` absl-py==2.3.1 accelerate==1.15.0 addict==2.4.0 aether==0.1.3.post1 aiobotocore==2.17.0 aiofiles==24.1.0 aiohappyeyeballs==2.6.1 aiohttp==3.13.2 aiohttp-cors==0.8.1 aioitertools==0.13.0 aiosignal==1.4.0 airecord-io-fastckpt @ http://storage.jd.local/bmp-packages/airecord/airecord-io/dev/airecord_io_fastckpt-1.0.2-py3-none-any.whl airecord-io-torch-gpu @ http://storage.jd.local/bmp-packages/airecord/airecord-io/release/airecord_io_torch_gpu-1.1.0.post97919065-cp310-cp310-linux_x86_64.whl aniso8601==10.0.1 annotated-types==0.7.0 antlr4-python3-runtime==4.9.3 anyio==4.12.0 apex @ file:///tmp/apex asciitree==0.3.3 astor==0.8.1 astunparse==1.6.3 async-timeout==5.0.1 attrdict==2.0.1 attrs==25.4.0 audioread==3.1.0 autoawq==0.2.7.post3 av==13.1.0 badam==1.2.2 bitarray==3.8.0 bitsandbytes==0.49.0 bitstring==4.3.1 blake3==1.0.8 blessed==1.25.0 blinker==1.9.0 boltons @ file:///croot/boltons_1677628692245/work boto3==1.35.54 botocore==1.35.93 braceexpand==0.1.7 bracex==2.6 brotli==1.2.0 brotlipy==0.7.0 cachetools==6.2.3 cbor2==5.7.1 certifi==2025.11.12 cffi==2.1.1 chardet==5.2.0 charset-normalizer==3.4.4 click==8.5.0 cloudpickle==3.1.2 cmake==4.2.0 codetiming==1.4.0 colorama==0.4.6 colorful==0.5.8 compressed-tensors==0.11.0 conda @ file:///croot/conda_1689269889729/work conda-content-trust @ file:///tmp/abs_5952f1c8-355c-4855-ad2e-538535021ba5h26t22e5/croots/recipe/conda-content-trust_1658126371814/work conda-libmamba-solver @ file:///croot/conda-libmamba-solver_1685032319139/work/src conda-package-handling @ file:///croot/conda-package-handling_1685024767917/work conda-package-streaming @ file:///croot/conda-package-streaming_1685019673878/work confluent-kafka==2.12.2 contourpy==1.3.2 coverage==7.13.0 cpm-kernels==1.0.11 crc32c==2.8 cryptography==50.0.1 cupy-cuda12x==13.6.0 cycler==0.12.1 dacite==1.8.1 dataproperty==1.1.0 datasets==3.5.0 decorator==5.2.1 decord==0.6.0 deepspeed==0.16.9 deprecated==1.3.1 depyf==0.19.0 diffusers==0.35.1 dill==0.3.8 diskcache==5.6.3 distlib==0.4.0 distro==1.9.0 dlrover==0.4.0 dnspython==2.8.0 docstring-parser==0.16 easydict==1.13 ebmlite==3.4.1 einops==0.8.0 elastic-transport==8.17.1 elasticsearch==8.15.1 email-validator==2.3.0 et-xmlfile==2.0.0 evaluate==0.4.6 exceptiongroup==1.3.1 fairscale==0.4.13 faiss-cpu==1.13.1 fastapi==0.115.4 fastapi-cli==0.0.16 fastapi-cloud-cli==0.6.0 fastar==0.8.0 fasteners==0.20 fastrlock==0.8.3 fbgemm-gpu==1.3.0+cu128 ffmpy==1.0.0 filelock==3.20.0 filetype==1.2.0 fire==0.7.1 flagembedding==1.3.5 flash-attn==2.7.4.post1 flask==3.1.0 flask-cors==5.0.1 flask-restful==0.3.10 fonttools==4.61.1 frozendict==2.4.7 frozenlist==1.8.0 fsspec==2024.12.0 ftfy==6.3.1 gguf==0.17.1 gitdb==4.0.12 gitpython==3.1.43 google-api-core==2.28.1 google-auth==2.43.0 googleapis-common-protos==1.72.0 gpustat==1.1.1 gputil==1.4.0 gradio==5.50.0 gradio-client==1.14.0 groovy==0.1.2 grouped-gemm @ file:///tmp/grouped_gemm-1.1.4-cp310-cp310-linux_x86_64.whl grpcio==1.62.1 grpcio-tools==1.62.1 h11==0.16.0 hf-xet==1.6.0 hjson==3.1.0 hstu-attn @ file:///hstu_attn-0.1.0%2Bcu12.8-cp310-cp310-linux_x86_64.whl hstu-hopper @ file:///hstu_hopper-0.1.0%2Bcu12.8-cp310-cp310-linux_x86_64.whl httpcore==1.0.9 httptools==0.7.1 httpx==0.28.1 huggingface-hub==0.36.2 hydra-core==1.3.2 idna==3.11 imageio==2.38.0 imageio-ffmpeg==0.6.0 importlib-metadata==8.7.0 iniconfig==2.3.0 interegular==0.3.3 iopath==0.1.10 itsdangerous==2.2.0 jieba==0.42.1 jinja2==3.1.6 jiter==0.12.0 jmespath==1.0.1 joblib==1.5.3 jsonlines==4.0.0 jsonpatch @ file:///tmp/build/80754af9/jsonpatch_1615747632069/work jsonpointer==2.1 jsonschema==4.25.1 jsonschema-specifications==2025.9.1 kafka-python==2.0.2 kiwisolver==1.4.9 lark==1.2.2 latex2mathml==3.78.1 lazy-loader==0.4 libmambapy @ file:///croot/mamba-split_1685993156657/work/libmambapy librosa==0.11.0 lightning-utilities==0.15.2 llguidance==0.7.30 llvmlite==0.44.0 lm-eval @ file:///opt/lm-evaluation-harness lm-format-enforcer==0.11.3 lmdeploy==0.7.0.post3 lxml==6.0.2 magnus-api @ file:///tmp/magnus-api -e file:///opt/magnus-t1 markdown==3.10 markdown-it-py==4.0.0 markupsafe==3.0.3 mathruler==0.1.0 matplotlib==3.10.8 mbstrdecoder==1.1.4 mdtex2html==1.3.0 mdurl==0.1.2 megatron-core @ file:///opt/Megatron-LM megatron-energon==6.0.1 mistral-common==1.8.6 ml-dtypes==0.5.4 mmengine==0.10.7 mmengine-lite==0.10.7 mmh3==5.0.1 modelscope==1.40.1 modelscope-hub==0.4.5 more-itertools==10.8.0 mpi4py==4.1.1 mpmath==1.3.0 msgpack==1.1.2 msgspec==0.20.0 multi-storage-client==0.38.0 multidict==6.7.0 multiprocess==0.70.16 mypy-extensions==1.1.0 natsort==8.4.0 networkx==3.4.2 ninja==1.13.0 nltk==3.9.1 numba==0.61.2 numcodecs==0.13.1 numexpr==2.14.1 numpy==1.26.4 nvidia-cublas-cu12==12.8.4.1 nvidia-cuda-cupti-cu12==12.8.90 nvidia-cuda-nvrtc-cu12==12.8.93 nvidia-cuda-runtime-cu12==12.8.90 nvidia-cudnn-cu12==9.10.2.21 nvidia-cufft-cu12==11.3.3.83 nvidia-cufile-cu12==1.13.1.3 nvidia-curand-cu12==10.3.9.90 nvidia-cusolver-cu12==11.7.3.90 nvidia-cusparse-cu12==12.5.8.93 nvidia-cusparselt-cu12==0.7.1 nvidia-ml-py==12.575.51 nvidia-nccl-cu12==2.27.3 nvidia-nvjitlink-cu12==12.8.93 nvidia-nvtx-cu12==12.8.90 nvtx==0.2.14 omegaconf==2.3.0 onnx==1.19.0 openai==2.11.0 openai-harmony==0.0.4 opencensus==0.11.4 opencensus-context==0.1.3 opencv-python==4.10.0.84 opencv-python-headless==4.11.0.86 openpyxl==3.1.5 opentelemetry-api==1.39.1 opentelemetry-exporter-prometheus==0.60b1 opentelemetry-proto==1.27.0 opentelemetry-sdk==1.39.1 opentelemetry-semantic-conventions==0.60b1 optimum==2.0.0 orjson==3.11.5 outlines-core==0.2.11 oxygenrl @ file:///oxygenrl-0.1.0-py3-none-any.whl packaging==25.0 pandas==2.3.3 parameterized==0.9.0 partial-json-parser==0.2.1.1.post7 pathvalidate==3.3.1 peft==0.18.0 pillow==11.3.0 pip==23.1.2 platformdirs==4.5.1 pluggy==1.6.0 pooch==1.8.2 portalocker==3.2.0 prettytable==3.17.0 prometheus-client==0.22.1 prometheus-fastapi-instrumentator==7.1.0 propcache==0.4.1 proto-plus==1.26.1 protobuf==4.25.3 psutil==7.1.3 pulp==2.9.0 py-cpuinfo==9.0.0 py-spy==0.4.1 pyarrow==20.0.0 pyasn1==0.6.1 pyasn1-modules==0.4.2 pybase64==1.4.2 pybind11==3.0.1 pycosat @ file:///croot/pycosat_1666805502580/work pycountry==24.6.1 pycparser==3.0 pydantic==2.11.7 pydantic-core==2.33.2 pydantic-extra-types==2.10.6 pydub==0.25.1 pygments==2.19.2 pylatexenc==2.10 pynvml==12.0.0 pyopenssl==26.4.0 pyparsing==3.2.5 pyre-extensions==0.0.32 pysocks @ file:///home/builder/ci_310/pysocks_1640793678128/work pytablewriter==1.2.1 pytest==8.4.1 pytest-cov==6.2.1 pytest-html==4.1.1 pytest-metadata==3.1.1 pytest-mock==3.15.1 pytest-random-order==1.2.0 python-dateutil==2.9.0.post0 python-dotenv==1.2.1 python-json-logger==4.0.0 python-multipart==0.0.20 pytz==2025.2 pyvers==0.1.0 pyyaml==6.0.3 pyzmq==27.1.0 qwen-vl-utils==0.0.8 ray==2.50.0 referencing==0.37.0 regex==2025.11.3 requests==2.32.5 rich==14.2.0 rich-toolkit==0.17.0 rignore==0.7.6 rouge-chinese==1.0.3 rouge-score==0.1.2 rpds-py==0.30.0 rsa==4.9.1 ruamel-yaml @ file:///croot/ruamel.yaml_1666304550667/work ruamel-yaml-clib @ file:///croot/ruamel.yaml.clib_1666302247304/work ruff==0.14.6 s3fs==2024.12.0 s3transfer==0.10.4 sacrebleu==2.5.1 safehttpx==0.1.7 safetensors==0.8.0 scikit-build==0.18.1 scikit-learn==1.7.2 scipy==1.15.3 seaborn==0.13.2 semantic-version==2.10.0 sentence-transformers==3.2.1 sentencepiece==0.2.1 sentry-sdk==2.47.0 setproctitle==1.3.7 setuptools==80.9.0 shellingham==1.5.4 shortuuid==1.0.13 shtab==1.7.1 six==1.17.0 smart-open==7.5.0 smmap==5.0.2 sniffio==1.3.1 sortedcontainers==2.4.0 soundfile==0.13.1 soxr==1.0.0 sqlitedict==2.1.0 sse-starlette==2.1.3 starlette==0.41.3 swissarmytransformer==0.4.12 sympy==1.14.0 tabledata==1.3.4 tabulate==0.9.0 tcolorpy==0.1.7 tenacity==9.0.0 tensorboard==2.20.0 tensorboard-data-server==0.7.2 tensorboardx==2.6.4 tensordict==0.10.0 tensorstore==0.1.78 termcolor==3.2.0 tfrecord==1.14.6 threadpoolctl==3.6.0 tiktoken==0.7.0 tilelang==0.1.6.post2 timm==1.0.11 tokenizers==0.22.2 toml==0.10.2 tomli==2.3.0 tomlkit==0.13.3 toolz @ file:///croot/toolz_1667464077321/work torch==2.8.0+cu128 torchaudio==2.8.0+cu128 torchdata==0.11.0 torchmetrics==1.0.3 torchprofile==0.0.4 torchrec==1.3.0+cu128 torchvision==0.23.0+cu128 tqdm==4.67.1 tqdm-multiprocess==0.0.11 transformer-engine==2.3.0 transformer-engine-cu12==2.3.0 transformer-engine-torch==2.3.0 transformers==4.57.1 transformers-stream-generator==0.0.5 triton==3.4.0 trl==0.26.1 typepy==1.3.4 typer==0.20.0 typing-extensions==4.16.0 typing-inspect==0.9.0 typing-inspection==0.4.2 tyro==0.8.14 tzdata==2025.3 uniattention @ http://storage.jd.local/bmp-packages/torch/uniattention/uniattention-0.4.0-cp310-cp310-linux_x86_64.whl unirec @ http://storage.jd.local/bmp-packages/tritium-dl/unirec/torch2.8/cuda12.8/unirec-1.7.0.post97913388-cp310-cp310-linux_x86_64.whl unsloth==2024.9.post3 unsloth-zoo==2024.11.4 urllib3==2.8.0 uv==0.8.17 uvicorn==0.38.0 uvloop==0.22.1 verl==0.6.1 virtualenv==20.33.1 vllm==0.11.0 vlmeval @ file:///opt/VLMEvalKit wandb==0.23.1 watchfiles==1.1.1 wcmatch==10.1 wcwidth==0.2.14 webdataset==1.0.2 websockets==15.0.1 werkzeug==3.1.4 wheel==0.45.1 word2number==1.1 wrapt==1.17.3 xattr==1.3.0 xformers==0.0.32.post1 xgrammar==0.1.25 xxhash==3.6.0 yapf==0.43.0 yarl==1.22.0 zarr==2.18.3 zipp==3.23.0 zstandard==0.25.0 ```

@mi804 看看能否帮助你定位具体问题

[HUXING8]

有针对人像图像编辑任务训练Qwen-Image-2.1的lora吗？之前使用FLUX.2 Klein 9B进行训练的任务放在Qwen-Image-2.1.sh的脚本进行训练，背景、人脸一致性效果都不怎么好。默认的1e-4学习率容易出脏图，调整到5e-6稍微好一些，但人像表现仍然不足。