Skip to content

fix(qwen): preserve checkpoint exports and bound QSA memory - #202

Draft
dingzhiqiang wants to merge 3 commits into
modelscope:mainfrom
dingzhiqiang:fix/qwen-ple-export-and-qsa-memory
Draft

dingzhiqiang wants to merge 3 commits into
modelscope:mainfrom
dingzhiqiang:fix/qwen-ple-export-and-qsa-memory

Conversation

@dingzhiqiang

@dingzhiqiang dingzhiqiang commented Sep 18, 2026

Copy link
Copy Markdown

Qwen PLE tables trained in BF16 must export their updated values without restoring stale FP8 scales. This change preserves trainable-table precision, chunks unpacked QSA scoring to bound workspace memory, and releases consumed PLE backward temporaries earlier.

The bridge also owns Qwen-specific checkpoint validation and export completion through mcore_bridge.utils.qwen4_exp_checkpoint: validate PLE assets, restore fixed vision weights without overwriting trained text weights, and produce export configuration consistent with MTP. These helpers and their tests move out of AReaL; AReaL retains distributed save coordination and its engine adapter. Integration: areal-project/AReaL#1731.

Validation:

  • Migrated checkpoint tests and AReaL adapter/save-coordination tests: 69 passed against this bridge revision.
  • Earlier GPU run for PLE export and QSA: 18 passed (tests/test_ple_checkpoint.py tests/test_ple_fp8_scale.py tests/test_qsa_indexer.py -k "not peak_memory"). Peak-memory benchmark excluded to avoid competing with active training.
  • Full repository pre-commit checks and commit hooks passed.
  • Multi-node SWE training and CP>1 runtime validation of the final bridge revision remain pending; earlier training used a different revision.

@dingzhiqiang dingzhiqiang changed the title fix(qwen): preserve PLE export precision and bound QSA memory fix(qwen): preserve checkpoint exports and bound QSA memory Sep 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant