[Fix][Relax][Frontend][Torch] Honor the dtype argument of aten.mean (torch.Tensor.mean / torch.mean) - #20241
Open
siyiweigeHEW wants to merge 1 commit into
Open
Conversation
PyTorch's `Tensor.mean` accepts an optional keyword-only `dtype` argument that controls both the accumulation and the output type. `torch.export` preserves it on the `aten.mean.dim` / `aten.mean.default` nodes, but the `_mean` converter never reads `node.kwargs["dtype"]`, so the argument is silently dropped and the output keeps the input dtype (e.g. fp32 mean(dtype=fp64) returns fp32). Match the existing `_sum` handling: when `dtype` is given, cast the input to the requested dtype with `relax.op.astype` before reducing. `relax.op.mean` has no `out_dtype`, so the cast-then-reduce form also matches PyTorch's documented semantics (cast input, then accumulate).
aten.diagonal from decomposed repeated-subscript einsumdtype argument of aten.mean (torch.Tensor.mean / torch.mean)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes: #20230
Summary
PyTorch's
Tensor.mean(and the equivalenttorch.mean) accepts anoptional keyword-only
dtypeargument that controls both theaccumulation type and the output type.
torch.exportpreserves it on theaten.mean.dim/aten.mean.defaultnodes, but the_meanconverter inbase_fx_graph_translator.pynever readsnode.kwargs["dtype"], so theargument is silently dropped and the output keeps the input dtype. For
example,
x.mean(dim=1, dtype=torch.float64)on an fp32xreturnsfloat32instead offloat64, and in the fp16→fp32 direction the valuesare also accumulated at the wrong (lower) precision.
This PR makes
_meanhonordtypeby casting the input to the requesteddtype before reducing — the same pattern
_sumalready uses.Root cause
_mean(python/tvm/relax/frontend/torch/base_fx_graph_translator.py:1646)only reads
dimandkeepdim:relax.op.mean(python/tvm/relax/op/statistical.py:54) has noout_dtypeparameter, so the requested dtype cannot be propagated.torch.exportdoes preserve thedtypekwarg on the node:The same converter is registered for
mean.dim/mean.default(exported-program path) and for
mean(fx-trace path), so both entrypoints are affected.
Fix
When
dtypeis present on the node, cast the input to the requested dtypebefore emitting the mean, mirroring the existing
_sumhandling:Casting the input and then reducing is exactly PyTorch's documented
semantics: the
dtypeargument controls the accumulation type (the inputis converted to
dtypebefore the reduction), so no precision is lostbetween the cast and the mean.
Validation
In-tree regression test
Extended
test_meanintests/python/relax/test_frontend_from_fx.pywithMeanDtype, an fx-traced module callinginput.mean(-1, dtype=torch.float64), and its expected IR: anR.astypeto"float64"followed by
R.meanproducing afloat64output. The produced module ischecked with
tvm.ir.assert_structural_equalvia the file'sverify_modelhelper.
Differential test
The prove_hum differential harness (
1复现_torch_mean.py, convertingtorch.exportoutput withfrom_exported_programand comparing againstnative PyTorch 2.10.0) was run on a runnable build of this change — 15
cases:
negative-axis
meanwithoutdtype— unchanged, all match nativePyTorch (dtype, shape, values).
mean(dtype=fp32)on fp32 — matches.dtype != input(6):fp32→fp64,fp64→fp32,fp16→fp32,fp32→fp16, including global and multi-dim + keepdim variants.All 6
dtype != inputcases now produce the requested output dtype inevery direction (pre-fix, all 6 returned the input dtype). Values match to
native PyTorch within fp16/fp32 rounding precision. Both converter entry
paths —
from_exported_programandfrom_fx— were verified.Files changed
python/tvm/relax/frontend/torch/base_fx_graph_translator.py—_meanreads
node.kwargs["dtype"]and casts the input withrelax.op.astypebefore emittingrelax.op.mean.tests/python/relax/test_frontend_from_fx.py— addMeanDtype/ExpectedDtypecoverage totest_mean.