YawNet

Full-circle head-yaw regression via biternion (cos/sin) + von Mises κ, DINOv3 teacher-student distillation on a purely synthetic 42k dataset.

An end-to-end pipeline — dataset construction, training, distillation and ONNX export — for 360° head-yaw estimation (biternion cos/sin regression) from synthetic data.

• Demo: https://github.com/PINTO0309/High-Angle_Robust_Fast_FaceAlignment

  output_yawnet.mp4

• Verification sample image

  

1. Motivation and design principles

• Most of the public datasets used in the existing head-pose literature impose strict non-commercial / research-only terms, which makes them practically unusable for real-world products.
• None of those public datasets covers the head orientations that actually occur in the real world — rear views above all.
• yawpose instead ships with the real-world noise already synthesized in (low resolution, blur, occlusion, compression), and covers a wide range of camera angles, head pitch, and the full 360° yaw circle. The synthesis also mass-produces samples with occluding items and appearance variation — eyeglasses, sunglasses, face masks, hoods, with/without hair, with/without beards, earrings, necklaces and so on — across genders and age groups.
• Training and validation use purely synthetic data only, so no dataset licensing problem can arise (the dataset itself is CC BY 4.0, §3).
• Every unnecessary parameter is cut away and the model specializes in the bare minimum — full-circle yaw only; in exchange, the perceived accuracy of the estimated yaw direction is high.

2. Setup (uv + venv)

Package versions are fully pinned by pyproject.toml / uv.lock (the same versions as High-Angle_Robust_Fast_FaceAlignment).

uv sync --frozen                                      # onnxruntime-gpu 1.22.0 (default)
# For TensorRT EP validation:
# uv sync --frozen --no-group ort --group tensorrt    # onnxruntime-gpu 1.26.0
uv run python scripts/build_yawpose_dataset.py

3. Data layout

This repository contains no actual data (data/ is empty). Everything needed to reproduce or run the pipeline is downloadable from the resources release: the yawpose dataset (yawpose.tar.gz → data/yawpose/), the helper ONNX models for models/ (DEIMv2 head detectors, sixdrepnet360, HRFFA landmarks, Depth Anything V2), and the trained runs / exported ONNX (yawnet_distill_*, dinov3_vitl16_320_unified_teacher_v6u.tar.gz → runs/). Place the following to use it:

• data/synthetic_001..006/ — the original synthetic data (full images + metadata)
• data/yawpose/ — the unified dataset generated by scripts/build_yawpose_dataset.py (320x320 head crops, train:val = 9:1). See docs/yawpose_dataset.md for the spec and the full label-fix history. The dataset itself is licensed under CC BY 4.0 (LICENSE / NOTICE are bundled with the dataset)
• models/ — inference ONNX models (head detector deimv2_wholebody49_boxes_only.onnx, label verification sixdrepnet360_1x3x224x224_full.onnx, landmarks hrffa_vitl_ibug68_1x3x320x320.onnx, etc.)
• ckpts/ — DINOv3 pretrained weights (not committed for license reasons; referenced at runtime)

3.1. Dataset analysis plots

The dataset build produces detailed analysis plots under data/yawpose/. The snapshots below are copies in docs/images/; each plot is regenerated by the script noted with it.

Yaw distribution (10° bins, stacked by source) — scripts/plot_yaw_distribution.py. Full-circle coverage of the final 42,135 cleaned labels. On top of the original frontal-heavy sources (s001–s003), the balancing batches fill the rear hemisphere (s004), reinforce the hard rear 3/4-view ranges 120°–180° / 210°–270° (s005) and fill the 100°–120° valley (s006).

Yaw distribution (polar form) — the same histogram on a circle: no direction around the full 360° is left empty, with the frontal peak (330°–30°) coming from the original sources.

Yaw / pitch / roll distributions — scripts/plot_ypr_distribution.py. Pitch (generation intent where available, sixdrepnet360 measurement otherwise) spans roughly ±120° including extreme upward/downward views; roll (sixdrepnet360 measurement; the generation intent is 0 for all sources) concentrates near 0.

Projected distribution after planned additions — scripts/plan_augmentation.py + scripts/plot_projected_distribution.py. The design tool for each balancing batch: hatched areas show the planned samples needed to raise every 10° bin to the target floor, before a generation batch is ordered.

Full-range yaw evaluation vs SemiUHPE — scripts/eval_semiuhpe.py. SemiUHPE's published weights evaluated on the same val split next to the distilled students, including per-rear-bin MAE (see §4 for the caveats on these numbers). Note that this comparison is heavily stacked against SemiUHPE and should not be read as a straightforward ranking: SemiUHPE is trained only on public datasets and has never seen anything like the extremely difficult, purely synthetic yawpose data (full-circle yaw including rear views, extreme pitch, label conventions aligned with our own pipeline), so the table says "our models fit our own domain" rather than "our models are better in general".

4. Models (parameter counts, compute and reference accuracy)

Compute is measured on the ONNX graphs (scripts/count_macs_onnx.py, accumulating Conv/MatMul/Gemm multiply-adds; GFLOPs = 2 × GMACs). Parameter counts are from the PyTorch implementations (slightly lower in ONNX after BN folding).

Model	Input	Params [M]	GMACs	GFLOPs	MAAE [deg]	med [deg]	acc15 [%]	acc30 [%]
YawNet-64	64x64	0.77	0.013	0.026	12.94	8.65	73.5	92.6
YawNet-96	96x96	0.77	0.028	0.057	12.96	8.62	74.2	92.6
YawNet-128	128x128	0.77	0.050	0.101	12.62	8.47	75.7	92.6
DINOv3 ViT-L + biternion head	320x320	304.20	130.684	261.367	0.97	0.78	99.98	100.0

Ready-to-use artifacts for every row — the exported ONNX files (fixed and N-batch, with and without the κ output) and the training runs with checkpoints — are attached to the resources release.

Reference accuracy (the four rightmost columns) is measured on the yawpose val split (4,211 images; circular angular error, see §5.7). The YawNet rows are the distilled students and the DINOv3 row is their teacher, all from the final product runs (runs/{yawnet_distill_{064,096,128},dinov3_vitl16_320}_unified_*), which are trained with --unified (train + val merged) — the val split is therefore not held out for these runs and the numbers are reference values, not held-out test scores (this inflates the teacher's row in particular, since the 304M teacher can memorize the merged data). As a held-out reference, the same DINOv3 teacher trained on the train split only reaches 13.55° val MAAE (v5, hflip TTA; see docs/yawpose_dataset.md).

No established head-pose benchmark dataset (300W-LP, AFLW2000, BIWI, CMU Panoptic, DAD-3DHeads, etc.) is used anywhere in this project — deliberately, neither for training nor for evaluation. All training and evaluation data is the purely synthetic yawpose dataset, so the numbers above are not comparable to accuracy reported on those benchmarks. For an external reference point on the same val split, SemiUHPE (full-range, published weights) scores 55.14° MAAE / acc30 53.5% (scripts/eval_semiuhpe.py, §7.4).

5. Training

(uv run below can be replaced with uv run --no-sync once the venv is synced)

5.1. VRAM tiers (--vram 8|16|96)

All batch-related settings live in scripts/vram_presets.py and are resolved simply by passing your machine's VRAM tier with --vram. The effective batch (micro_batch × grad_accum) is identical across all tiers, so training dynamics do not depend on the machine and --resume across tiers is possible (the consistency check uses the effective batch).

Task	Effective batch	96GB	16GB	8GB
yawnet (size ≤128)	256	256×1	256×1	256×1
yawnet (size ≥192)	128	128×1	64×2	32×4
DINOv3 teacher	64	64×1 (all blocks)	16×4 (all blocks)	8×8 (only the last 8 blocks unfrozen)
Distillation	128	128×1	128×1	64×2

5.2. Direct training (baseline, 64/96/128)

uv run python scripts/train_yawnet.py \
--size 64

uv run python scripts/train_yawnet.py \
--size 96

uv run python scripts/train_yawnet.py \
--size 128

Main defaults: --epochs 100 --batch 256 --lr 3e-3 --kappa 2.0 --width 1.0 --balance inv --balance-bin 10. Artifacts go to runs/yawnet_<size>[_tag]/ (last.pt / best_{maae:.6f}.pt / train_log.jsonl / result.json).

5.3. Teacher training (320x320, for distillation)

uv run python scripts/train_yawnet.py \
--size 320 \
--width 2.0 \
--batch 32 \
--tag teacher

The teacher is never deployed, so it is exempt from the 2M-parameter budget (about 3.0M at width 2.0).

5.4. DINOv3 ViT-L teacher (320x320, best accuracy)

DINOv3 pretrained backbone (304M) + biternion head. Weights and code are referenced at runtime the same way as in HRFFA and are not bundled for license reasons (the teacher is never deployed). Input normalization is ImageNet (it differs from the students' center05; the distillation side handles this automatically).

uv run python scripts/train_teacher_dinov3.py \
--vram 96 \
--variant vitl16 \
--tag teacher

uv run python scripts/train_teacher_dinov3.py \
--vram 96 \
--variant vitl16 \
--tag teacher \
--resume

Recipe: the backbone is fully frozen at epoch 0 (head only), then the preset range is unfrozen; differential lr (backbone 2e-5 / head 2e-4), bf16, grad_clip 1.0. Since v5 the defaults include a κ (confidence) head + von Mises NLL (hard samples are automatically down-weighted through κ; --no-kappa-head restores the old behavior) and EMA (--ema-decay 0.999, 0 disables it; evaluation and best checkpoints use the EMA weights). The per-epoch mean of κ is logged as kappa_mean in train_log.jsonl. The lr schedule is --lr-schedule cosine (default) or wsd (warmup → constant lr → cosine decay to 0 over the last --decay-epochs; since the constant phase does not depend on the total epoch count, --epochs can be rewritten on resume to extend or shorten the run — setting epochs = current epoch + decay_epochs enters the decay immediately). Artifacts go to runs/dinov3_vitl16_320[_tag]/ (last.pt is roughly 2GB including optimizer state).

5.5. Distillation (teacher → student)

uv run python scripts/distill_yawnet.py \
--teacher runs/dinov3_vitl16_320_teacher \
--student-size 96 \
--vram 8 \
--alpha 0.7 \
--beta 0.3

--teacher accepts a run directory (best_*.pt is auto-discovered) or a .pt file. The teacher type (YawNet / DINOv3) and its input normalization are detected automatically from the checkpoint. Identical-condition pair scheme (geometry + degradations are applied once on the 320 side and the student sees a downscaled copy of it; everything except resolution is identical), with loss = α·NLL(student, teacher) + β·NLL(student, GT) (defaults α=0.7/β=0.3). Artifacts go to runs/yawnet_distill_<size>[_tag]/.

5.6. Resuming after interruption

Every script resumes from runs/<run_name>/last.pt with the same arguments + --resume, fully restoring all state (model / optimizer / scheduler / scaler / RNG):

uv run python scripts/train_yawnet.py \
--size 320 \
--width 2.0 \
--batch 32 \
--tag teacher \
--resume

uv run python scripts/distill_yawnet.py \
--teacher runs/yawnet_320_teacher \
--student-size 96 \
--resume

5.7. Reading the validation metrics

Metrics printed and logged at every epoch on the validation split (4,211 images). All errors are computed as circular angular differences (e.g., the error between a prediction of 359° and a ground truth of 1° is 2°, not a 358° blunder).

Metric	Unit	Meaning	How to read it
maae	deg	Mean Absolute Angular Error	The model-selection criterion (best_*.pt is the epoch with the best value). Closer to 0 is better; random prediction gives 90
med	deg	Median angular error	The "typical" error, robust to outliers. A large gap from maae indicates a few large misses (e.g., 180°-class left/right confusion on rear views)
acc15	%	Fraction of samples within 15° error	The rate of "practically pointing in the right direction". Closer to 100 is better
acc30	%	Fraction of samples within 30° error	Same, looser threshold
per_bin_mae	deg	Mean error per 30° bin of ground-truth yaw	train_log.jsonl / result.json only. Used to spot direction-specific weaknesses, e.g. frontal (000-030 / 330-360) vs. rear (150-210) accuracy gaps

6. ONNX export

uv run python scripts/export_onnx.py \
--ckpt runs/yawnet_distill_128_unified_v6u

# To include κ (confidence) as a second output:
uv run python scripts/export_onnx.py \
--ckpt runs/yawnet_distill_128_unified_v6u \
--with-kappa

Teacher/student and the presence of the κ head are detected automatically from the checkpoint. A single invocation runs: batch-1 export → onnxslim (no Gemm fusion) → onnxsim → graph canonicalization → torch vs ORT parity check → N-batch conversion (with batch 1/2/3 equivalence checks) → audit, producing <run>/<stem>_{1,N}x3xSxS.onnx. Input contract: images (N,3,S,S). Students expect center05 normalization (x/127.5 − 1), the teacher expects ImageNet normalization. The output cos_sin (N,2) is a unit vector; with --with-kappa, kappa (N) (von Mises concentration = confidence) is added.

7. Repository layout

7.1. Documents

• docs/yawpose_dataset.md — spec of the unified yawpose dataset (crop rule, yaw convention, the full record of label fixes and teacher relabeling, file list)
• docs/synthetic_004_generation_spec.md — generation brief for full-circle yaw balancing
• docs/synthetic_005_generation_spec.md — generation brief for reinforcing the hard ranges (120°–180° / 210°–270°)
• docs/synthetic_006_generation_spec.md — generation brief for reinforcing the 90°–120° range

7.2. Dataset construction and label quality

• scripts/build_yawpose_dataset.py — dataset build (DEIM CUDA detection → 5%-margin square crop → 320x320, resumable)
• scripts/verify_labels_sixd.py — full-dataset yaw label verification with sixdrepnet360 (→ qa_sixd.jsonl)
• scripts/fix_labels.py — label fixes (sign-convention correction, sign recovery via sixd / landmarks, dropping unverifiable rows)
• scripts/study_landmark_yaw.py — feasibility study / calibration of landmark-geometry yaw-sign estimation
• scripts/relabel_rear_teacher.py — teacher-based audit and relabeling of the s001 rear labels (fully reversible)
• scripts/plan_augmentation.py — computes the additional-generation plan for distribution balancing
• scripts/plot_yaw_distribution.py / plot_projected_distribution.py / plot_ypr_distribution.py — distribution visualization

7.3. Models and training

• scripts/yawnet.py — YawNet (MBConv+SE+SiLU, biternion output, ~0.77M parameters, optional κ head)
• scripts/dinov3_yaw.py — DINOv3 backbone + biternion head teacher model (HRFFA-style runtime loading)
• scripts/augment.py — yaw-only adaptation of HRFFA's D4 geometric augmentation (all geometric transforms are composed into a single 3x3 matrix and the image is warped only once; includes photometric / motion blur / random erase / low-res jitter)
• scripts/verify_cam_yaw_sign.py — measures the coefficient a camera-rotation warp applies to appearance-based yaw (CAM_YAW_COEF = 0.166)
• scripts/yaw_dataset.py — Dataset / balanced sampler / identical-condition pair Dataset for distillation
• scripts/ema.py — EMA (exponential moving average) helper
• scripts/vram_presets.py — per-VRAM-tier (8/16/96GB) batch/accumulation/precision presets (effective batch shared across tiers)
• scripts/train_yawnet.py — direct training (64/96/128, von Mises NLL, AMP, balance sampler, --unified)
• scripts/train_teacher_dinov3.py — DINOv3 ViT-L teacher training (freeze→unfreeze, differential lr, bf16, κ + EMA, wsd)
• scripts/distill_yawnet.py — distillation from the 320x320 teacher to low-resolution students (checkpoint / resume / logging conventions shared with train_yawnet.py)
• scripts/val_preview.py / render_preview.py — ring-dial visualization of validation samples (on best update / offline)

7.4. Export and evaluation

• scripts/export_onnx.py — ONNX export (optimization, parity, N-batching and audit in one run)
• scripts/count_macs_onnx.py — MACs measured from the ONNX graph
• scripts/eval_semiuhpe.py — SOTA comparison: full-range yaw evaluation of SemiUHPE (arXiv 2404.02544) on the yawpose val split

8. Citation and acknowledgements

8.1. Citing this repository

@software{hyodo2026yawnet,
  author    = {Katsuya Hyodo},
  title     = {{YawNet}: Full-Circle Head-Yaw Regression via Biternion ({cos}/{sin}) + von Mises kappa, {DINOv3} Teacher-Student Distillation},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.22161811},
  url       = {https://github.com/PINTO0309/YawNet}
}

8.2. Backbones, pretrained weights and tools

DINOv3 — teacher backbone (ViT-L/16). The official implementation is cloned into the torch.hub cache and imported at runtime; the weights are distributed under the DINOv3 License (neither is bundled in this repository). Paper: https://arxiv.org/abs/2508.10104 · Code: https://github.com/facebookresearch/dinov3

@misc{simeoni2025dinov3,
  title         = {{DINOv3}},
  author        = {Sim{\'e}oni, Oriane and Vo, Huy V. and Seitzer, Maximilian and Baldassarre, Federico and Oquab, Maxime and Jose, Cijo and Khalidov, Vasil and Szafraniec, Marc and Yi, Seungeun and Ramamonjisoa, Micha{\"e}l and Massa, Francisco and Haziza, Daniel and Wehrstedt, Luca and Wang, Jianyuan and Darcet, Timoth{\'e}e and Moutakanni, Th{\'e}o and Sentana, Leonel and Roberts, Claire and Vedaldi, Andrea and Tolan, Jamie and Brandt, John and Couprie, Camille and Mairal, Julien and J{\'e}gou, Herv{\'e} and Labatut, Patrick and Bojanowski, Piotr},
  year          = {2025},
  eprint        = {2508.10104},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CV},
  url           = {https://arxiv.org/abs/2508.10104}
}

DEIMv2 / DEIMv2-Wholebody49 — head detection during dataset construction (models/deimv2_wholebody49_boxes_only.onnx, Apache-2.0). Paper: https://arxiv.org/abs/2509.20787 · Code: https://github.com/Intellindust-AI-Lab/DEIMv2 · Model: https://github.com/PINTO0309/PINTO_model_zoo/tree/main/488_DEIMv2-Wholebody49

@article{huang2025deimv2,
  title   = {Real-Time Object Detection Meets {DINOv3}},
  author  = {Huang, Shihua and Hou, Yongjie and Liu, Longfei and Yu, Xuanlong and Shen, Xi},
  journal = {arXiv preprint arXiv:2509.20787},
  year    = {2025},
  url     = {https://arxiv.org/abs/2509.20787}
}

@software{DEIMv2-Wholebody49,
  author = {Katsuya Hyodo},
  title  = {Unified multi-task model for detection, pose estimation, and instance segmentation. 49 classes.},
  url    = {https://github.com/PINTO0309/PINTO_model_zoo/tree/main/488_DEIMv2-Wholebody49},
  year   = {2026},
  month  = {05},
  doi    = {10.5281/zenodo.10229410}
}

6DRepNet / 6DRepNet360 — used for full-dataset yaw label verification and sign recovery (models/sixdrepnet360_1x3x224x224_full.onnx; ONNX from PINTO_model_zoo #423). Papers: https://arxiv.org/abs/2202.12555 · https://arxiv.org/abs/2309.07654

@inproceedings{hempel2022sixdrepnet,
  title     = {{6D} Rotation Representation For Unconstrained Head Pose Estimation},
  author    = {Hempel, Thorsten and Abdelrahman, Ahmed A. and Al-Hamadi, Ayoub},
  booktitle = {IEEE International Conference on Image Processing (ICIP)},
  year      = {2022},
  url       = {https://arxiv.org/abs/2202.12555}
}

@article{hempel2023sixdrepnet360,
  title   = {Towards Robust and Unconstrained Full Range of Rotation Head Pose Estimation},
  author  = {Hempel, Thorsten and Abdelrahman, Ahmed A. and Al-Hamadi, Ayoub},
  journal = {arXiv preprint arXiv:2309.07654},
  year    = {2023},
  url     = {https://arxiv.org/abs/2309.07654}
}

HRFFA — source of the landmark-based yaw-sign recovery model (models/hrffa_vitl_ibug68_1x3x320x320.onnx), and the origin of the geometric augmentation (D4 single-matrix warp), the DINOv3 teacher recipe, the VRAM tier design and the ONNX export/verification pipeline ported into this repository. https://github.com/PINTO0309/High-Angle_Robust_Fast_FaceAlignment

@software{hyodo2026hrffa,
  author    = {Katsuya Hyodo},
  title     = {HRFFA: High-Angle Robust Fast FaceAlignment},
  month     = {aug},
  year      = {2026},
  publisher = {Zenodo},
  doi       = {10.5281/zenodo.22161811},
  url       = {https://doi.org/10.5281/zenodo.22161811},
}

yawpose dataset — the unified synthetic dataset built by this repository (data/yawpose/, ~42k images of fictitious people created with image-generation AI) is licensed under CC BY 4.0; the LICENSE and NOTICE files are bundled with the dataset. When reusing it, please attribute "yawpose" by Katsuya Hyodo (https://github.com/PINTO0309/YawNet).

8.3. Methods referenced (reimplemented from the papers; no code was copied)

BiternionNet — YawNet is an original architecture that borrows only the biternion output representation (unit (cos θ, sin θ) regression) and the von Mises loss from this paper. Paper: https://lucasb.eyer.be/academic/biternions/biternions_gcpr15.pdf · Original implementation (MIT): https://github.com/lucasb-eyer/BiternionNet

@inproceedings{beyer2015biternion,
  title     = {Biternion Nets: Continuous Head Pose Regression from Discrete Training Labels},
  author    = {Beyer, Lucas and Hermans, Alexander and Leibe, Bastian},
  booktitle = {German Conference on Pattern Recognition (GCPR)},
  year      = {2015},
  url       = {https://lucasb.eyer.be/academic/biternions/biternions_gcpr15.pdf}
}

SemiUHPE — the SOTA comparison target. Its published weights (DAD-WildHead-EffNetV2-S) are evaluated full-range on the yawpose val split (scripts/eval_semiuhpe.py; the inference path is a reimplementation following the official predict.py). Paper: https://arxiv.org/abs/2404.02544 · Code: https://github.com/hnuzhy/SemiUHPE

@article{zhou2024semiuhpe,
  title   = {Semi-Supervised Unconstrained Head Pose Estimation in the Wild},
  author  = {Zhou, Huayi and Jiang, Fei and Yuan, Jin and Rui, Yong and Lu, Hongtao and Jia, Kui},
  journal = {arXiv preprint arXiv:2404.02544},
  year    = {2024},
  url     = {https://arxiv.org/abs/2404.02544}
}

8.4. Acknowledgements

• Thanks to Meta AI for DINOv3, the DEIMv2 authors, the 6DRepNet360 authors, and the SemiUHPE authors (whose published weights are used for the comparison).
• The training data (~42k images) consists entirely of fictitious people created with image-generation AI; no real individuals are included.
• The ONNX post-processing know-how (batch-axis preservation, fixed batch-1 → N-batch conversion, graph canonicalization and auditing) builds on HRFFA's export pipeline and the PINTO_model_zoo conventions.