Skip to content

Fix oversized Parquet file validation - #558

Open
avinashgola wants to merge 1 commit into
togethercomputer:mainfrom
avinashgola:codex/fix-oversized-parquet-validation
Open

Fix oversized Parquet file validation#558
avinashgola wants to merge 1 commit into
togethercomputer:mainfrom
avinashgola:codex/fix-oversized-parquet-validation

Conversation

@avinashgola

Copy link
Copy Markdown

Fixes #485

What changed

  • Return immediately when a file exceeds the configured maximum size.
  • Preserve the measured file size in the validation report.
  • Add a regression test proving oversized Parquet files cannot be marked valid by later format checks.

Verification

  • /private/tmp/together-py-test-env/bin/pytest -q tests/unit/test_files_checks.py
  • 41 passed

@broly-code-security-scanner

Copy link
Copy Markdown

Broly Security Scan

Warning

⚠️ Potential results — Broly is still completing analysis or verification. This comment will update automatically.
Files pending AI analysis: 2; findings pending verification: 0.

Note

Re-scan this PR anytime with /broly scan — useful after /broly undismiss, or to refresh findings without a new push.

Broly — SAST (zai-org/GLM-5.3-Flash) · Secrets · SCA · IaC · GH Actions · Base Images · Supply Chain Threats · Exploit Chains · Adversarial Verification

We're continuously improving Broly's accuracy and finding quality — your feedback is valuable. False positives, missed findings, bugs, and feature requests all welcome.

Ask in #security-engineering   Powered by Together AI

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

check_file: an oversize .parquet file passes validation because the size rejection is overwritten

1 participant