Skip to content

fix: enable strict positional enforcement for the test suite under pytest - #2802

Open
skippdot wants to merge 2 commits into
googleapis:mainfrom
skippdot:fix-positional-enforcement
Open

fix: enable strict positional enforcement for the test suite under pytest#2802
skippdot wants to merge 2 commits into
googleapis:mainfrom
skippdot:fix-positional-enforcement

Conversation

@skippdot

@skippdot skippdot commented Sep 1, 2026

Copy link
Copy Markdown

Fixes #2755

The test suite is written against strict positional parameter enforcement (TypeError on violation), but that mode was only ever switched on inside the nose-era setup_package() hook in tests/__init__.py — a hook pytest never calls. Under pytest the suite silently runs in the default warning-only mode.

The reason full runs still pass is an accident of ordering, which is the second half of the problem: the PositionalTests methods in tests/test__helpers.py mutate the module-level positional_parameters_enforcement flag and never restore it. Alphabetical execution leaves test_usage (which sets EXCEPTION) as the last write, so tests/test_discovery.py happens to run strict. Any other order breaks it — reproducible on an unmodified main:

pytest "tests/test__helpers.py::PositionalTests::test_enforcement_ignore" \
       "tests/test_discovery.py::DiscoveryErrors::test_tests_should_be_run_with_strict_positional_enforcement"
# → 1 failed (AssertionError: should have raised a TypeError exception over missing http=.)

as is running tests/test_discovery.py on its own.

Changes

  • tests/__init__.py: set positional_parameters_enforcement = util.POSITIONAL_EXCEPTION at import time (using the constant rather than the raw string) and delete the dead setup_package() hook — nothing references it.
  • tests/test__helpers.py: add setUp/tearDown to PositionalTests to save and restore the enforcement flag, so these tests stop leaking global state into the rest of the suite.

Two files, +13/−4.

Verification (Python 3.14)

Scenario before after
strict test in isolation failed passed
test_enforcement_ignore ordered before the strict test failed 2 passed
tests/test_discovery.py alone 1 failed, 172 passed 173 passed
full tests/ folder passed by luck 338 passed, 1 skipped
reversed file order (test_discovery.py before test__helpers.py) order-dependent 183 passed

black --check clean; flake8 --select=E9,F63,F7,F82 (the set CI enforces) clean.

…test

- Set `positional_parameters_enforcement = POSITIONAL_EXCEPTION` at import
  time in tests/__init__.py and delete the nose-era `setup_package()` hook,
  which pytest never calls, leaving the suite in warning-only mode
- Save and restore the enforcement setting in PositionalTests, whose methods
  mutate the module-level flag and previously leaked whatever value the
  last-run test set, making other tests pass or fail depending on order
- The suite only passed in full runs by accident: alphabetical ordering left
  `test_usage` (EXCEPTION) as the last write; any other order, or running
  test_discovery.py alone, failed
  `test_tests_should_be_run_with_strict_positional_enforcement`

Fixes googleapis#2755
@skippdot
skippdot requested a review from a team as a code owner September 1, 2026 12:14
@product-auto-label product-auto-label Bot added the size: s Pull request size is small. label Sep 1, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the test suite initialization to set positional parameter enforcement at import time, ensuring compatibility with pytest, and adds state restoration to PositionalTests to prevent side effects. The reviewer suggests using self.addCleanup in setUp as a more robust and idiomatic way to restore the module-level enforcement setting instead of using a separate tearDown method.

Comment thread tests/test__helpers.py Outdated
- Replace the setUp/tearDown pair in PositionalTests with a single setUp
  registering addCleanup, as suggested in review
- Cleanup now runs even if setUp or the test itself fails midway
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size: s Pull request size is small.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test: strict positional parameter enforcement is ignored under pytest

1 participant