dima

on software
Posts from blog by tag pytest:

The Hidden Cost of Test Inheritance

I'm subscribed to Adam Johnson's blog and usually really enjoy his writing - it's practical, deep, and no-bullshit. But one recent post, Python: sharing common tests in unittest, caught me off guard.

It describes a "neat" pattern: write reusable test logic in a base class, subclass it to test multiple objects, hiding the base class from unittest discovery. While the intent is fine - DRYing out duplicated test code - the result is fragile, confusing, and just not worth it.

Here's why.

The Pattern: DRY Tests via Subclassing

# Sample units to test
class Armadillo:
    def speak(self) -> str:
        return "Hrrr!"

class Okapi:
    def speak(self) -> str:
        return "Gronk!"

# Test module
class BaseAnimalTests(TestCase):
    animal_class: type

    def test_speak(self):
        sound = self.animal_class().speak()
        self.assertIsInstance(sound, str)
        self.assertGreater(len(sound), 0)

class ArmadilloTests(BaseAnimalTests):
    animal_class = Armadillo

class OkapiTests(BaseAnimalTests):
    animal_class = Okapi

del BaseAnimalTests

Yes, it works and it reduces duplication. But it comes at the cost of everything else that makes tests maintainable.

The Problems

IDE and DX Pain

IDE

When a test fails, I want to jump to it in my IDE, set a breakpoint, and debug. With this pattern - good luck.

The method doesn't exist in ArmadilloTests, it's buried in a deleted parent class. You have to manually hunt it down, re-declare the test method just to put a breakpoint and debug it, and pray the animal_class setup matches what failed:

class ArmadilloTests(TestCase):
    animal_class = Armadillo

    def test_speak(self):
        super().test_speak()

breakpoint

It's tedious and wastes time. All this to avoid writing a 3-line test twice?

class ArmadilloTests(TestCase):
    def test_speak(self):
        sound = Armadillo().speak()
        self.assertIsInstance(sound, str)
        self.assertGreater(len(sound), 0)

Clear, simple, debug-friendly. Worth the few extra lines.

CI Failures Are Confusing

If a shared test fails in CI, you get something like:

test_speak (tests.ArmadilloTests.test_speak) ... FAIL
...
Traceback (most recent call last):
  File ".../tests.py", line 20, in test_speak
    self.assertGreater(len(sound), 0)
AssertionError: 0 not greater than 0

But the method isn't defined in ArmadilloTests, and Search everywhere won't help at all:

nothing found

So now you have to reverse-engineer which base class it came from and how to recreate it locally.

This isn't clever. It's just fragile.

When It Kinda Makes Sense

There are rare cases:

  • dozens of classes implementing the same interface
  • you're the only one maintaining the codebase
  • you run everything headless in CI

But even then, you're building test framework plumbing to save what, a hundred lines?

The Clean Alternative: Parametrize It

Pytest Style

@pytest.mark.parametrize('animal_class', [Armadillo, Okapi])
def test_speak(animal_class):
    sound = animal_class().speak()
    assert isinstance(sound, str)
    assert len(sound) > 0

You see all the parameters. You see where the test lives. Failures are explicit:

test_speak[Armadillo] FAILED
test_speak[Okapi] PASSED

You can re-run just the failing test. You can debug with a conditional breakpoint. You don't need to explain how the tests are wired together - because they're not.

unittest Style (Optional, Not Ideal)

from parameterized import parameterized_class

@parameterized_class([
    {'animal_class': Armadillo},
    {'animal_class': Okapi},
], class_name_func=get_class_name)
class AnimalTests(TestCase):
    def test_speak(self):
        sound = self.animal_class().speak()
        self.assertIsInstance(sound, str)
        self.assertGreater(len(sound), 0)

Using parameterized_class from parameterized is still better than inheritance, but clunkier. Output is readable if you customize class_name_func. IDE support isn't great. Pytest remains the better option for anything dynamic.

Final Verdict

Tests should fail clearly, debug easily, and be readable years later. This pattern fails all three.

DRY is good. But in tests, visible duplication beats invisible abstraction.

Adam's trick technically works, but in practice, it makes tests harder to navigate, harder to trust, and harder to work with.

Stick to the boring version - you'll thank yourself later.

Pytest Fish shell autocompletion

TL;DR https://github.com/ddoroshev/pytest.fish

Typing repetitive commands or copying and pasting test names can eat up valuable time. To help, I've created pytest.fish - a Fish shell plugin that simplifies your pytest workflow. It's lightweight, simple to set up, and makes testing more efficient.

How to Use

Autocomplete test paths

Type pytest and hit TAB to get suggestions for test paths and functions:

Support for -k filter

Narrow down tests with -k and get name suggestions:

The plugin dynamically scans your project, so suggestions stay up-to-date.

Installation

Install with Fisher:

fisher install ddoroshev/pytest.fish

Or manually copy the files from the repository into your Fish configuration.

How It Works

The plugin doesn't rely on pytest directly (yet). Instead, it scans the current directory for test files and searches for test functions inside them, making the process relatively fast and efficient.

Other shells?

Since I primarily use Fish in my local development environment, I created a plugin specifically for this shell. However, if you use Bash or Zsh, feel free to create your own - or switch to Fish already. 😉

This Week in Changelogs: flask, pytest, IPython, etc

pyenv 2.3.13, 2.3.14

Highlights from the changelog:

  • added versions 3.10.10, 3.11.2, and 3.12.0a5;
  • fixed versions 3.5.10 and 3.6.15 for macOS and modern 64-bit platforms.

This one made me laugh a bit:

  • a7b181c introduce an indentation issue
  • 3829742 fix the indendation issue.

That's how programming actually works!

TIL: head -n123 is a part of POSIX, head -123 is a shorthand that can be missing in some operating systems (pull request).

IPython 8.11.0

Highlights from the changelog:

  • %autoreload supports meaningful parameters (%autoreload all, %autoreload off, etc), not only numbers (%autoreload 0, %autoreload 2, etc).

I like the log of the pull request, it illustrates the approach of implementing a feature step-by-step, one frame at a time:

Also, this fragment is quite interesting, print and logger.info need to be used carefully for logging and protected from being overwritten during hot-reload:

p = print
logger = logging.getLogger("autoreload")
l = logger.info

def pl(msg):
    p(msg)
    l(msg)

Everything you wanted to know about GitHub actions:

flask 2.2.3

Although the changelog is not that big, I like the thing about flask run --debug.

Previously, it was flask --debug run , and it was awkward. The fix itself is quite small, but there's a lot of changes in docs, and also a PyCharm screenshot was changed. Nice and pure!

pytest 7.2.1, 7.2.2

The changelogs contains mostly bug fixes. One of them is about pytest.approx() causing ZeroDivisionError on dicts.

Another one fixes type checkers behaviour for the following code, which I think should be illegal:

with pytest.raises(RuntimeError) if val else contextlib.nullcontext() as excinfo:

(Please, don't write the code like this.)

And they fixed a race condition when creating directories in parallel, using os.makedirs(..., exists_ok=True). Simple, but helpful.

whitenoise 6.4.0

The changelog mentions support for Django 4.2. It was good to know, by the way, that STATICFILES_STORAGE is going to be changed to STORAGES dict (pull request).

django-cors-headers 3.14.0

Changelog:

  • added support for Django 4.2,
  • switched from urlparse to urlsplit.

The latter is the most interesting, urlsplit is slightly faster. Also, it's cached, so sometimes you gain a huge performance.

The difference between these functions is that urlparse includes parsing of the "parameters" section of a URL:

scheme://netloc/path;parameters?query#fragment
                     ^ this

Since it's not widely used, in most cases it's safe to switch from urlparse to urlsplit.