Speedy pytest test discovery with pytest_finder

pytest is a very useful tool for writing and running python tests. I’ve used it extensively over the past 10 or so years and I really like it. In my opinion there are some less good features about it such as the slightly less-than-transparent importing of things from conftest but in general it’s an excellent package.

pytest fixtures (and parameterised fixtures) ¹, in particular, are a powerful feature which allow programmers to express multiple test cases in a succinctly and pleasant way. Take, for example, a test I wrote recently for a personal project ². test_trivial_boards tests four different parameter sets in one go, granted I could have written them out as separate tests, or I could have used a single tests and repeated the KnightProblem setup each time but both of those feel less neat: They’ll be harder to reason about, and be less clear for people trying to familiarise themselves with a codebase.

My main gripe with pytest fixtures is how they are executed from the command line. Pytest has syntax for specifying which test, and which parametrisation of a test, should be run - once you include some pytest runtime options (YMMV if you’re using a bash shell instead of zsh ³) it looks a bit like this:

>>> pytest -xsvvvv "tests/test_knight_problem.py::test_trivial_boards[knight_squares2-2-1-2]"

Tab completion can get you some of the way but it’s still painful to write. It’s not even quite as simple as reading the parameter values and hyphen concatenating them because pytest will rename things which aren’t trivially string-able (you can see it’s done it to the knight_squares test fixture in the above example).

To try and help make fixtures even more useful and efficient to run, I wrote a little tool, pytest_finder, and make the source available here ⁴.

It’s very short, about 100 lines total, and works by using the pytests hooks API ⁵ to find all the possible parametrised tests in a given project, and then passing these to the fantastic fzf tool ⁶ which displays them and provide speedy fuzzy string searching based on test and parametrisation names. You can optionally pass through pytest runtime options and test directories too.

Here’s a demo of it in action:

It’s pretty basic at the moment so there are still quite a few features I want to add, the main ones being@

• Better fuzzy matching (on the command line input)
• Making it pip installable/better packaging in general,
• Adding some state so that the last pytest_finder run is cached for repeat test invocations,
• I don’t know how it would behave with python breakpoints…my guess is badly! So I should probably fix that too

But in general it’s good enough to use as it is at the moment, I use it most days and on most python projects I work on.

References