chore(deps): update dependency pytest to v8
This MR contains the following updates:
Package | Update | Change |
---|---|---|
pytest (changelog) | major |
==4.6.3 -> ==8.3.4
|
Release Notes
pytest-dev/pytest
v8.3.4
pytest 8.3.4 (2024-12-01)
Bug fixes
-
#12592: Fixed
KeyError
{.interpreted-text role="class"} crash when using--import-mode=importlib
in a directory layout where a directory contains a child directory with the same name. -
#12818: Assertion rewriting now preserves the source ranges of the original instructions, making it play well with tools that deal with the
AST
, like executing. -
#12849: ANSI escape codes for colored output now handled correctly in
pytest.fail
{.interpreted-text role="func"} with [pytrace=False]{.title-ref}. -
#9353:
pytest.approx
{.interpreted-text role="func"} now uses strict equality when given booleans.
Improved documentation
-
#10558: Fix ambiguous docstring of
pytest.Config.getoption
{.interpreted-text role="func"}. -
#10829: Improve documentation on the current handling of the
--basetemp
option and its lack of retention functionality (temporary directory location and retention
{.interpreted-text role="ref"}). -
#12866: Improved cross-references concerning the
recwarn
{.interpreted-text role="fixture"} fixture. -
#12966: Clarify
filterwarnings
{.interpreted-text role="ref"} docs on filter precedence/order when using multiple@pytest.mark.filterwarnings <pytest.mark.filterwarnings ref>
{.interpreted-text role="ref"} marks.
Contributor-facing changes
- #12497: Fixed two failing pdb-related tests on Python 3.13.
v8.3.3
pytest 8.3.3 (2024-09-09)
Bug fixes
-
#12446: Avoid calling
@property
(and other instance descriptors) during fixture discovery -- byasottile
{.interpreted-text role="user"} -
#12659: Fixed the issue of not displaying assertion failure differences when using the parameter
--import-mode=importlib
in pytest>=8.1. -
#12667: Fixed a regression where type change in [ExceptionInfo.errisinstance]{.title-ref} caused [mypy]{.title-ref} to fail.
-
#12744: Fixed typing compatibility with Python 3.9 or less -- replaced [typing.Self]{.title-ref} with [typing_extensions.Self]{.title-ref} -- by
Avasam
{.interpreted-text role="user"} -
#12745: Fixed an issue with backslashes being incorrectly converted in nodeid paths on Windows, ensuring consistent path handling across environments.
-
#6682: Fixed bug where the verbosity levels where not being respected when printing the "msg" part of failed assertion (as in
assert condition, msg
). -
#9422: Fix bug where disabling the terminal plugin via
-p no:terminal
would cause crashes related to missing theverbose
option.-- by
GTowers1
{.interpreted-text role="user"}
Improved documentation
- #12663: Clarify that the [pytest_deselected]{.title-ref} hook should be called from [pytest_collection_modifyitems]{.title-ref} hook implementations when items are deselected.
- #12678: Remove erroneous quotes from [tmp_path_retention_policy]{.title-ref} example in docs.
Miscellaneous internal changes
- #12769: Fix typos discovered by codespell and add codespell to pre-commit hooks.
v8.3.2
pytest 8.3.2 (2024-07-24)
Bug fixes
-
#12652: Resolve regression [conda]{.title-ref} environments where no longer being automatically detected.
-- by
RonnyPfannschmidt
{.interpreted-text role="user"}
v8.3.1
pytest 8.3.1 (2024-07-20)
The 8.3.0 release failed to include the change notes and docs for the release. This patch release remedies this. There are no other changes.
v8.3.0
pytest 8.3.0 (2024-07-20)
New features
-
#12231: Added [--xfail-tb]{.title-ref} flag, which turns on traceback output for XFAIL results.
- If the [--xfail-tb]{.title-ref} flag is not given, tracebacks for XFAIL results are NOT shown.
- The style of traceback for XFAIL is set with [--tb]{.title-ref}, and can be [auto|long|short|line|native|no]{.title-ref}.
- Note: Even if you have [--xfail-tb]{.title-ref} set, you won't see them if [--tb=no]{.title-ref}.
Some history:
With pytest 8.0, [-rx]{.title-ref} or [-ra]{.title-ref} would not only turn on summary reports for xfail, but also report the tracebacks for xfail results. This caused issues with some projects that utilize xfail, but don't want to see all of the xfail tracebacks.
This change detaches xfail tracebacks from [-rx]{.title-ref}, and now we turn on xfail tracebacks with [--xfail-tb]{.title-ref}. With this, the default [-rx]{.title-ref}/ [-ra]{.title-ref} behavior is identical to pre-8.0 with respect to xfail tracebacks. While this is a behavior change, it brings default behavior back to pre-8.0.0 behavior, which ultimately was considered the better course of action.
-
#12281: Added support for keyword matching in marker expressions.
Now tests can be selected by marker keyword arguments. Supported values are
int
{.interpreted-text role="class"}, (unescaped)str
{.interpreted-text role="class"},bool
{.interpreted-text role="class"} &None
{.interpreted-text role="data"}.See
marker examples <marker_keyword_expression_example>
{.interpreted-text role="ref"} for more information.-- by
lovetheguitar
{.interpreted-text role="user"} -
#12567: Added
--no-fold-skipped
command line option.If this option is set, then skipped tests in short summary are no longer grouped by reason but all tests are printed individually with their nodeid in the same way as other statuses.
-- by
pbrezina
{.interpreted-text role="user"}
Improvements in existing functionality
-
#12469: The console output now uses the "third-party plugins" terminology, replacing the previously established but confusing and outdated reference to
setuptools <setuptools:index>
{.interpreted-text role="std:doc"} -- bywebknjaz
{.interpreted-text role="user"}. -
#12544, #12545: Python virtual environment detection was improved by checking for a
pyvenv.cfg
{.interpreted-text role="file"} file, ensuring reliable detection on various platforms -- byzachsnickers
{.interpreted-text role="user"}. -
#2871: Do not truncate arguments to functions in output when running with [-vvv]{.title-ref}.
-
#389: The readability of assertion introspection of bound methods has been enhanced -- by
farbodahm
{.interpreted-text role="user"},webknjaz
{.interpreted-text role="user"},obestwalter
{.interpreted-text role="user"},flub
{.interpreted-text role="user"} andglyphack
{.interpreted-text role="user"}.Earlier, it was like:
=================================== FAILURES =================================== _____________________________________ test _____________________________________ def test(): > assert Help().fun() == 2 E assert 1 == 2 E + where 1 = <bound method Help.fun of <example.Help instance at 0x256a830>>() E + where <bound method Help.fun of <example.Help instance at 0x256a830>> = <example.Help instance at 0x256a830>.fun E + where <example.Help instance at 0x256a830> = Help() example.py:7: AssertionError =========================== 1 failed in 0.03 seconds ===========================
And now it's like:
=================================== FAILURES =================================== _____________________________________ test _____________________________________ def test(): > assert Help().fun() == 2 E assert 1 == 2 E + where 1 = fun() E + where fun = <test_local.Help object at 0x1074be230>.fun E + where <test_local.Help object at 0x1074be230> = Help() test_local.py:13: AssertionError =========================== 1 failed in 0.03 seconds ===========================
-
#7662: Added timezone information to the testsuite timestamp in the JUnit XML report.
Bug fixes
-
#11706: Fixed reporting of teardown errors in higher-scoped fixtures when using [--maxfail]{.title-ref} or [--stepwise]{.title-ref}.
Originally added in pytest 8.0.0, but reverted in 8.0.2 due to a regression in pytest-xdist. This regression was fixed in pytest-xdist 3.6.1.
-
#11797:
pytest.approx
{.interpreted-text role="func"} now correctly handlesSequence <collections.abc.Sequence>
{.interpreted-text role="class"}-like objects. -
#12204, #12264: Fixed a regression in pytest 8.0 where tracebacks get longer and longer when multiple tests fail due to a shared higher-scope fixture which raised -- by
bluetech
{.interpreted-text role="user"}.Also fixed a similar regression in pytest 5.4 for collectors which raise during setup.
The fix necessitated internal changes which may affect some plugins:
-
FixtureDef.cached_result[2]
is now a tuple(exc, tb)
instead ofexc
. -
SetupState.stack
failures are now a tuple(exc, tb)
instead ofexc
.
-
-
#12275: Fixed collection error upon encountering an
abstract <abc>
{.interpreted-text role="mod"} class, including abstract [unittest.TestCase]{.title-ref} subclasses. -
#12328: Fixed a regression in pytest 8.0.0 where package-scoped parameterized items were not correctly reordered to minimize setups/teardowns in some cases.
-
#12424: Fixed crash with [assert testcase is not None]{.title-ref} assertion failure when re-running unittest tests using plugins like pytest-rerunfailures. Regressed in 8.2.2.
-
#12472: Fixed a crash when returning category
"error"
or"failed"
with a custom test status frompytest_report_teststatus
{.interpreted-text role="hook"} hook --pbrezina
{.interpreted-text role="user"}. -
#12505: Improved handling of invalid regex patterns in
pytest.raises(match=r'...') <pytest.raises>
{.interpreted-text role="func"} by providing a clear error message. -
#12580: Fixed a crash when using the cache class on Windows and the cache directory was created concurrently.
-
#6962: Parametrization parameters are now compared using [==]{.title-ref} instead of [is]{.title-ref} ([is]{.title-ref} is still used as a fallback if the parameter does not support [==]{.title-ref}). This fixes use of parameters such as lists, which have a different [id]{.title-ref} but compare equal, causing fixtures to be re-computed instead of being cached.
-
#7166: Fixed progress percentages (the
[ 87%]
at the edge of the screen) sometimes not aligning correctly when running with pytest-xdist-n
.
Improved documentation
-
#12153: Documented using
PYTEST_VERSION
{.interpreted-text role="envvar"} to detect if code is running from within a pytest run. -
#12469: The external plugin mentions in the documentation now avoid mentioning
setuptools entry-points <setuptools:index>
{.interpreted-text role="std:doc"} as the concept is much more generic nowadays. Instead, the terminology of "external", "installed", or "third-party" plugins (or packages) replaces that.-- by
webknjaz
{.interpreted-text role="user"} -
#12577: [CI]{.title-ref} and [BUILD_NUMBER]{.title-ref} environment variables role is discribed in the reference doc. They now also appear when doing [pytest -h]{.title-ref} -- by
MarcBresson
{.interpreted-text role="user"}.
Contributor-facing changes
-
#12467: Migrated all internal type-annotations to the python3.10+ style by using the [annotations]{.title-ref} future import.
-- by
RonnyPfannschmidt
{.interpreted-text role="user"} -
#11771, #12557: The PyPy runtime version has been updated to 3.9 from 3.8 that introduced a flaky bug at the garbage collector which was not expected to fix there as the 3.8 is EoL.
-- by
x612skm
{.interpreted-text role="user"} -
#12493: The change log draft preview integration has been refactored to use a third party extension
sphinxcontib-towncrier
. The previous in-repo script was putting the change log preview file atdoc/en/_changelog_towncrier_draft.rst
{.interpreted-text role="file"}. Said file is no longer ignored in Git and might show up among untracked files in the development environments of the contributors. To address that, the contributors can run the following command that will clean it up:$ git clean -x -i -- doc/en/_changelog_towncrier_draft.rst
-- by
webknjaz
{.interpreted-text role="user"} -
#12498: All the undocumented
tox
environments now have descriptions. They can be listed in one's development environment by invokingtox -av
in a terminal.-- by
webknjaz
{.interpreted-text role="user"} -
#12501: The changelog configuration has been updated to introduce more accurate audience-tailored categories. Previously, there was a
trivial
change log fragment type with an unclear and broad meaning. It was removed and we now havecontrib
,misc
andpackaging
in place of it.The new change note types target the readers who are downstream packagers and project contributors. Additionally, the miscellaneous section is kept for unspecified updates that do not fit anywhere else.
-- by
webknjaz
{.interpreted-text role="user"} -
#12502: The UX of the GitHub automation making pull requests to update the plugin list has been updated. Previously, the maintainers had to close the automatically created pull requests and re-open them to trigger the CI runs. From now on, they only need to click the [Ready for review]{.title-ref} button instead.
-- by
webknjaz
{.interpreted-text role="user"} -
#12522: The
:pull:
RST role has been replaced with a shorter:pr:
due to starting to use the implementation from the third-partysphinx-issues
{.interpreted-text role="pypi"} Sphinx extension -- bywebknjaz
{.interpreted-text role="user"}. -
#12531: The coverage reporting configuration has been updated to exclude pytest's own tests marked as expected to fail from the coverage report. This has an effect of reducing the influence of flaky tests on the resulting number.
-- by
webknjaz
{.interpreted-text role="user"} -
#12533: The
extlinks
Sphinx extension is no longer enabled. The:bpo:
role it used to declare has been removed with that. BPO itself has migrated to GitHub some years ago and it is possible to link the respective issues by using their GitHub issue numbers and the:issue:
role that thesphinx-issues
extension implements.-- by
webknjaz
{.interpreted-text role="user"} -
#12562: Possible typos in using the
:user:
RST role is now being linted through the pre-commit tool integration -- bywebknjaz
{.interpreted-text role="user"}.
v8.2.2
pytest 8.2.2 (2024-06-04)
Bug Fixes
- #12355: Fix possible catastrophic performance slowdown on a certain parametrization pattern involving many higher-scoped parameters.
- #12367: Fix a regression in pytest 8.2.0 where unittest class instances (a fresh one is created for each test) were not released promptly on test teardown but only on session teardown.
-
#12381: Fix possible "Directory not empty" crashes arising from concurent cache dir (
.pytest_cache
) creation. Regressed in pytest 8.2.0.
Improved Documentation
- #12290: Updated Sphinx theme to use Furo instead of Flask, enabling Dark mode theme.
- #12356: Added a subsection to the documentation for debugging flaky tests to mention lack of thread safety in pytest as a possible source of flakyness.
- #12363: The documentation webpages now links to a canonical version to reduce outdated documentation in search engine results.
v8.2.1
pytest 8.2.1 (2024-05-19)
Improvements
- #12334: Support for Python 3.13 (beta1 at the time of writing).
Bug Fixes
- #12120: Fix [PermissionError]{.title-ref} crashes arising from directories which are not selected on the command-line.
- #12191: Keyboard interrupts and system exits are now properly handled during the test collection.
- #12300: Fixed handling of 'Function not implemented' error under squashfuse_ll, which is a different way to say that the mountpoint is read-only.
-
#12308: Fix a regression in pytest 8.2.0 where the permissions of automatically-created
.pytest_cache
directories becamerwx------
instead of the expectedrwxr-xr-x
.
Trivial/Internal Changes
- #12333: pytest releases are now attested using the recent Artifact Attestation support from GitHub, allowing users to verify the provenance of pytest's sdist and wheel artifacts.
v8.2.0
pytest 8.2.0 (2024-04-27)
Deprecations
-
#12069: A deprecation warning is now raised when implementations of one of the following hooks request a deprecated
py.path.local
parameter instead of thepathlib.Path
parameter which replaced it:-
pytest_ignore_collect
{.interpreted-text role="hook"} - thepath
parameter - usecollection_path
instead. -
pytest_collect_file
{.interpreted-text role="hook"} - thepath
parameter - usefile_path
instead. -
pytest_pycollect_makemodule
{.interpreted-text role="hook"} - thepath
parameter - usemodule_path
instead. -
pytest_report_header
{.interpreted-text role="hook"} - thestartdir
parameter - usestart_path
instead. -
pytest_report_collectionfinish
{.interpreted-text role="hook"} - thestartdir
parameter - usestart_path
instead.
The replacement parameters are available since pytest 7.0.0. The old parameters will be removed in pytest 9.0.0.
See
legacy-path-hooks-deprecated
{.interpreted-text role="ref"} for more details. -
Features
-
#11871: Added support for reading command line arguments from a file using the prefix character
@
, like e.g.:pytest @​tests.txt
. The file must have one argument per line.See
Read arguments from file <args-from-file>
{.interpreted-text role="ref"} for details.
Improvements
-
#11523:
pytest.importorskip
{.interpreted-text role="func"} will now issue a warning if the module could be found, but raisedImportError
{.interpreted-text role="class"} instead ofModuleNotFoundError
{.interpreted-text role="class"}.The warning can be suppressed by passing
exc_type=ImportError
topytest.importorskip
{.interpreted-text role="func"}.See
import-or-skip-import-error
{.interpreted-text role="ref"} for details. -
#11728: For
unittest
-based tests, exceptions during class cleanup (as raised by functions registered withTestCase.addClassCleanup <unittest.TestCase.addClassCleanup>
{.interpreted-text role="meth"}) are now reported instead of silently failing. -
#11777: Text is no longer truncated in the
short test summary info
section when-vv
is given. -
#12112: Improved namespace packages detection when
consider_namespace_packages
{.interpreted-text role="confval"} is enabled, covering more situations (like editable installs). -
#9502: Added
PYTEST_VERSION
{.interpreted-text role="envvar"} environment variable which is defined at the start of the pytest session and undefined afterwards. It contains the value ofpytest.__version__
, and among other things can be used to easily check if code is running from within a pytest run.
Bug Fixes
-
#12065: Fixed a regression in pytest 8.0.0 where test classes containing
setup_method
and tests using@staticmethod
or@classmethod
would crash withAttributeError: 'NoneType' object has no attribute 'setup_method'
.Now the
request.instance <pytest.FixtureRequest.instance>
{.interpreted-text role="attr"} attribute of tests using@staticmethod
and@classmethod
is no longerNone
, but a fresh instance of the class, like in non-static methods. Previously it wasNone
, and all fixtures of such tests would share a singleself
. -
#12135: Fixed issue where fixtures adding their finalizer multiple times to fixtures they request would cause unreliable and non-intuitive teardown ordering in some instances.
-
#12194: Fixed a bug with
--importmode=importlib
and--doctest-modules
where child modules did not appear as attributes in parent modules. -
#1489: Fixed some instances where teardown of higher-scoped fixtures was not happening in the reverse order they were initialized in.
Trivial/Internal Changes
-
#12069:
pluggy>=1.5.0
is now required. -
#12167:
cache <cache>
{.interpreted-text role="ref"}: create supporting files (CACHEDIR.TAG
,.gitignore
, etc.) in a temporary directory to provide atomic semantics.
v8.1.2
pytest 8.1.2 (2024-04-26)
Bug Fixes
-
#12114: Fixed error in
pytest.approx
{.interpreted-text role="func"} when used with [numpy]{.title-ref} arrays and comparing with other types.
v8.1.1
pytest 8.1.1 (2024-03-08)
::: {.note} ::: {.title} Note :::
This release is not a usual bug fix release -- it contains features and improvements, being a follow up
to 8.1.0
, which has been yanked from PyPI.
:::
Features
-
#11475: Added the new
consider_namespace_packages
{.interpreted-text role="confval"} configuration option, defaulting toFalse
.If set to
True
, pytest will attempt to identify modules that are part of namespace packages when importing modules. -
#11653: Added the new
verbosity_test_cases
{.interpreted-text role="confval"} configuration option for fine-grained control of test execution verbosity. SeeFine-grained verbosity <pytest.fine_grained_verbosity>
{.interpreted-text role="ref"} for more details.
Improvements
-
#10865:
pytest.warns
{.interpreted-text role="func"} now validates thatwarnings.warn
{.interpreted-text role="func"} was called with a [str]{.title-ref} or a [Warning]{.title-ref}. Currently in Python it is possible to use other types, however this causes an exception whenwarnings.filterwarnings
{.interpreted-text role="func"} is used to filter those warnings (see CPython #103577 for a discussion). While this can be considered a bug in CPython, we decided to put guards in pytest as the error message produced without this check in place is confusing. -
#11311: When using
--override-ini
for paths in invocations without a configuration file defined, the current working directory is used as the relative directory.Previoulsy this would raise an
AssertionError
{.interpreted-text role="class"}. -
#11475:
--import-mode=importlib <import-mode-importlib>
{.interpreted-text role="ref"} now tries to import modules using the standard import mechanism (but still without changing :pysys.path
{.interpreted-text role="data"}), falling back to importing modules directly only if that fails.This means that installed packages will be imported under their canonical name if possible first, for example
app.core.models
, instead of having the module name always be derived from their path (for example.env310.lib.site_packages.app.core.models
). -
#11801: Added the
iter_parents() <_pytest.nodes.Node.iter_parents>
{.interpreted-text role="func"} helper method on nodes. It is similar tolistchain <_pytest.nodes.Node.listchain>
{.interpreted-text role="func"}, but goes from bottom to top, and returns an iterator, not a list. -
#11850: Added support for
sys.last_exc
{.interpreted-text role="data"} for post-mortem debugging on Python>=3.12. -
#11962: In case no other suitable candidates for configuration file are found, a
pyproject.toml
(even without a[tool.pytest.ini_options]
table) will be considered as the configuration file and define therootdir
. -
#11978: Add
--log-file-mode
option to the logging plugin, enabling appending to log-files. This option accepts either"w"
or"a"
and defaults to"w"
.Previously, the mode was hard-coded to be
"w"
which truncates the file before logging. -
#12047: When multiple finalizers of a fixture raise an exception, now all exceptions are reported as an exception group. Previously, only the first exception was reported.
Bug Fixes
-
#11475: Fixed regression where
--importmode=importlib
would import non-test modules more than once. -
#11904: Fixed a regression in pytest 8.0.0 that would cause test collection to fail due to permission errors when using
--pyargs
.This change improves the collection tree for tests specified using
--pyargs
, see12043
{.interpreted-text role="pull"} for a comparison with pytest 8.0 and <8. -
#12011: Fixed a regression in 8.0.1 whereby
setup_module
xunit-style fixtures are not executed when--doctest-modules
is passed. -
#12014: Fix the
stacklevel
used when warning about marks used on fixtures. -
#12039: Fixed a regression in
8.0.2
where tests created usingtmp_path
{.interpreted-text role="fixture"} have been collected multiple times in CI under Windows.
Improved Documentation
-
#11790: Documented the retention of temporary directories created using the
tmp_path
fixture in more detail.
Trivial/Internal Changes
-
#11785: Some changes were made to private functions which may affect plugins which access them:
-
FixtureManager._getautousenames()
now takes aNode
itself instead of the nodeid. -
FixtureManager.getfixturedefs()
now takes theNode
itself instead of the nodeid. - The
_pytest.nodes.iterparentnodeids()
function is removed without replacement. Prefer to traverse the node hierarchy itself instead. If you really need to, copy the function from the previous pytest release.
-
-
#12069: Delayed the deprecation of the following features to
9.0.0
:-
node-ctor-fspath-deprecation
{.interpreted-text role="ref"}. -
legacy-path-hooks-deprecated
{.interpreted-text role="ref"}.
It was discovered after
8.1.0
was released that the warnings about the impeding removal were not being displayed, so the team decided to revert the removal.This is the reason for
8.1.0
being yanked. -
pytest 8.1.0 (YANKED)
::: {.note} ::: {.title} Note :::
This release has been yanked: it broke some plugins without the proper warning period, due to some warnings not showing up as expected.
See #12069. :::
v8.1.0
pytest 8.1.0 (YANKED)
[!IMPORTANT]
This release has been yanked: it broke some plugins without the proper warning period, due to some warnings not showing up as expected. See #12069.
Features
-
#11475: Added the new
consider_namespace_packages
{.interpreted-text role="confval"} configuration option, defaulting toFalse
.If set to
True
, pytest will attempt to identify modules that are part of namespace packages when importing modules. -
#11653: Added the new
verbosity_test_cases
{.interpreted-text role="confval"} configuration option for fine-grained control of test execution verbosity. SeeFine-grained verbosity <pytest.fine_grained_verbosity>
{.interpreted-text role="ref"} for more details.
Improvements
-
#10865:
pytest.warns
{.interpreted-text role="func"} now validates thatwarnings.warn
{.interpreted-text role="func"} was called with a [str]{.title-ref} or a [Warning]{.title-ref}. Currently in Python it is possible to use other types, however this causes an exception whenwarnings.filterwarnings
{.interpreted-text role="func"} is used to filter those warnings (see CPython #103577 for a discussion). While this can be considered a bug in CPython, we decided to put guards in pytest as the error message produced without this check in place is confusing. -
#11311: When using
--override-ini
for paths in invocations without a configuration file defined, the current working directory is used as the relative directory.Previoulsy this would raise an
AssertionError
{.interpreted-text role="class"}. -
#11475:
--import-mode=importlib <import-mode-importlib>
{.interpreted-text role="ref"} now tries to import modules using the standard import mechanism (but still without changing :pysys.path
{.interpreted-text role="data"}), falling back to importing modules directly only if that fails.This means that installed packages will be imported under their canonical name if possible first, for example
app.core.models
, instead of having the module name always be derived from their path (for example.env310.lib.site_packages.app.core.models
). -
#11801: Added the
iter_parents() <_pytest.nodes.Node.iter_parents>
{.interpreted-text role="func"} helper method on nodes. It is similar tolistchain <_pytest.nodes.Node.listchain>
{.interpreted-text role="func"}, but goes from bottom to top, and returns an iterator, not a list. -
#11850: Added support for
sys.last_exc
{.interpreted-text role="data"} for post-mortem debugging on Python>=3.12. -
#11962: In case no other suitable candidates for configuration file are found, a
pyproject.toml
(even without a[tool.pytest.ini_options]
table) will be considered as the configuration file and define therootdir
. -
#11978: Add
--log-file-mode
option to the logging plugin, enabling appending to log-files. This option accepts either"w"
or"a"
and defaults to"w"
.Previously, the mode was hard-coded to be
"w"
which truncates the file before logging. -
#12047: When multiple finalizers of a fixture raise an exception, now all exceptions are reported as an exception group. Previously, only the first exception was reported.
Bug Fixes
-
#11904: Fixed a regression in pytest 8.0.0 that would cause test collection to fail due to permission errors when using
--pyargs
.This change improves the collection tree for tests specified using
--pyargs
, see12043
{.interpreted-text role="pull"} for a comparison with pytest 8.0 and <8. -
#12011: Fixed a regression in 8.0.1 whereby
setup_module
xunit-style fixtures are not executed when--doctest-modules
is passed. -
#12014: Fix the
stacklevel
used when warning about marks used on fixtures. -
#12039: Fixed a regression in
8.0.2
where tests created usingtmp_path
{.interpreted-text role="fixture"} have been collected multiple times in CI under Windows.
Improved Documentation
-
#11790: Documented the retention of temporary directories created using the
tmp_path
fixture in more detail.
Trivial/Internal Changes
-
#11785: Some changes were made to private functions which may affect plugins which access them:
-
FixtureManager._getautousenames()
now takes aNode
itself instead of the nodeid. -
FixtureManager.getfixturedefs()
now takes theNode
itself instead of the nodeid. - The
_pytest.nodes.iterparentnodeids()
function is removed without replacement. Prefer to traverse the node hierarchy itself instead. If you really need to, copy the function from the previous pytest release.
-
v8.0.2
pytest 8.0.2 (2024-02-24)
Bug Fixes
-
#11895: Fix collection on Windows where initial paths contain the short version of a path (for example
c:\MROGRA~1\tests
). -
#11953: Fix an
IndexError
crash raising fromgetstatementrange_ast
. - #12021: Reverted a fix to [--maxfail]{.title-ref} handling in pytest 8.0.0 because it caused a regression in pytest-xdist whereby session fixture teardowns may get executed multiple times when the max-fails is reached.
v8.0.1
pytest 8.0.1 (2024-02-16)
Bug Fixes
-
#11875: Correctly handle errors from
getpass.getuser
{.interpreted-text role="func"} in Python 3.13. -
#11879: Fix an edge case where
ExceptionInfo._stringify_exception
could crashpytest.raises
{.interpreted-text role="func"}. -
#11906: Fix regression with
pytest.warns
{.interpreted-text role="func"} using custom warning subclasses which have more than one parameter in their [__init__]{.title-ref}. -
#11907: Fix a regression in pytest 8.0.0 whereby calling
pytest.skip
{.interpreted-text role="func"} and similar control-flow exceptions within apytest.warns()
{.interpreted-text role="func"} block would get suppressed instead of propagating. - #11929: Fix a regression in pytest 8.0.0 whereby autouse fixtures defined in a module get ignored by the doctests in the module.
- #11937: Fix a regression in pytest 8.0.0 whereby items would be collected in reverse order in some circumstances.
v8.0.0
See 8.0.0rc1 and 8.0.0rc2 for the full changes since pytest 7.4!
Bug Fixes
-
#11842: Properly escape the
reason
of askip <pytest.mark.skip ref>
{.interpreted-text role="ref"} mark when writing JUnit XML files. -
#11861: Avoid microsecond exceeds
1_000_000
when usinglog-date-format
with%f
specifier, which might cause the test suite to crash.
v7.4.4
Bug Fixes
- #11140: Fix non-string constants at the top of file being detected as docstrings on Python>=3.8.
-
#11572: Handle an edge case where
sys.stderr
{.interpreted-text role="data"} andsys.__stderr__
{.interpreted-text role="data"} might already be closed whenfaulthandler
{.interpreted-text role="ref"} is tearing down. - #11710: Fixed tracebacks from collection errors not getting pruned.
-
#7966: Removed unhelpful error message from assertion rewrite mechanism when exceptions are raised in
__iter__
methods. Now they are treated un-iterable instead.
Improved Documentation
-
#11091: Updated documentation to refer to hyphenated options: replaced
--junitxml
with--junit-xml
and--collectonly
with--collect-only
.
v7.4.3
Bug Fixes
-
#10447: Markers are now considered in the reverse mro order to ensure base class markers are considered first -- this resolves a regression.
-
#11239: Fixed
:=
in asserts impacting unrelated test cases. -
#11439: Handled an edge case where :data:
sys.stderr
might already be closed when :ref:faulthandler
is tearing down.
v7.4.2
Bug Fixes
-
#11237: Fix doctest collection of
functools.cached_property
objects. -
#11306: Fixed bug using
--importmode=importlib
which would cause package__init__.py
files to be imported more than once in some cases. -
#11367: Fixed bug where
user_properties
where not being saved in the JUnit XML file if a fixture failed during teardown. -
#11394: Fixed crash when parsing long command line arguments that might be interpreted as files.
Improved Documentation
- #11391: Improved disclaimer on pytest plugin reference page to better indicate this is an automated, non-curated listing.
v7.4.1
Bug Fixes
-
#10337: Fixed bug where fake intermediate modules generated by
--import-mode=importlib
would not include the child modules as attributes of the parent modules. -
#10702: Fixed error assertion handling in
pytest.approx
whenNone
is an expected or received value when comparing dictionaries. -
#10811: Fixed issue when using
--import-mode=importlib
together with--doctest-modules
that caused modules to be imported more than once, causing problems with modules that have import side effects.
v7.4.0
pytest 7.4.0 (2023-06-23)
Features
-
#10901: Added
ExceptionInfo.from_exception() <pytest.ExceptionInfo.from_exception>
{.interpreted-text role="func"}, a simpler way to create an~pytest.ExceptionInfo
{.interpreted-text role="class"} from an exception. This can replaceExceptionInfo.from_exc_info() <pytest.ExceptionInfo.from_exc_info()>
{.interpreted-text role="func"} for most uses.
Improvements
-
#10872: Update test log report annotation to named tuple and fixed inconsistency in docs for
pytest_report_teststatus
{.interpreted-text role="hook"} hook. -
#10907: When an exception traceback to be displayed is completely filtered out (by mechanisms such as
__tracebackhide__
, internal frames, and similar), now only the exception string and the following message are shown:"All traceback entries are hidden. Pass [--full-trace]{.title-ref} to see hidden and internal frames.".
Previously, the last frame of the traceback was shown, even though it was hidden.
-
#10940: Improved verbose output (
-vv
) ofskip
andxfail
reasons by performing text wrapping while leaving a clear margin for progress output.Added
TerminalReporter.wrap_write()
as a helper for that. -
#10991: Added handling of
%f
directive to print microseconds in log format options, such aslog-date-format
. -
#11005: Added the underlying exception to the cache provider's path creation and write warning messages.
-
#11013: Added warning when
testpaths
{.interpreted-text role="confval"} is set, but paths are not found by glob. In this case, pytest will fall back to searching from the current directory. -
#11043: When [--confcutdir]{.title-ref} is not specified, and there is no config file present, the conftest cutoff directory ([--confcutdir]{.title-ref}) is now set to the
rootdir <rootdir>
{.interpreted-text role="ref"}. Previously in such cases, [conftest.py]{.title-ref} files would be probed all the way to the root directory of the filesystem. If you are badly affected by this change, consider adding an empty config file to your desired cutoff directory, or explicitly set [--confcutdir]{.title-ref}. -
#11081: The
norecursedirs
{.interpreted-text role="confval"} check is now performed in apytest_ignore_collect
{.interpreted-text role="hook"} implementation, so plugins can affect it.If after updating to this version you see that your [norecursedirs]{.title-ref} setting is not being respected, it means that a conftest or a plugin you use has a bad [pytest_ignore_collect]{.title-ref} implementation. Most likely, your hook returns [False]{.title-ref} for paths it does not want to ignore, which ends the processing and doesn't allow other plugins, including pytest itself, to ignore the path. The fix is to return [None]{.title-ref} instead of [False]{.title-ref} for paths your hook doesn't want to ignore.
-
#8711:
caplog.set_level() <pytest.LogCaptureFixture.set_level>
{.interpreted-text role="func"} andcaplog.at_level() <pytest.LogCaptureFixture.at_level>
{.interpreted-text role="func"} will temporarily enable the requestedlevel
iflevel
was disabled globally vialogging.disable(LEVEL)
.
Bug Fixes
-
#10831: Terminal Reporting: Fixed bug when running in
--tb=line
mode wherepytest.fail(pytrace=False)
tests reportNone
. -
#11068: Fixed the
--last-failed
whole-file skipping functionality ("skipped N files") fornon-python test files <non-python tests>
{.interpreted-text role="ref"}. -
#11104: Fixed a regression in pytest 7.3.2 which caused to
testpaths
{.interpreted-text role="confval"} to be considered for loading initial conftests, even when it was not utilized (e.g. when explicit paths were given on the command line). Now thetestpaths
are only considered when they are in use. -
#1904: Fixed traceback entries hidden with
__tracebackhide__ = True
still being shown for chained exceptions (parts after "... the above exception ..." message). -
#7781: Fix writing non-encodable text to log file when using
--debug
.
Improved Documentation
-
#9146: Improved documentation for
caplog.set_level() <pytest.LogCaptureFixture.set_level>
{.interpreted-text role="func"}.
Trivial/Internal Changes
-
#11031: Enhanced the CLI flag for
-c
to now include--config-file
to make it clear that this flag applies to the usage of a custom config file.
v7.3.2
pytest 7.3.2 (2023-06-10)
Bug Fixes
-
#10169: Fix bug where very long option names could cause pytest to break with
OSError: [Errno 36] File name too long
on some systems. - #10894: Support for Python 3.12 (beta at the time of writing).
-
#10987:
testpaths
{.interpreted-text role="confval"} is now honored to load rootconftests
. - #10999: The [monkeypatch]{.title-ref} [setitem]{.title-ref}/[delitem]{.title-ref} type annotations now allow [TypedDict]{.title-ref} arguments.
- #11028: Fixed bug in assertion rewriting where a variable assigned with the walrus operator could not be used later in a function call.
-
#11054: Fixed
--last-failed
's "(skipped N files)" functionality for files inside of packages (directories with [__init__.py]{.title-ref} files).
v7.3.1
pytest 7.3.1 (2023-04-14)
Improvements
-
#10875: Python 3.12 support: fixed
RuntimeError: TestResult has no addDuration method
when runningunittest
tests. -
#10890: Python 3.12 support: fixed
shutil.rmtree(onerror=...)
deprecation warning when usingtmp_path
{.interpreted-text role="fixture"}.
Bug Fixes
-
#10896: Fixed performance regression related to
tmp_path
{.interpreted-text role="fixture"} and the newtmp_path_retention_policy
{.interpreted-text role="confval"} option. -
#10903: Fix crash
INTERNALERROR IndexError: list index out of range
which happens when displaying an exception where all entries are hidden. This reverts the change "Correctly handle__tracebackhide__
for chained exceptions." introduced in version 7.3.0.
v7.3.0
pytest 7.3.0 (2023-04-08)
Features
-
#10525: Test methods decorated with
@classmethod
can now be discovered as tests, following the same rules as normal methods. This fills the gap that static methods were discoverable as tests but not class methods. -
#10755:
console_output_style
{.interpreted-text role="confval"} now supportsprogress-even-when-capture-no
to force the use of the progress output even when capture is disabled. This is useful in large test suites where capture may have significant performance impact. -
#7431:
--log-disable
CLI option added to disable individual loggers. -
#8141: Added
tmp_path_retention_count
{.interpreted-text role="confval"} andtmp_path_retention_policy
{.interpreted-text role="confval"} configuration options to control how directories created by thetmp_path
{.interpreted-text role="fixture"} fixture are kept.
Improvements
-
#10226: If multiple errors are raised in teardown, we now re-raise an
ExceptionGroup
of them instead of discarding all but the last. -
#10658: Allow
-p
arguments to include spaces (eg:-p no:logging
instead of-pno:logging
). Mostly useful in theaddopts
section of the configuration file. -
#10710: Added
start
andstop
timestamps toTestReport
objects. -
#10727: Split the report header for
rootdir
,config file
andtestpaths
so each has its own line. - #10840: pytest should no longer crash on AST with pathological position attributes, for example testing AST produced by [Hylang <https://github.com/hylang/hy>\__]{.title-ref}.
- #6267: The full output of a test is no longer truncated if the truncation message would be longer than the hidden text. The line number shown has also been fixed.
Bug Fixes
- #10743: The assertion rewriting mechanism now works correctly when assertion expressions contain the walrus operator.
-
#10765: Fixed
tmp_path
{.interpreted-text role="fixture"} fixture always raisingOSError
{.interpreted-text role="class"} onemscripten
platform due to missingos.getuid
{.interpreted-text role="func"}. -
#1904: Correctly handle
__tracebackhide__
for chained exceptions.
Improved Documentation
-
#10782: Fixed the minimal example in
goodpractices
{.interpreted-text role="ref"}:pip install -e .
requires aversion
entry inpyproject.toml
to run successfully.
Trivial/Internal Changes
- #10669: pytest no longer depends on the [attrs]{.title-ref} package (don't worry, nice diffs for attrs classes are still supported).
v7.2.2
pytest 7.2.2 (2023-03-03)
Bug Fixes
-
#10533: Fixed
pytest.approx
{.interpreted-text role="func"} handling of dictionaries containing one or more values of [0.0]{.title-ref}. - #10592: Fixed crash if [--cache-show]{.title-ref} and [--help]{.title-ref} are passed at the same time.
-
#10597: Fixed bug where a fixture method named
teardown
would be called as part ofnose
teardown stage. -
#10626: Fixed crash if
--fixtures
and--help
are passed at the same time. -
#10660: Fixed :py
pytest.raises
{.interpreted-text role="func"} to return a 'ContextManager' so that type-checkers could narrowpytest.raises(...) if ... else nullcontext()
down to 'ContextManager' rather than 'object'.
Improved Documentation
- #10690: Added [CI]{.title-ref} and [BUILD_NUMBER]{.title-ref} environment variables to the documentation.
- #10721: Fixed entry-points declaration in the documentation example using Hatch.
- #10753: Changed wording of the module level skip to be very explicit about not collecting tests and not executing the rest of the module.
v7.2.1
pytest 7.2.1 (2023-01-13)
Bug Fixes
- #10452: Fix 'importlib.abc.TraversableResources' deprecation warning in Python 3.12.
- #10457: If a test is skipped from inside a fixture, the test summary now shows the test location instead of the fixture location.
-
#10506: Fix bug where sometimes pytest would use the file system root directory as
rootdir <rootdir>
{.interpreted-text role="ref"} on Windows. - #10607: Fix a race condition when creating junitxml reports, which could occur when multiple instances of pytest execute in parallel.
- #10641: Fix a race condition when creating or updating the stepwise plugin's cache, which could occur when multiple xdist worker nodes try to simultaneously update the stepwise plugin's cache.
v7.2.0
pytest 7.2.0 (2022-10-23)
Deprecations
-
#10012: Update
pytest.PytestUnhandledCoroutineWarning
{.interpreted-text role="class"} to a deprecation; it will raise an error in pytest 8. -
#10396: pytest no longer depends on the
py
library.pytest
provides a vendored copy ofpy.error
andpy.path
modules but will use thepy
library if it is installed. If you need otherpy.*
modules, continue to install the deprecatedpy
library separately, otherwise it can usually be removed as a dependency. -
#4562: Deprecate configuring hook specs/impls using attributes/marks.
Instead use :py
pytest.hookimpl
{.interpreted-text role="func"} and :pypytest.hookspec
{.interpreted-text role="func"}. For more details, see thedocs <legacy-path-hooks-deprecated>
{.interpreted-text role="ref"}. -
#9886: The functionality for running tests written for
nose
has been officially deprecated.This includes:
- Plain
setup
andteardown
functions and methods: this might catch users by surprise, assetup()
andteardown()
are not pytest idioms, but part of thenose
support. - Setup/teardown using the @with_setup decorator.
For more details, consult the
deprecation docs <nose-deprecation>
{.interpreted-text role="ref"}. - Plain
Features
-
#9897: Added shell-style wildcard support to
testpaths
.
Improvements
-
#10218:
@pytest.mark.parametrize()
(and similar functions) now accepts anySequence[str]
for the argument names, instead of justlist[str]
andtuple[str, ...]
.(Note that
str
, which is itself aSequence[str]
, is still treated as a comma-delimited name list, as before). -
#10381: The
--no-showlocals
flag has been added. This can be passed directly to tests to override--showlocals
declared throughaddopts
. -
#3426: Assertion failures with strings in NFC and NFD forms that normalize to the same string now have a dedicated error message detailing the issue, and their utf-8 representation is expresed instead.
-
#7337: A warning is now emitted if a test function returns something other than [None]{.title-ref}. This prevents a common mistake among beginners that expect that returning a [bool]{.title-ref} (for example [return foo(a, b) == result]{.title-ref}) would cause a test to pass or fail, instead of using [assert]{.title-ref}.
-
#8508: Introduce multiline display for warning matching via :py
pytest.warns
{.interpreted-text role="func"} and enhance match comparison for :py_pytest._code.ExceptionInfo.match
{.interpreted-text role="func"} as returned by :pypytest.raises
{.interpreted-text role="func"}. -
#8646: Improve :py
pytest.raises
{.interpreted-text role="func"}. Previously passing an empty tuple would give a confusing error. We now raise immediately with a more helpful message. -
#9741: On Python 3.11, use the standard library's
tomllib
{.interpreted-text role="mod"} to parse TOML.tomli
{.interpreted-text role="mod"}` is no longer a dependency on Python 3.11. -
#9742: Display assertion message without escaped newline characters with
-vv
. -
#9823: Improved error message that is shown when no collector is found for a given file.
-
#9873: Some coloring has been added to the short test summary.
-
#9883: Normalize the help description of all command-line options.
-
#9920: Display full crash messages in
short test summary info
, when runng in a CI environment. -
#9987: Added support for hidden configuration file by allowing
.pytest.ini
as an alternative topytest.ini
.
Bug Fixes
-
#10150:
sys.stdin
{.interpreted-text role="data"} now contains all expected methods of a file-like object when capture is enabled. -
#10382: Do not break into pdb when
raise unittest.SkipTest()
appears top-level in a file. -
#7792: Marks are now inherited according to the full MRO in test classes. Previously, if a test class inherited from two or more classes, only marks from the first super-class would apply.
When inheriting marks from super-classes, marks from the sub-classes are now ordered before marks from the super-classes, in MRO order. Previously it was the reverse.
When inheriting marks from super-classes, the [pytestmark]{.title-ref} attribute of the sub-class now only contains the marks directly applied to it. Previously, it also contained marks from its super-classes. Please note that this attribute should not normally be accessed directly; use
pytest.Node.iter_markers
{.interpreted-text role="func"} instead. -
#9159: Showing inner exceptions by forcing native display in
ExceptionGroups
even when using display options other than--tb=native
. A temporary step before full implementation of pytest-native display for inner exceptions inExceptionGroups
. -
#9877: Ensure
caplog.get_records(when)
returns current/correct data after invokingcaplog.clear()
.
Improved Documentation
-
#10344: Update information on writing plugins to use
pyproject.toml
instead ofsetup.py
. - #9248: The documentation is now built using Sphinx 5.x (up from 3.x previously).
-
#9291: Update documentation on how
pytest.warns
{.interpreted-text role="func"} affectsDeprecationWarning
{.interpreted-text role="class"}.
Trivial/Internal Changes
-
#10196:
~pytest.PytestReturnNotNoneWarning
{.interpreted-text role="class"} is now a subclass of~pytest.PytestRemovedIn8Warning
{.interpreted-text role="class"}: the plan is to make returning non-None
from tests an error in the future. -
#10313: Made
_pytest.doctest.DoctestItem
exportpytest.DoctestItem
for type check and runtime purposes. Made [_pytest.doctest]{.title-ref} use internal APIs to avoid circular imports. -
#9906: Made
_pytest.compat
re-exportimportlib_metadata
in the eyes of type checkers. -
#9910: Fix default encoding warning (
EncodingWarning
) incacheprovider
- #9984: Improve the error message when we attempt to access a fixture that has been torn down. Add an additional sentence to the docstring explaining when it's not a good idea to call getfixturevalue.
v7.1.3
pytest 7.1.3 (2022-08-31)
Bug Fixes
-
#10060: When running with
--pdb
,TestCase.tearDown
is no longer called for tests when the class has been skipped viaunittest.skip
orpytest.mark.skip
. - #10190: Invalid XML characters in setup or teardown error messages are now properly escaped for JUnit XML reports.
-
#10230: Ignore
.py
files created bypyproject.toml
-based editable builds introduced in pip 21.3. -
#3396: Doctests now respect the
--import-mode
flag. -
#9514: Type-annotate
FixtureRequest.param
asAny
as a stop gap measure until8073
{.interpreted-text role="issue"} is fixed. -
#9791: Fixed a path handling code in
rewrite.py
that seems to work fine, but was incorrect and fails in some systems. -
#9917: Fixed string representation for
pytest.approx
{.interpreted-text role="func"} when used to compare tuples.
Improved Documentation
-
#9937: Explicit note that
tmpdir
{.interpreted-text role="fixture"} fixture is discouraged in favour oftmp_path
{.interpreted-text role="fixture"}.
Trivial/Internal Changes
- #10114: Replace atomicwrites dependency on windows with [os.replace]{.title-ref}.
v7.1.2
pytest 7.1.2 (2022-04-23)
Bug Fixes
-
#9726: An unnecessary
numpy
import insidepytest.approx
{.interpreted-text role="func"} was removed. -
#9820: Fix comparison of
dataclasses
withInitVar
. -
#9869: Increase
stacklevel
for theNODE_CTOR_FSPATH_ARG
deprecation to point to the user's code, not pytest. - #9871: Fix a bizarre (and fortunately rare) bug where the [temp_path]{.title-ref} fixture could raise an internal error while attempting to get the current user's username.
v7.1.1
pytest 7.1.1 (2022-03-17)
Bug Fixes
- #9767: Fixed a regression in pytest 7.1.0 where some conftest.py files outside of the source tree (e.g. in the [site-packages]{.title-ref} directory) were not picked up.
v7.1.0
pytest 7.1.0 (2022-03-13)
Breaking Changes
-
#8838: As per our policy, the following features have been deprecated in the 6.X series and are now removed:
-
pytest._fillfuncargs
function. -
pytest_warning_captured
hook - usepytest_warning_recorded
instead. -
-k -foobar
syntax - use-k 'not foobar'
instead. -
-k foobar:
syntax. -
pytest.collect
module - import frompytest
directly.
For more information consult Deprecations and Removals in the docs.
-
-
#9437: Dropped support for Python 3.6, which reached end-of-life at 2021-12-23.
Improvements
-
#5192: Fixed test output for some data types where
-v
would show less information.Also, when showing diffs for sequences,
-q
would produce full diffs instead of the expected diff. -
#9362: pytest now avoids specialized assert formatting when it is detected that the default
__eq__
is overridden inattrs
ordataclasses
. -
#9536: When
-vv
is given on command line, show skipping and xfail reasons in full instead of truncating them to fit the terminal width. -
#9644: More information about the location of resources that led Python to raise
ResourceWarning
{.interpreted-text role="class"} can now be obtained by enablingtracemalloc
{.interpreted-text role="mod"}.See
resource-warnings
{.interpreted-text role="ref"} for more information. -
#9678: More types are now accepted in the
ids
argument to@pytest.mark.parametrize
. Previously only [str]{.title-ref}, [float]{.title-ref}, [int]{.title-ref} and [bool]{.title-ref} were accepted; now [bytes]{.title-ref}, [complex]{.title-ref}, [re.Pattern]{.title-ref}, [Enum]{.title-ref} and anything with a [__name__]{.title-ref} are also accepted. -
#9692:
pytest.approx
{.interpreted-text role="func"} now raises aTypeError
{.interpreted-text role="class"} when given an unordered sequence (such asset
{.interpreted-text role="class"}).Note that this implies that custom classes which only implement
__iter__
and__len__
are no longer supported as they don't guarantee order.
Bug Fixes
-
#8242: The deprecation of raising
unittest.SkipTest
{.interpreted-text role="class"} to skip collection of tests during the pytest collection phase is reverted - this is now a supported feature again. -
#9493: Symbolic link components are no longer resolved in conftest paths. This means that if a conftest appears twice in collection tree, using symlinks, it will be executed twice. For example, given
tests/real/conftest.py tests/real/test_it.py tests/link -> tests/real
running
pytest tests
now imports the conftest twice, once astests/real/conftest.py
and once astests/link/conftest.py
. This is a fix to match a similar change made to test collection itself in pytest 6.0 (see6523
{.interpreted-text role="pull"} for details). -
#9626: Fixed count of selected tests on terminal collection summary when there were errors or skipped modules.
If there were errors or skipped modules on collection, pytest would mistakenly subtract those from the selected count.
-
#9645: Fixed regression where
--import-mode=importlib
used together withPYTHONPATH
{.interpreted-text role="envvar"} orpythonpath
{.interpreted-text role="confval"} would cause import errors in test suites. -
#9708:
pytester
{.interpreted-text role="fixture"} now requests amonkeypatch
{.interpreted-text role="fixture"} fixture instead of creating one internally. This solves some issues with tests that involve pytest environment variables. -
#9730: Malformed
pyproject.toml
files now produce a clearer error message.
v7.0.1
pytest 7.0.1 (2022-02-11)
Bug Fixes
-
#9608: Fix invalid importing of
importlib.readers
in Python 3.9. - #9610: Restore [UnitTestFunction.obj]{.title-ref} to return unbound rather than bound method. Fixes a crash during a failed teardown in unittest TestCases with non-default [__init__]{.title-ref}. Regressed in pytest 7.0.0.
-
#9636: The
pythonpath
plugin was renamed topython_path
. This avoids a conflict with thepytest-pythonpath
plugin. -
#9642: Fix running tests by id with
::
in the parametrize portion. -
#9643: Delay issuing a
~pytest.PytestWarning
{.interpreted-text role="class"} about diamond inheritance involving~pytest.Item
{.interpreted-text role="class"} and~pytest.Collector
{.interpreted-text role="class"} so it can be filtered usingstandard warning filters <warnings>
{.interpreted-text role="ref"}.
v7.0.0
pytest 7.0.0 (2022-02-03)
(Please see the full set of changes for this release also in the 7.0.0rc1 notes below)
Deprecations
-
#9488: If custom subclasses of nodes like
pytest.Item
{.interpreted-text role="class"} override the__init__
method, they should take**kwargs
. Seeuncooperative-constructors-deprecated
{.interpreted-text role="ref"} for details.Note that a deprection warning is only emitted when there is a conflict in the arguments pytest expected to pass. This deprecation was already part of pytest 7.0.0rc1 but wasn't documented.
Bug Fixes
- #9355: Fixed error message prints function decorators when using assert in Python 3.8 and above.
-
#9396: Ensure
pytest.Config.inifile
{.interpreted-text role="attr"} is available during thepytest_cmdline_main <_pytest.hookspec.pytest_cmdline_main>
{.interpreted-text role="func"} hook (regression during7.0.0rc1
).
Improved Documentation
- #9404: Added extra documentation on alternatives to common misuses of [pytest.warns(None)]{.title-ref} ahead of its deprecation.
- #9505: Clarify where the configuration files are located. To avoid confusions documentation mentions that configuration file is located in the root of the repository.
Trivial/Internal Changes
- #9521: Add test coverage to assertion rewrite path.
pytest 7.0.0rc1 (2021-12-06)
Breaking Changes
-
#7259: The
Node.reportinfo() <non-python tests>
{.interpreted-text role="ref"} function first return value type has been expanded from [py.path.local | str]{.title-ref} to [os.PathLike[str] | str]{.title-ref}.Most plugins which refer to [reportinfo()]{.title-ref} only define it as part of a custom
pytest.Item
{.interpreted-text role="class"} implementation. Since [py.path.local]{.title-ref} is a [os.PathLike[str]]{.title-ref}, these plugins are unaffacted.Plugins and users which call [reportinfo()]{.title-ref}, use the first return value and interact with it as a [py.path.local]{.title-ref}, would need to adjust by calling [py.path.local(fspath)]{.title-ref}. Although preferably, avoid the legacy [py.path.local]{.title-ref} and use [pathlib.Path]{.title-ref}, or use [item.location]{.title-ref} or [item.path]{.title-ref}, instead.
Note: pytest was not able to provide a deprecation period for this change.
-
#8246:
--version
now writes version information tostdout
rather thanstderr
. -
#8733: Drop a workaround for pyreadline that made it work with
--pdb
.The workaround was introduced in #1281 in 2015, however since then pyreadline seems to have gone unmaintained, is generating warnings, and will stop working on Python 3.10.
-
#9061: Using
pytest.approx
{.interpreted-text role="func"} in a boolean context now raises an error hinting at the proper usage.It is apparently common for users to mistakenly use
pytest.approx
like this:assert pytest.approx(actual, expected)
While the correct usage is:
assert actual == pytest.approx(expected)
The new error message helps catch those mistakes.
-
#9277: The
pytest.Instance
collector type has been removed. Importingpytest.Instance
or_pytest.python.Instance
returns a dummy type and emits a deprecation warning. Seeinstance-collector-deprecation
{.interpreted-text role="ref"} for details. -
#9308: PytestRemovedIn7Warning deprecation warnings are now errors by default.
Following our plan to remove deprecated features with as little disruption as possible, all warnings of type
PytestRemovedIn7Warning
now generate errors instead of warning messages by default.The affected features will be effectively removed in pytest 7.1, so please consult the
deprecations
{.interpreted-text role="ref"} section in the docs for directions on how to update existing code.In the pytest
7.0.X
series, it is possible to change the errors back into warnings as a stopgap measure by adding this to yourpytest.ini
file:[pytest] filterwarnings = ignore::pytest.PytestRemovedIn7Warning
But this will stop working when pytest
7.1
is released.If you have concerns about the removal of a specific feature, please add a comment to
9308
{.interpreted-text role="issue"}.
Deprecations
-
#7259:
py.path.local
arguments for hooks have been deprecated. Seethe deprecation note <legacy-path-hooks-deprecated>
{.interpreted-text role="ref"} for full details.py.path.local
arguments to Node constructors have been deprecated. Seethe deprecation note <node-ctor-fspath-deprecation>
{.interpreted-text role="ref"} for full details.::: {.note} ::: {.admonition-title} Note :::
The name of the
~_pytest.nodes.Node
{.interpreted-text role="class"} arguments and attributes (the new attribute beingpath
) is the opposite of the situation for hooks (the old argument beingpath
).This is an unfortunate artifact due to historical reasons, which should be resolved in future versions as we slowly get rid of the
py
{.interpreted-text role="pypi"} dependency (see9283
{.interpreted-text role="issue"} for a longer discussion). ::: -
#7469: Directly constructing the following classes is now deprecated:
_pytest.mark.structures.Mark
_pytest.mark.structures.MarkDecorator
_pytest.mark.structures.MarkGenerator
_pytest.python.Metafunc
_pytest.runner.CallInfo
_pytest._code.ExceptionInfo
_pytest.config.argparsing.Parser
_pytest.config.argparsing.OptionGroup
_pytest.pytester.HookRecorder
These constructors have always been considered private, but now issue a deprecation warning, which may become a hard error in pytest 8.
-
#8242: Raising
unittest.SkipTest
{.interpreted-text role="class"} to skip collection of tests during the pytest collection phase is deprecated. Usepytest.skip
{.interpreted-text role="func"} instead.Note: This deprecation only relates to using
unittest.SkipTest
{.interpreted-text role="class"} during test collection. You are probably not doing that. Ordinary usage ofunittest.SkipTest
{.interpreted-text role="class"} /unittest.TestCase.skipTest
{.interpreted-text role="meth"} /unittest.skip
{.interpreted-text role="func"} in unittest test cases is fully supported. -
#8315: Several behaviors of
Parser.addoption <pytest.Parser.addoption>
{.interpreted-text role="meth"} are now scheduled for removal in pytest 8 (deprecated since pytest 2.4.0):-
parser.addoption(..., help=".. %default ..")
- use%(default)s
instead. -
parser.addoption(..., type="int/string/float/complex")
- usetype=int
etc. instead.
-
-
#8447: Defining a custom pytest node type which is both an
pytest.Item <Item>
{.interpreted-text role="class"} and apytest.Collector <Collector>
{.interpreted-text role="class"} (e.g.pytest.File <File>
{.interpreted-text role="class"}) now issues a warning. It was never sanely supported and triggers hard to debug errors.See
the deprecation note <diamond-inheritance-deprecated>
{.interpreted-text role="ref"} for full details. -
#8592:
pytest_cmdline_preparse
{.interpreted-text role="hook"} has been officially deprecated. It will be removed in a future release. Usepytest_load_initial_conftests
{.interpreted-text role="hook"} instead.See
the deprecation note <cmdline-preparse-deprecated>
{.interpreted-text role="ref"} for full details. -
#8645:
pytest.warns(None) <pytest.warns>
{.interpreted-text role="func"} is now deprecated because many people used it to mean "this code does not emit warnings", but it actually had the effect of checking that the code emits at least one warning of any type - likepytest.warns()
orpytest.warns(Warning)
. -
#8948:
pytest.skip(msg=...) <pytest.skip>
{.interpreted-text role="func"},pytest.fail(msg=...) <pytest.fail>
{.interpreted-text role="func"} andpytest.exit(msg=...) <pytest.exit>
{.interpreted-text role="func"} signatures now accept areason
argument instead ofmsg
. Usingmsg
still works, but is deprecated and will be removed in a future release.This was changed for consistency with
pytest.mark.skip <pytest.mark.skip>
{.interpreted-text role="func"} andpytest.mark.xfail <pytest.mark.xfail>
{.interpreted-text role="func"} which both acceptreason
as an argument. -
#8174: The following changes have been made to types reachable through
pytest.ExceptionInfo.traceback
{.interpreted-text role="attr"}:- The
path
property of_pytest.code.Code
returnsPath
instead ofpy.path.local
. - The
path
property of_pytest.code.TracebackEntry
returnsPath
instead ofpy.path.local
.
There was no deprecation period for this change (sorry!).
- The
Features
-
#5196: Tests are now ordered by definition order in more cases.
In a class hierarchy, tests from base classes are now consistently ordered before tests defined on their subclasses (reverse MRO order).
-
#7132: Added two environment variables
PYTEST_THEME
{.interpreted-text role="envvar"} andPYTEST_THEME_MODE
{.interpreted-text role="envvar"} to let the users customize the pygments theme used. -
#7259: Added
cache.mkdir() <pytest.Cache.mkdir>
{.interpreted-text role="meth"}, which is similar to the existingcache.makedir() <pytest.Cache.makedir>
{.interpreted-text role="meth"}, but returns apathlib.Path
{.interpreted-text role="class"} instead of a legacypy.path.local
.Added a
paths
type toparser.addini() <pytest.Parser.addini>
{.interpreted-text role="meth"}, as inparser.addini("mypaths", "my paths", type="paths")
, which is similar to the existingpathlist
, but returns a list ofpathlib.Path
{.interpreted-text role="class"} instead of legacypy.path.local
. -
#7469: The types of objects used in pytest's API are now exported so they may be used in type annotations.
The newly-exported types are:
-
pytest.Config
forConfig <pytest.Config>
{.interpreted-text role="class"}. -
pytest.Mark
formarks <pytest.Mark>
{.interpreted-text role="class"}. -
pytest.MarkDecorator
formark decorators <pytest.MarkDecorator>
{.interpreted-text role="class"}. -
pytest.MarkGenerator
for thepytest.mark <pytest.MarkGenerator>
{.interpreted-text role="class"} singleton. -
pytest.Metafunc
for themetafunc <pytest.MarkGenerator>
{.interpreted-text role="class"} argument to thepytest_generate_tests
{.interpreted-text role="hook"} hook. -
pytest.CallInfo
for theCallInfo <pytest.CallInfo>
{.interpreted-text role="class"} type passed to various hooks. -
pytest.PytestPluginManager
forPytestPluginManager <pytest.PytestPluginManager>
{.interpreted-text role="class"}. -
pytest.ExceptionInfo
for theExceptionInfo <pytest.ExceptionInfo>
{.interpreted-text role="class"} type returned frompytest.raises
{.interpreted-text role="func"} and passed to various hooks. -
pytest.Parser
for theParser <pytest.Parser>
{.interpreted-text role="class"} type passed to thepytest_addoption
{.interpreted-text role="hook"} hook. -
pytest.OptionGroup
for theOptionGroup <pytest.OptionGroup>
{.interpreted-text role="class"} type returned from theparser.addgroup <pytest.Parser.getgroup>
{.interpreted-text role="func"} method. -
pytest.HookRecorder
for theHookRecorder <pytest.HookRecorder>
{.interpreted-text role="class"} type returned from~pytest.Pytester
{.interpreted-text role="class"}. -
pytest.RecordedHookCall
for theRecordedHookCall <pytest.HookRecorder>
{.interpreted-text role="class"} type returned from~pytest.HookRecorder
{.interpreted-text role="class"}. -
pytest.RunResult
for theRunResult <pytest.RunResult>
{.interpreted-text role="class"} type returned from~pytest.Pytester
{.interpreted-text role="class"}. -
pytest.LineMatcher
for theLineMatcher <pytest.RunResult>
{.interpreted-text role="class"} type used in~pytest.RunResult
{.interpreted-text role="class"} and others. -
pytest.TestReport
for theTestReport <pytest.TestReport>
{.interpreted-text role="class"} type used in various hooks. -
pytest.CollectReport
for theCollectReport <pytest.CollectReport>
{.interpreted-text role="class"} type used in various hooks.
Constructing most of them directly is not supported; they are only meant for use in type annotations. Doing so will emit a deprecation warning, and may become a hard-error in pytest 8.0.
Subclassing them is also not supported. This is not currently enforced at runtime, but is detected by type-checkers such as mypy.
-
-
#7856:
--import-mode=importlib <import-modes>
{.interpreted-text role="ref"} now works with features that depend on modules being on :pysys.modules
{.interpreted-text role="data"}, such aspickle
{.interpreted-text role="mod"} anddataclasses
{.interpreted-text role="mod"}. -
#8144: The following hooks now receive an additional
pathlib.Path
argument, equivalent to an existingpy.path.local
argument:-
pytest_ignore_collect
{.interpreted-text role="hook"} - Thecollection_path
parameter (equivalent to existingpath
parameter). -
pytest_collect_file
{.interpreted-text role="hook"} - Thefile_path
parameter (equivalent to existingpath
parameter). -
pytest_pycollect_makemodule
{.interpreted-text role="hook"} - Themodule_path
parameter (equivalent to existingpath
parameter). -
pytest_report_header
{.interpreted-text role="hook"} - Thestart_path
parameter (equivalent to existingstartdir
parameter). -
pytest_report_collectionfinish
{.interpreted-text role="hook"} - Thestart_path
parameter (equivalent to existingstartdir
parameter).
::: {.note} ::: {.admonition-title} Note :::
The name of the
~_pytest.nodes.Node
{.interpreted-text role="class"} arguments and attributes (the new attribute beingpath
) is the opposite of the situation for hooks (the old argument beingpath
).This is an unfortunate artifact due to historical reasons, which should be resolved in future versions as we slowly get rid of the
py
{.interpreted-text role="pypi"} dependency (see9283
{.interpreted-text role="issue"} for a longer discussion). ::: -
-
#8251: Implement
Node.path
as apathlib.Path
. Both the oldfspath
and this new attribute gets set no matter whetherpath
orfspath
(deprecated) is passed to the constructor. It is a replacement for thefspath
attribute (which represents the same path aspy.path.local
). Whilefspath
is not deprecated yet due to the ongoing migration of methods like~_pytest.Item.reportinfo
{.interpreted-text role="meth"}, we expect to deprecate it in a future release.::: {.note} ::: {.admonition-title} Note :::
The name of the
~_pytest.nodes.Node
{.interpreted-text role="class"} arguments and attributes (the new attribute beingpath
) is the opposite of the situation for hooks (the old argument beingpath
).This is an unfortunate artifact due to historical reasons, which should be resolved in future versions as we slowly get rid of the
py
{.interpreted-text role="pypi"} dependency (see9283
{.interpreted-text role="issue"} for a longer discussion). ::: -
#8421:
pytest.approx
{.interpreted-text role="func"} now works on~decimal.Decimal
{.interpreted-text role="class"} within mappings/dicts and sequences/lists. -
#8606: pytest invocations with
--fixtures-per-test
and--fixtures
have been enriched with:- Fixture location path printed with the fixture name.
- First section of the fixture's docstring printed under the fixture name.
- Whole of fixture's docstring printed under the fixture name using
--verbose
option.
-
#8761: New
version-tuple
{.interpreted-text role="ref"} attribute, which makes it simpler for users to do something depending on the pytest version (such as declaring hooks which are introduced in later versions). -
#8789: Switch TOML parser from
toml
totomli
for TOML v1.0.0 support inpyproject.toml
. -
#8920: Added
pytest.Stash
{.interpreted-text role="class"}, a facility for plugins to store their data on~pytest.Config
{.interpreted-text role="class"} and~_pytest.nodes.Node
{.interpreted-text role="class"}s in a type-safe and conflict-free manner. Seeplugin-stash
{.interpreted-text role="ref"} for details. -
#8953:
RunResult <_pytest.pytester.RunResult>
{.interpreted-text role="class"} methodassert_outcomes <_pytest.pytester.RunResult.assert_outcomes>
{.interpreted-text role="meth"} now accepts awarnings
argument to assert the total number of warnings captured. -
#8954:
--debug
flag now accepts astr
{.interpreted-text role="class"} file to route debug logs into, remains defaulted to [pytestdebug.log]{.title-ref}. -
#9023: Full diffs are now always shown for equality assertions of iterables when [CI]{.title-ref} or
BUILD_NUMBER
is found in the environment, even when-v
isn't used. -
#9113:
RunResult <_pytest.pytester.RunResult>
{.interpreted-text role="class"} methodassert_outcomes <_pytest.pytester.RunResult.assert_outcomes>
{.interpreted-text role="meth"} now accepts adeselected
argument to assert the total number of deselected tests. -
#9114: Added
pythonpath
{.interpreted-text role="confval"} setting that adds listed paths tosys.path
{.interpreted-text role="data"} for the duration of the test session. If you currently use the pytest-pythonpath or pytest-srcpaths plugins, you should be able to replace them with built-in [pythonpath]{.title-ref} setting.
Improvements
-
#7480: A deprecation scheduled to be removed in a major version X (e.g. pytest 7, 8, 9, ...) now uses warning category [PytestRemovedInXWarning]{.title-ref}, a subclass of
~pytest.PytestDeprecationWarning
{.interpreted-text role="class"}, instead ofPytestDeprecationWarning
{.interpreted-text role="class"} directly.See
backwards-compatibility
{.interpreted-text role="ref"} for more details. -
#7864: Improved error messages when parsing warning filters.
Previously pytest would show an internal traceback, which besides being ugly sometimes would hide the cause of the problem (for example an
ImportError
while importing a specific warning type). -
#8335: Improved
pytest.approx
{.interpreted-text role="func"} assertion messages for sequences of numbers.The assertion messages now dumps a table with the index and the error of each diff. Example:
> assert [1, 2, 3, 4] == pytest.approx([1, 3, 3, 5]) E assert comparison failed for 2 values: E Index | Obtained | Expected E 1 | 2 | 3 +- 3.0e-06 E 3 | 4 | 5 +- 5.0e-06
-
#8403: By default, pytest will truncate long strings in assert errors so they don't clutter the output too much, currently at
240
characters by default.However, in some cases the longer output helps, or is even crucial, to diagnose a failure. Using
-v
will now increase the truncation threshold to2400
characters, and-vv
or higher will disable truncation entirely. -
#8509: Fixed issue where
unittest.TestCase.setUpClass
{.interpreted-text role="meth"} is not called when a test has [/]{.title-ref} in its name since pytest 6.2.0.This refers to the path part in pytest node IDs, e.g.
TestClass::test_it
in the node IDtests/test_file.py::TestClass::test_it
.Now, instead of assuming that the test name does not contain
/
, it is assumed that test path does not contain::
. We plan to hopefully make both of these work in the future. -
#8803: It is now possible to add colors to custom log levels on cli log.
By using
add_color_level <_pytest.logging.add_color_level>
{.interpreted-text role="func"} from apytest_configure
hook, colors can be added:logging_plugin = config.pluginmanager.get_plugin('logging-plugin') logging_plugin.log_cli_handler.formatter.add_color_level(logging.INFO, 'cyan') logging_plugin.log_cli_handler.formatter.add_color_level(logging.SPAM, 'blue')
See
log_colors
{.interpreted-text role="ref"} for more information. -
#8822: When showing fixture paths in [--fixtures]{.title-ref} or [--fixtures-by-test]{.title-ref}, fixtures coming from pytest itself now display an elided path, rather than the full path to the file in the [site-packages]{.title-ref} directory.
-
#8898: Complex numbers are now treated like floats and integers when generating parameterization IDs.
-
#9062:
--stepwise-skip
now implicitly enables--stepwise
and can be used on its own. -
#9205:
pytest.Cache.set
{.interpreted-text role="meth"} now preserves key order when saving dicts.
Bug Fixes
-
#7124: Fixed an issue where
__main__.py
would raise anImportError
when--doctest-modules
was provided. -
#8061: Fixed failing
staticmethod
test cases if they are inherited from a parent test class. -
#8192:
testdir.makefile
now silently accepts values which don't start with.
to maintain backward compatibility with older pytest versions.pytester.makefile
now issues a clearer error if the.
is missing in theext
argument. -
#8258: Fixed issue where pytest's
faulthandler
support would not dump traceback on crashes if thefaulthandler
{.interpreted-text role="mod"} module was already enabled during pytest startup (usingpython -X dev -m pytest
for example). -
#8317: Fixed an issue where illegal directory characters derived from
getpass.getuser()
raised anOSError
. -
#8367: Fix
Class.from_parent
so it forwards extra keyword arguments to the constructor. -
#8377: The test selection options
pytest -k
andpytest -m
now support matching names containing forward slash (/
) characters. -
#8384: The
@pytest.mark.skip
decorator now correctly handles its arguments. When thereason
argument is accidentally given both positional and as a keyword (e.g. because it was confused withskipif
), aTypeError
now occurs. Before, such tests were silently skipped, and the positional argument ignored. Additionally,reason
is now documented correctly as positional or keyword (rather than keyword-only). -
#8394: Use private names for internal fixtures that handle classic setup/teardown so that they don't show up with the default
--fixtures
invocation (but they still show up with--fixtures -v
). -
#8456: The
required_plugins
{.interpreted-text role="confval"} config option now works correctly when pre-releases of plugins are installed, rather than falsely claiming that those plugins aren't installed at all. -
#8464:
-c <config file>
now also properly definesrootdir
as the directory that contains<config file>
. -
#8503:
pytest.MonkeyPatch.syspath_prepend
{.interpreted-text role="meth"} no longer fails whensetuptools
is not installed. It now only callspkg_resources.fixup_namespace_packages
{.interpreted-text role="func"} ifpkg_resources
was previously imported, because it is not needed otherwise. -
#8548: Introduce fix to handle precision width in
log-cli-format
in turn to fix output coloring for certain formats. -
#8796: Fixed internal error when skipping doctests.
-
#8983: The test selection options
pytest -k
andpytest -m
now support matching names containing backslash ([\]{.title-ref}) characters. Backslashes are treated literally, not as escape characters (the values being matched against are already escaped). -
#8990: Fix [pytest -vv]{.title-ref} crashing with an internal exception [AttributeError: 'str' object has no attribute 'relative_to']{.title-ref} in some cases.
-
#9077: Fixed confusing error message when
request.fspath
/request.path
was accessed from a session-scoped fixture. -
#9163: The end line number and end column offset are now properly set for rewritten assert statements.
-
#9169: Support for the
files
API fromimportlib.resources
within rewritten files. -
#9272: The nose compatibility module-level fixtures [setup()]{.title-ref} and [teardown()]{.title-ref} are now only called once per module, instead of for each test function. They are now called even if object-level [setup]{.title-ref}/[teardown]{.title-ref} is defined.
Improved Documentation
- #4320: Improved docs for [pytester.copy_example]{.title-ref}.
-
#5105: Add automatically generated
plugin-list
{.interpreted-text role="ref"}. The list is updated on a periodic schedule. -
#8337: Recommend numpy.testing module on
pytest.approx
{.interpreted-text role="func"} documentation. -
#8655: Help text for
--pdbcls
more accurately reflects the option's behavior. -
#9210: Remove incorrect docs about
confcutdir
being a configuration option: it can only be set through the--confcutdir
command-line option. - #9242: Upgrade readthedocs configuration to use a newer Ubuntu version` with better unicode support for PDF docs.
-
#9341: Various methods commonly used for
non-python tests
{.interpreted-text role="ref"} are now correctly documented in the reference docs. They were undocumented previously.
Trivial/Internal Changes
-
#8133: Migrate to
setuptools_scm
6.x to useSETUPTOOLS_SCM_MRETEND_VERSION_FOR_PYTEST
for more robust release tooling. -
#8174: The following changes have been made to internal pytest types/functions:
- The
_pytest.code.getfslineno()
function returnsPath
instead ofpy.path.local
. - The
_pytest.python.path_matches_patterns()
function takesPath
instead ofpy.path.local
. - The
_pytest._code.Traceback.cut()
function accepts anyos.PathLike[str]
, not justpy.path.local
.
- The
-
#8248: Internal Restructure: let
python.PyObjMixin
inherit fromnodes.Node
to carry over typing information. -
#8432: Improve error message when
pytest.skip
{.interpreted-text role="func"} is used at module level without passing [allow_module_level=True]{.title-ref}. -
#8818: Ensure
regendoc
opts out ofTOX_ENV
cachedir selection to ensure independent example test runs. -
#8913: The private
CallSpec2._arg2scopenum
attribute has been removed after an internal refactoring. -
#8967:
pytest_assertion_pass
{.interpreted-text role="hook"} is no longer considered experimental and future changes to it will be considered more carefully. - #9202: Add github action to upload coverage report to codecov instead of bash uploader.
- #9225: Changed the command used to create sdist and wheel artifacts: using the build package instead of setup.py.
- #9351: Correct minor typos in doc/en/example/special.rst.
v6.2.5
pytest 6.2.5 (2021-08-29)
Trivial/Internal Changes
v6.2.4
pytest 6.2.4 (2021-05-04)
Bug Fixes
- #8539: Fixed assertion rewriting on Python 3.10.
v6.2.3
pytest 6.2.3 (2021-04-03)
Bug Fixes
-
#8414: pytest used to create directories under
/tmp
with world-readable permissions. This means that any user in the system was able to read information written by tests in temporary directories (such as those created by thetmp_path
/tmpdir
fixture). Now the directories are created with private permissions.pytest used silenty use a pre-existing
/tmp/pytest-of-<username>
directory, even if owned by another user. This means another user could pre-create such a directory and gain control of another user's temporary directory. Now such a condition results in an error.
v6.2.2
pytest 6.2.2 (2021-01-25)
Bug Fixes
- #8152: Fixed "(<Skipped instance>)" being shown as a skip reason in the verbose test summary line when the reason is empty.
-
#8249: Fix the
faulthandler
plugin for occasions when running withtwisted.logger
and usingpytest --capture=no
.
v6.2.1
pytest 6.2.1 (2020-12-15)
Bug Fixes
-
#7678: Fixed bug where
ImportPathMismatchError
would be raised for files compiled in the host and loaded later from an UNC mounted path (Windows). -
#8132: Fixed regression in
approx
: in 6.2.0approx
no longer raisesTypeError
when dealing with non-numeric types, falling back to normal comparison. Before 6.2.0, array types like tf.DeviceArray fell through to the scalar case, and happened to compare correctly to a scalar if they had only one element. After 6.2.0, these types began failing, because they inherited neither from standard Python number hierarchy nor fromnumpy.ndarray
.approx
now converts arguments tonumpy.ndarray
if they expose the array protocol and are not scalars. This treats array-like objects like numpy arrays, regardless of size.
v6.2.0
pytest 6.2.0 (2020-12-12)
Breaking Changes
- #7808: pytest now supports python3.6+ only.
Deprecations
-
#7469: Directly constructing/calling the following classes/functions is now deprecated:
_pytest.cacheprovider.Cache
_pytest.cacheprovider.Cache.for_config()
_pytest.cacheprovider.Cache.clear_cache()
_pytest.cacheprovider.Cache.cache_dir_from_config()
_pytest.capture.CaptureFixture
_pytest.fixtures.FixtureRequest
_pytest.fixtures.SubRequest
_pytest.logging.LogCaptureFixture
_pytest.pytester.Pytester
_pytest.pytester.Testdir
_pytest.recwarn.WarningsRecorder
_pytest.recwarn.WarningsChecker
_pytest.tmpdir.TempPathFactory
_pytest.tmpdir.TempdirFactory
These have always been considered private, but now issue a deprecation warning, which may become a hard error in pytest 7.0.0.
-
#7530: The
--strict
command-line option has been deprecated, use--strict-markers
instead.We have plans to maybe in the future to reintroduce
--strict
and make it an encompassing flag for all strictness related options (--strict-markers
and--strict-config
at the moment, more might be introduced in the future). -
#7988: The
@pytest.yield_fixture
decorator/function is now deprecated. Use pytest.fixture instead.yield_fixture
has been an alias forfixture
for a very long time, so can be search/replaced safely.
Features
-
#5299: pytest now warns about unraisable exceptions and unhandled thread exceptions that occur in tests on Python>=3.8. See unraisable for more information.
-
#7425: New pytester fixture, which is identical to testdir but its methods return pathlib.Path when appropriate instead of
py.path.local
.This is part of the movement to use pathlib.Path objects internally, in order to remove the dependency to
py
in the future.Internally, the old Testdir <_pytest.pytester.Testdir> is now a thin wrapper around Pytester <_pytest.pytester.Pytester>, preserving the old interface.
-
#7695: A new hook was added, pytest_markeval_namespace which should return a dictionary. This dictionary will be used to augment the "global" variables available to evaluate skipif/xfail/xpass markers.
Pseudo example
conftest.py
:def pytest_markeval_namespace(): return {"color": "red"}
test_func.py
:@​pytest.mark.skipif("color == 'blue'", reason="Color is not red") def test_func(): assert False
-
#8006: It is now possible to construct a ~pytest.MonkeyPatch object directly as
pytest.MonkeyPatch()
, in cases when the monkeypatch fixture cannot be used. Previously some users imported it from the private _pytest.monkeypatch.MonkeyPatch namespace.Additionally, MonkeyPatch.context <pytest.MonkeyPatch.context> is now a classmethod, and can be used as
with MonkeyPatch.context() as mp: ...
. This is the recommended way to useMonkeyPatch
directly, since unlike themonkeypatch
fixture, an instance created directly is notundo()
-ed automatically.
Improvements
-
#1265: Added an
__str__
implementation to the ~pytest.pytester.LineMatcher class which is returned frompytester.run_pytest().stdout
and similar. It returns the entire output, like the existingstr()
method. -
#2044: Verbose mode now shows the reason that a test was skipped in the test's terminal line after the "SKIPPED", "XFAIL" or "XPASS".
-
#7469 The types of builtin pytest fixtures are now exported so they may be used in type annotations of test functions. The newly-exported types are:
-
pytest.FixtureRequest
for the request fixture. -
pytest.Cache
for the cache fixture. -
pytest.CaptureFixture[str]
for the capfd and capsys fixtures. -
pytest.CaptureFixture[bytes]
for the capfdbinary and capsysbinary fixtures. -
pytest.LogCaptureFixture
for the caplog fixture. -
pytest.Pytester
for the pytester fixture. -
pytest.Testdir
for the testdir fixture. -
pytest.TempdirFactory
for the tmpdir_factory fixture. -
pytest.TempPathFactory
for the tmp_path_factory fixture. -
pytest.MonkeyPatch
for the monkeypatch fixture. -
pytest.WarningsRecorder
for the recwarn fixture.
Constructing them is not supported (except for MonkeyPatch); they are only meant for use in type annotations. Doing so will emit a deprecation warning, and may become a hard-error in pytest 7.0.
Subclassing them is also not supported. This is not currently enforced at runtime, but is detected by type-checkers such as mypy.
-
-
#7527: When a comparison between namedtuple <collections.namedtuple> instances of the same type fails, pytest now shows the differing field names (possibly nested) instead of their indexes.
-
#7615: Node.warn <_pytest.nodes.Node.warn> now permits any subclass of Warning, not just PytestWarning <pytest.PytestWarning>.
-
#7701: Improved reporting when using
--collected-only
. It will now show the number of collected tests in the summary stats. -
#7710: Use strict equality comparison for non-numeric types in pytest.approx instead of raising TypeError.
This was the undocumented behavior before 3.7, but is now officially a supported feature.
-
#7938: New
--sw-skip
argument which is a shorthand for--stepwise-skip
. -
#8023: Added
'node_modules'
to default value for norecursedirs. -
#8032: doClassCleanups <unittest.TestCase.doClassCleanups> (introduced in unittest in Python and 3.8) is now called appropriately.
Bug Fixes
- #4824: Fixed quadratic behavior and improved performance of collection of items using autouse fixtures and xunit fixtures.
-
#7758: Fixed an issue where some files in packages are getting lost from
--lf
even though they contain tests that failed. Regressed in pytest 5.4.0. - #7911: Directories created by by tmp_path and tmpdir are now considered stale after 3 days without modification (previous value was 3 hours) to avoid deleting directories still in use in long running test suites.
- #7913: Fixed a crash or hang in pytester.spawn <_pytest.pytester.Pytester.spawn> when the readline module is involved.
- #7951: Fixed handling of recursive symlinks when collecting tests.
- #7981: Fixed symlinked directories not being followed during collection. Regressed in pytest 6.1.0.
-
#8016: Fixed only one doctest being collected when using
pytest --doctest-modules path/to/an/__init__.py
.
Improved Documentation
- #7429: Add more information and use cases about skipping doctests.
-
#7780: Classes which should not be inherited from are now marked
final class
in the API reference. -
#7872:
_pytest.config.argparsing.Parser.addini()
accepts explicitNone
and"string"
. - #7878: In pull request section, ask to commit after editing changelog and authors file.
Trivial/Internal Changes
-
#7802: The
attrs
dependency requirement is now >=19.2.0 instead of >=17.4.0. - #8014: .pyc files created by pytest's assertion rewriting now conform to the newer PEP-552 format on Python>=3.7. (These files are internal and only interpreted by pytest itself.)
v6.1.2
pytest 6.1.2 (2020-10-28)
Bug Fixes
-
#7758: Fixed an issue where some files in packages are getting lost from
--lf
even though they contain tests that failed. Regressed in pytest 5.4.0. - #7911: Directories created by tmpdir are now considered stale after 3 days without modification (previous value was 3 hours) to avoid deleting directories still in use in long running test suites.
Improved Documentation
-
#7815: Improve deprecation warning message for
pytest._fillfuncargs()
.
v6.1.1
pytest 6.1.1 (2020-10-03)
Bug Fixes
- #7807: Fixed regression in pytest 6.1.0 causing incorrect rootdir to be determined in some non-trivial cases where parent directories have config files as well.
- #7814: Fixed crash in header reporting when testpaths is used and contains absolute paths (regression in 6.1.0).
v6.1.0
pytest 6.1.0 (2020-09-26)
Breaking Changes
-
#5585: As per our policy, the following features which have been deprecated in the 5.X series are now removed:
- The
funcargnames
read-only property ofFixtureRequest
,Metafunc
, andFunction
classes. Usefixturenames
attribute. -
@pytest.fixture
no longer supports positional arguments, pass all arguments by keyword instead. - Direct construction of
Node
subclasses now raise an error, usefrom_parent
instead. - The default value for
junit_family
has changed toxunit2
. If you require the old format, addjunit_family=xunit1
to your configuration file. - The
TerminalReporter
no longer has awriter
attribute. Plugin authors may use the public functions of theTerminalReporter
instead of accessing theTerminalWriter
object directly. - The
--result-log
option has been removed. Users are recommended to use the pytest-reportlog plugin instead.
For more information consult Deprecations and Removals in the docs.
- The
Deprecations
-
#6981: The
pytest.collect
module is deprecated: all its names can be imported frompytest
directly. -
#7097: The
pytest._fillfuncargs
function is deprecated. This function was kept for backward compatibility with an older plugin.It's functionality is not meant to be used directly, but if you must replace it, use function._request._fillfixtures() instead, though note this is not a public API and may break in the future.
-
#7210: The special
-k '-expr'
syntax to-k
is deprecated. Use-k 'not expr'
instead.The special
-k 'expr:'
syntax to-k
is deprecated. Please open an issue if you use this and want a replacement. -
#7255: The pytest_warning_captured <_pytest.hookspec.pytest_warning_captured> hook is deprecated in favor of pytest_warning_recorded <_pytest.hookspec.pytest_warning_recorded>, and will be removed in a future version.
-
#7648: The
gethookproxy()
andisinitpath()
methods ofFSCollector
andPackage
are deprecated; useself.session.gethookproxy()
andself.session.isinitpath()
instead. This should work on all pytest versions.
Features
-
#7667: New
--durations-min
command-line flag controls the minimal duration for inclusion in the slowest list of tests shown by--durations
. Previously this was hard-coded to0.005s
.
Improvements
-
#6681: Internal pytest warnings issued during the early stages of initialization are now properly handled and can filtered through filterwarnings or
--pythonwarnings/-W
.This also fixes a number of long standing issues: #2891, #7620, #7426.
-
#7572: When a plugin listed in
required_plugins
is missing or an unknown config key is used with--strict-config
, a simple error message is now shown instead of a stacktrace. -
#7685: Added two new attributes rootpath <_pytest.config.Config.rootpath> and inipath <_pytest.config.Config.inipath> to Config <_pytest.config.Config>. These attributes are pathlib.Path versions of the existing rootdir <_pytest.config.Config.rootdir> and inifile <_pytest.config.Config.inifile> attributes, and should be preferred over them when possible.
-
#7780: Public classes which are not designed to be inherited from are now marked @final. Code which inherits from these classes will trigger a type-checking (e.g. mypy) error, but will still work in runtime. Currently the
final
designation does not appear in the API Reference but hopefully will in the future.
Bug Fixes
-
#1953: Fixed error when overwriting a parametrized fixture, while also reusing the super fixture value.
conftest.py
import pytest
@​pytest.fixture(params=[1, 2])
def foo(request):
return request.param
test_foo.py
import pytest
@​pytest.fixture
def foo(foo):
return foo * 2
```
-
#4984: Fixed an internal error crash with
IndexError: list index out of range
when collecting a module which starts with a decorated function, the decorator raises, and assertion rewriting is enabled. - #7591: pylint shouldn't complain anymore about unimplemented abstract methods when inheriting from File <non-python tests>.
-
#7628: Fixed test collection when a full path without a drive letter was passed to pytest on Windows (for example
\projects\tests\test.py
instead ofc:\projects\tests\pytest.py
). -
#7638: Fix handling of command-line options that appear as paths but trigger an OS-level syntax error on Windows, such as the options used internally by
pytest-xdist
. -
#7742: Fixed INTERNALERROR when accessing locals / globals with faulty
exec
.
Improved Documentation
- #1477: Removed faq.rst and its reference in contents.rst.
Trivial/Internal Changes
-
#7536: The internal
junitxml
plugin has rewritten to usexml.etree.ElementTree
. The order of attributes in XML elements might differ. Some unneeded escaping is no longer performed. -
#7587: The dependency on the
more-itertools
package has been removed. - #7631: The result type of capfd.readouterr() <_pytest.capture.CaptureFixture.readouterr> (and similar) is no longer a namedtuple, but should behave like one in all respects. This was done for technical reasons.
-
#7671: When collecting tests, pytest finds test classes and functions by examining the
attributes of python objects (modules, classes and instances). To speed up this
process, pytest now ignores builtin attributes (like
__class__
,__delattr__
and__new__
) without consulting the python_classes and python_functions configuration options and without passing them to plugins using the pytest_pycollect_makeitem <_pytest.hookspec.pytest_pycollect_makeitem> hook.
v6.0.2
pytest 6.0.2 (2020-09-04)
Bug Fixes
-
#7148: Fixed
--log-cli
potentially causing unrelatedprint
output to be swallowed. -
#7672: Fixed log-capturing level restored incorrectly if
caplog.set_level
is called more than once. - #7686: Fixed NotSetType.token being used as the parameter ID when the parametrization list is empty. Regressed in pytest 6.0.0.
-
#7707: Fix internal error when handling some exceptions that contain multiple lines or the style uses multiple lines (
--tb=line
for example).
v6.0.1
pytest 6.0.1 (2020-07-30)
Bug Fixes
-
#7394: Passing an empty
help
value toParser.add_option
is now accepted instead of crashing when runningpytest --help
. PassingNone
raises a more informativeTypeError
. -
#7558: Fix pylint
not-callable
lint onpytest.mark.parametrize()
and the other builtin marks:skip
,skipif
,xfail
,usefixtures
,filterwarnings
. -
#7559: Fix regression in plugins using
TestReport.longreprtext
(such aspytest-html
) whenTestReport.longrepr
is not a string. -
#7569: Fix logging capture handler's level not reset on teardown after a call to
caplog.set_level()
.
v6.0.0
pytest 6.0.0 (2020-07-28)
(Please see the full set of changes for this release also in the 6.0.0rc1 notes below)
Breaking Changes
-
#5584: PytestDeprecationWarning are now errors by default.
Following our plan to remove deprecated features with as little disruption as possible, all warnings of type
PytestDeprecationWarning
now generate errors instead of warning messages.The affected features will be effectively removed in pytest 6.1, so please consult the Deprecations and Removals section in the docs for directions on how to update existing code.
In the pytest
6.0.X
series, it is possible to change the errors back into warnings as a stopgap measure by adding this to yourpytest.ini
file:[pytest] filterwarnings = ignore::pytest.PytestDeprecationWarning
But this will stop working when pytest
6.1
is released.If you have concerns about the removal of a specific feature, please add a comment to #5584.
-
#7472: The
exec_()
andis_true()
methods of_pytest._code.Frame
have been removed.
Features
- #7464: Added support for NO_COLOR and FORCE_COLOR environment variables to control colored output.
Improvements
-
#7467:
--log-file
CLI option andlog_file
ini marker now create subdirectories if needed. -
#7489: The pytest.raises function has a clearer error message when
match
equals the obtained string but is not a regex match. In this case it is suggested to escape the regex.
Bug Fixes
-
#7392: Fix the reported location of tests skipped with
@pytest.mark.skip
when--runxfail
is used. - #7491: tmpdir and tmp_path no longer raise an error if the lock to check for stale temporary directories is not accessible.
-
#7517: Preserve line endings when captured via
capfd
. -
#7534: Restored the previous formatting of
TracebackEntry.__str__
which was changed by accident.
Improved Documentation
-
#7422: Clarified when the
usefixtures
mark can apply fixtures to test. -
#7441: Add a note about
-q
option used in getting started guide.
Trivial/Internal Changes
-
#7389: Fixture scope
package
is no longer considered experimental.
pytest 6.0.0rc1 (2020-07-08)
Breaking Changes
-
#1316:
TestReport.longrepr
is now always an instance ofReprExceptionInfo
. Previously it was astr
when a test failed withpytest.fail(..., pytrace=False)
. -
#5965: symlinks are no longer resolved during collection and matching conftest.py files with test file paths.
Resolving symlinks for the current directory and during collection was introduced as a bugfix in 3.9.0, but it actually is a new feature which had unfortunate consequences in Windows and surprising results in other platforms.
The team decided to step back on resolving symlinks at all, planning to review this in the future with a more solid solution (see discussion in #6523 for details).
This might break test suites which made use of this feature; the fix is to create a symlink for the entire test tree, and not only to partial files/tress as it was possible previously.
-
#6505:
Testdir.run().parseoutcomes()
now always returns the parsed nouns in plural form.Originally
parseoutcomes()
would always returns the nouns in plural form, but a change meant to improve the terminal summary by using singular form single items (1 warning
or1 error
) caused an unintended regression by changing the keys returned byparseoutcomes()
.Now the API guarantees to always return the plural form, so calls like this:
result = testdir.runpytest() result.assert_outcomes(error=1)
Need to be changed to:
result = testdir.runpytest() result.assert_outcomes(errors=1)
-
#6903: The
os.dup()
function is now assumed to exist. We are not aware of any supported Python 3 implementations which do not provide it. -
#7040:
-k
no longer matches against the names of the directories outside the test session root.Also,
pytest.Package.name
is now just the name of the directory containing the package's__init__.py
file, instead of the full path. This is consistent with how the other nodes are named, and also one of the reasons why-k
would match against any directory containing the test suite. -
#7122: Expressions given to the
-m
and-k
options are no longer evaluated using Python's eval. The format supportsor
,and
,not
, parenthesis and general identifiers to match against. Python constants, keywords or other operators are no longer evaluated differently. -
#7135: Pytest now uses its own
TerminalWriter
class instead of using the one from thepy
library. Plugins generally access this class throughTerminalReporter.writer
,TerminalReporter.write()
(and similar methods), or_pytest.config.create_terminal_writer()
.The following breaking changes were made:
- Output (
write()
method and others) no longer flush implicitly; the flushing behavior of the underlying file is respected. To flush explicitly (for example, if you want output to be shown before an end-of-line is printed), usewrite(flush=True)
orterminal_writer.flush()
. - Explicit Windows console support was removed, delegated to the colorama library.
- Support for writing
bytes
was removed. - The
reline
method andchars_on_current_line
property were removed. - The
stringio
andencoding
arguments was removed. - Support for passing a callable instead of a file was removed.
- Output (
-
#7224: The item.catch_log_handler and item.catch_log_handlers attributes, set by the logging plugin and never meant to be public , are no longer available.
The deprecated
--no-print-logs
option is removed. Use--show-capture
instead. -
#7226: Removed the unused
args
parameter frompytest.Function.__init__
. -
#7418: Removed the pytest_doctest_prepare_content hook specification. This hook hasn't been triggered by pytest for at least 10 years.
-
#7438: Some changes were made to the internal
_pytest._code.source
, listed here for the benefit of plugin authors who may be using it:- The
deindent
argument toSource()
has been removed, now it is always true. - Support for zero or multiple arguments to
Source()
has been removed. - Support for comparing
Source
with anstr
has been removed. - The methods
Source.isparseable()
andSource.putaround()
have been removed. - The method
Source.compile()
and function_pytest._code.compile()
have been removed; use plaincompile()
instead. - The function
_pytest._code.source.getsource()
has been removed; useSource()
directly instead.
- The
Deprecations
-
#7210: The special
-k '-expr'
syntax to-k
is deprecated. Use-k 'not expr'
instead.The special
-k 'expr:'
syntax to-k
is deprecated. Please open an issue if you use this and want a replacement. -
#4049:
pytest_warning_captured
is deprecated in favor of thepytest_warning_recorded
hook.
Features
-
#1556: pytest now supports
pyproject.toml
files for configuration.The configuration options is similar to the one available in other formats, but must be defined in a
[tool.pytest.ini_options]
table to be picked up by pytest:
pyproject.toml
[tool.pytest.ini_options]
minversion = "6.0"
addopts = "-ra -q"
testpaths = [
"tests",
"integration",
]
```
More information can be found [in the docs](https://docs.pytest.org/en/stable/customize.html#configuration-file-formats).
-
#3342: pytest now includes inline type annotations and exposes them to user programs. Most of the user-facing API is covered, as well as internal code.
If you are running a type checker such as mypy on your tests, you may start noticing type errors indicating incorrect usage. If you run into an error that you believe to be incorrect, please let us know in an issue.
The types were developed against mypy version 0.780. Versions before 0.750 are known not to work. We recommend using the latest version. Other type checkers may work as well, but they are not officially verified to work by pytest yet.
-
#4049: Introduced a new hook named pytest_warning_recorded to convey information about warnings captured by the internal pytest warnings plugin.
This hook is meant to replace pytest_warning_captured, which is deprecated and will be removed in a future release.
-
#6471: New command-line flags:
- `--no-header`: disables the initial header, including platform, version, and plugins.
- `--no-summary`: disables the final test summary, including warnings.
-
#6856: A warning is now shown when an unknown key is read from a config INI file.
The --strict-config flag has been added to treat these warnings as errors.
-
#6906: Added --code-highlight command line option to enable/disable code highlighting in terminal output.
-
#7245: New
--import-mode=importlib
option that uses importlib to import test modules.Traditionally pytest used
__import__
while changingsys.path
to import test modules (which also changessys.modules
as a side-effect), which works but has a number of drawbacks, like requiring test modules that don't live in packages to have unique names (as they need to reside under a unique name insys.modules
).--import-mode=importlib
uses more fine grained import mechanisms fromimportlib
which don't require pytest to changesys.path
orsys.modules
at all, eliminating much of the drawbacks of the previous mode.We intend to make
--import-mode=importlib
the default in future versions, so users are encouraged to try the new mode and provide feedback (both positive or negative) in issue #7245.You can read more about this option in the documentation.
-
#7305: New
required_plugins
configuration option allows the user to specify a list of plugins, including version information, that are required for pytest to run. An error is raised if any required plugins are not found when running pytest.
Improvements
-
#4375: The
pytest
command now suppresses theBrokenPipeError
error message that is printed to stderr when the output ofpytest
is piped and and the pipe is closed by the piped-to program (common examples areless
andhead
). -
#4391: Improved precision of test durations measurement.
CallInfo
items now have a new<CallInfo>.duration
attribute, created usingtime.perf_counter()
. This attribute is used to fill the<TestReport>.duration
attribute, which is more accurate than the previous<CallInfo>.stop - <CallInfo>.start
(as these are based ontime.time()
). -
#4675: Rich comparison for dataclasses and attrs-classes is now recursive.
-
#6285: Exposed the pytest.FixtureLookupError exception which is raised by request.getfixturevalue() (where request is a FixtureRequest fixture) when a fixture with the given name cannot be returned.
-
#6433: If an error is encountered while formatting the message in a logging call, for example
logging.warning("oh no!: %s: %s", "first")
(a second argument is missing), pytest now propagates the error, likely causing the test to fail.Previously, such a mistake would cause an error to be printed to stderr, which is not displayed by default for passing tests. This change makes the mistake visible during testing.
You may supress this behavior temporarily or permanently by setting
logging.raiseExceptions = False
. -
#6817: Explicit new-lines in help texts of command-line options are preserved, allowing plugins better control of the help displayed to users.
-
#6940: When using the
--duration
option, the terminal message output is now more precise about the number and duration of hidden items. -
#6991: Collected files are displayed after any reports from hooks, e.g. the status from
--lf
. -
#7091: When
fd
capturing is used, through--capture=fd
or thecapfd
andcapfdbinary
fixtures, and the file descriptor (0, 1, 2) cannot be duplicated, FD capturing is still performed. Previously, direct writes to the file descriptors would fail or be lost in this case. -
#7119: Exit with an error if the
--basetemp
argument is empty, is the current working directory or is one of the parent directories. This is done to protect against accidental data loss, as any directory passed to this argument is cleared. -
#7128: pytest --version now displays just the pytest version, while pytest --version --version displays more verbose information including plugins. This is more consistent with how other tools show --version.
-
#7133: caplog.set_level() <_pytest.logging.LogCaptureFixture.set_level> will now override any log_level set via the CLI or configuration file.
-
#7159: caplog.set_level() <_pytest.logging.LogCaptureFixture.set_level> and caplog.at_level() <_pytest.logging.LogCaptureFixture.at_level> no longer affect the level of logs that are shown in the Captured log report report section.
-
#7348: Improve recursive diff report for comparison asserts on dataclasses / attrs.
-
#7385:
--junitxml
now includes the exception cause in themessage
XML attribute for failures during setup and teardown.Previously:
<error message="test setup failure">
Now:
<error message="failed on setup with "ValueError: Some error during setup"">
Bug Fixes
-
#1120: Fix issue where directories from tmpdir are not removed properly when multiple instances of pytest are running in parallel.
-
#4583: Prevent crashing and provide a user-friendly error when a marker expression (-m) invoking of eval raises any exception.
-
#4677: The path shown in the summary report for SKIPPED tests is now always relative. Previously it was sometimes absolute.
-
#5456: Fix a possible race condition when trying to remove lock files used to control access to folders created by tmp_path and tmpdir.
-
#6240: Fixes an issue where logging during collection step caused duplication of log messages to stderr.
-
#6428: Paths appearing in error messages are now correct in case the current working directory has changed since the start of the session.
-
#6755: Support deleting paths longer than 260 characters on windows created inside tmpdir.
-
#6871: Fix crash with captured output when using capsysbinary.
-
#6909: Revert the change introduced by #6330, which required all arguments to
@pytest.mark.parametrize
to be explicitly defined in the function signature.The intention of the original change was to remove what was expected to be an unintended/surprising behavior, but it turns out many people relied on it, so the restriction has been reverted.
-
#6910: Fix crash when plugins return an unknown stats while using the
--reportlog
option. -
#6924: Ensure a
unittest.IsolatedAsyncioTestCase
is actually awaited. -
#6925: Fix TerminalRepr instances to be hashable again.
-
#6947: Fix regression where functions registered with unittest.TestCase.addCleanup were not being called on test failures.
-
#6951: Allow users to still set the deprecated
TerminalReporter.writer
attribute. -
#6956: Prevent pytest from printing ConftestImportFailure traceback to stdout.
-
#6991: Fix regressions with --lf filtering too much since pytest 5.4.
-
#6992: Revert "tmpdir: clean up indirection via config for factories" #6767 as it breaks pytest-xdist.
-
#7061: When a yielding fixture fails to yield a value, report a test setup error instead of crashing.
-
#7076: The path of file skipped by
@pytest.mark.skip
in the SKIPPED report is now relative to invocation directory. Previously it was relative to root directory. -
#7110: Fixed regression:
asyncbase.TestCase
tests are executed correctly again. -
#7126:
--setup-show
now doesn't raise an error when a bytes value is used as aparametrize
parameter when Python is called with the-bb
flag. -
#7143: Fix pytest.File.from_parent so it forwards extra keyword arguments to the constructor.
-
#7145: Classes with broken
__getattribute__
methods are displayed correctly during failures. -
#7150: Prevent hiding the underlying exception when
ConfTestImportFailure
is raised. -
#7180: Fix
_is_setup_py
for files encoded differently than locale. -
#7215: Fix regression where running with
--pdb
would call unittest.TestCase.tearDown for skipped tests. -
#7253: When using
pytest.fixture
on a function directly, as inpytest.fixture(func)
, if theautouse
orparams
arguments are also passed, the function is no longer ignored, but is marked as a fixture. -
#7360: Fix possibly incorrect evaluation of string expressions passed to
pytest.mark.skipif
andpytest.mark.xfail
, in rare circumstances where the exact same string is used but refers to different global values. -
#7383: Fixed exception causes all over the codebase, i.e. use raise new_exception from old_exception when wrapping an exception.
Improved Documentation
- #7202: The development guide now links to the contributing section of the docs and RELEASING.rst on GitHub.
-
#7233: Add a note about
--strict
and--strict-markers
and the preference for the latter one. - #7345: Explain indirect parametrization and markers for fixtures.
Trivial/Internal Changes
-
#7035: The
originalname
attribute of_pytest.python.Function
now defaults toname
if not provided explicitly, and is always set. -
#7264: The dependency on the
wcwidth
package has been removed. -
#7291: Replaced
py.iniconfig
with iniconfig. -
#7295:
src/_pytest/config/__init__.py
now uses thewarnings
module to report warnings instead ofsys.stderr.write
. -
#7356: Remove last internal uses of deprecated slave term from old
pytest-xdist
. -
#7357:
py
>=1.8.2 is now required.
v5.4.3
pytest 5.4.3 (2020-06-02)
Bug Fixes
- #6428: Paths appearing in error messages are now correct in case the current working directory has changed since the start of the session.
- #6755: Support deleting paths longer than 260 characters on windows created inside tmpdir.
- #6956: Prevent pytest from printing ConftestImportFailure traceback to stdout.
-
#7150: Prevent hiding the underlying exception when
ConfTestImportFailure
is raised. -
#7215: Fix regression where running with
--pdb
would call thetearDown
methods ofunittest.TestCase
subclasses for skipped tests.
v5.4.2
pytest 5.4.2 (2020-05-08)
Bug Fixes
- #6871: Fix crash with captured output when using the capsysbinary fixture <capsysbinary>.
-
#6924: Ensure a
unittest.IsolatedAsyncioTestCase
is actually awaited. - #6925: Fix TerminalRepr instances to be hashable again.
-
#6947: Fix regression where functions registered with
TestCase.addCleanup
were not being called on test failures. -
#6951: Allow users to still set the deprecated
TerminalReporter.writer
attribute. - #6992: Revert "tmpdir: clean up indirection via config for factories" #6767 as it breaks pytest-xdist.
-
#7110: Fixed regression:
asyncbase.TestCase
tests are executed correctly again. -
#7143: Fix
File.from_constructor
so it forwards extra keyword arguments to the constructor. -
#7145: Classes with broken
__getattribute__
methods are displayed correctly during failures. -
#7180: Fix
_is_setup_py
for files encoded differently than locale.
v5.4.1
pytest 5.4.1 (2020-03-13)
Bug Fixes
-
#6909: Revert the change introduced by #6330, which required all arguments to
@pytest.mark.parametrize
to be explicitly defined in the function signature.The intention of the original change was to remove what was expected to be an unintended/surprising behavior, but it turns out many people relied on it, so the restriction has been reverted.
-
#6910: Fix crash when plugins return an unknown stats while using the
--reportlog
option.
v5.4.0
Breaking Changes
-
#6316: Matching of
-k EXMRESSION
to test names is now case-insensitive. -
#6443: Plugins specified with
-p
are now loaded after internal plugins, which results in their hooks being called before the internal ones.This makes the
-p
behavior consistent withPYTEST_PLUGINS
. -
#6637: Removed the long-deprecated
pytest_itemstart
hook.This hook has been marked as deprecated and not been even called by pytest for over 10 years now.
-
#6673: Reversed / fix meaning of "+/-" in error diffs. "-" means that sth. expected is missing in the result and "+" means that there are unexpected extras in the result.
-
#6737: The
cached_result
attribute ofFixtureDef
is now set toNone
when the result is unavailable, instead of being deleted.If your plugin performs checks like
hasattr(fixturedef, 'cached_result')
, for example in apytest_fixture_post_finalizer
hook implementation, replace it withfixturedef.cached_result is not None
. If youdel
the attribute, set it toNone
instead.
Deprecations
-
#3238: Option
--no-print-logs
is deprecated and meant to be removed in a future release. If you use--no-print-logs
, please try out--show-capture
and provide feedback.--show-capture
command-line option was added inpytest 3.5.0
and allows to specify how to display captured output when tests fail:no
,stdout
,stderr
,log
orall
(the default). -
#571: Deprecate the unused/broken [pytest_collect_directory]{.title-ref} hook. It was misaligned since the removal of the
Directory
collector in 2010 and incorrect/unusable as soon as collection was split from test execution. -
#5975: Deprecate using direct constructors for
Nodes
.Instead they are new constructed via
Node.from_parent
.This transitional mechanism enables us to detangle the very intensely entangled
Node
relationships by enforcing more controlled creation/configruation patterns.As part of that session/config are already disallowed parameters and as we work on the details we might need disallow a few more as well.
Subclasses are expected to use [super().from_parent]{.title-ref} if they intend to expand the creation of [Nodes]{.title-ref}.
-
#6779: The
TerminalReporter.writer
attribute has been deprecated and should no longer be used. This was inadvertently exposed as part of the public API of that plugin and ties it too much withpy.io.TerminalWriter
.
Features
-
#4597: New
--capture=tee-sys <capture-method>
{.interpreted-text role="ref"} option to allow both live printing and capturing of test output. -
#5712: Now all arguments to
@pytest.mark.parametrize
need to be explicitly declared in the function signature or viaindirect
. Previously it was possible to omit an argument if a fixture with the same name existed, which was just an accident of implementation and was not meant to be a part of the API. -
#6454: Changed default for [-r]{.title-ref} to [fE]{.title-ref}, which displays failures and errors in the
short test summary <pytest.detailed_failed_tests_usage>
{.interpreted-text role="ref"}. [-rN]{.title-ref} can be used to disable it (the old behavior). -
#6469: New options have been added to the
junit_logging
{.interpreted-text role="confval"} option:log
,out-err
, andall
. - #6834: Excess warning summaries are now collapsed per file to ensure readable display of warning summaries.
Improvements
-
#1857:
pytest.mark.parametrize
accepts integers forids
again, converting it to strings. -
#449: Use "yellow" main color with any XPASSED tests.
-
#4639: Revert "A warning is now issued when assertions are made for
None
".The warning proved to be less useful than initially expected and had quite a few false positive cases.
-
#5686:
tmpdir_factory.mktemp
now fails when given absolute and non-normalized paths. -
#5984: The
pytest_warning_captured
hook now receives alocation
parameter with the code location that generated the warning. -
#6213: pytester: the
testdir
fixture respects environment settings from themonkeypatch
fixture for inner runs. -
#6247:
--fulltrace
is honored with collection errors. -
#6384: Make [--showlocals]{.title-ref} work also with [--tb=short]{.title-ref}.
-
#6653: Add support for matching lines consecutively with
LineMatcher <_pytest.pytester.LineMatcher>
{.interpreted-text role="attr"}'s~_pytest.pytester.LineMatcher.fnmatch_lines
{.interpreted-text role="func"} and~_pytest.pytester.LineMatcher.re_match_lines
{.interpreted-text role="func"}. -
#6658: Code is now highlighted in tracebacks when
pygments
is installed.Users are encouraged to install
pygments
into their environment and provide feedback, because the plan is to makepygments
a regular dependency in the future. -
#6795: Import usage error message with invalid [-o]{.title-ref} option.
-
#759:
pytest.mark.parametrize
supports iterators and generators forids
.
Bug Fixes
-
#310: Add support for calling [pytest.xfail()]{.title-ref} and [pytest.importorskip()]{.title-ref} with doctests.
-
#3823:
--trace
now works with unittests. -
#4445: Fixed some warning reports produced by pytest to point to the correct location of the warning in the user's code.
-
#5301: Fix
--last-failed
to collect new tests from files with known failures. -
#5928: Report
PytestUnknownMarkWarning
at the level of the user's code, notpytest
's. -
#5991: Fix interaction with
--pdb
and unittests: do not use unittest'sTestCase.debug()
. -
#6334: Fix summary entries appearing twice when
f/F
ands/S
report chars were used at the same time in the-r
command-line option (for example-rFf
).The upper case variants were never documented and the preferred form should be the lower case.
-
#6409: Fallback to green (instead of yellow) for non-last items without previous passes with colored terminal progress indicator.
-
#6454: [--disable-warnings]{.title-ref} is honored with [-ra]{.title-ref} and [-rA]{.title-ref}.
-
#6497: Fix bug in the comparison of request key with cached key in fixture.
A construct
if key == cached_key:
can fail either because==
is explicitly disallowed, or for, e.g., NumPy arrays, where the result ofa == b
cannot generally be converted to [bool]{.title-ref}. The implemented fix replaces [==]{.title-ref} withis
. -
#6557: Make capture output streams
.write()
method return the same return value from original streams. -
#6566: Fix
EncodedFile.writelines
to call the underlying buffer'swritelines
method. -
#6575: Fix internal crash when
faulthandler
starts initialized (for example withPYTHONFAULTHANDLER=1
environment variable set) andfaulthandler_timeout
defined in the configuration file. -
#6597: Fix node ids which contain a parametrized empty-string variable.
-
#6646: Assertion rewriting hooks are (re)stored for the current item, which fixes them being still used after e.g. pytester's
testdir.runpytest <_pytest.pytester.Testdir.runpytest>
{.interpreted-text role="func"} etc. -
#6660:
pytest.exit() <_pytest.outcomes.exit>
{.interpreted-text role="func"} is handled when emitted from thepytest_sessionfinish <_pytest.hookspec.pytest_sessionfinish>
{.interpreted-text role="func"} hook. This includes quitting from a debugger. -
#6752: When :py
pytest.raises
{.interpreted-text role="func"} is used as a function (as opposed to a context manager), a [match]{.title-ref} keyword argument is now passed through to the tested function. Previously it was swallowed and ignored (regression in pytest 5.1.0). -
#6801: Do not display empty lines inbetween traceback for unexpected exceptions with doctests.
-
#6802: The
testdir fixture <testdir>
{.interpreted-text role="fixture"} works within doctests now.
Improved Documentation
- #6696: Add list of fixtures to start of fixture chapter.
- #6742: Expand first sentence on fixtures into a paragraph.
Trivial/Internal Changes
-
#6404: Remove usage of
parser
module, deprecated in Python 3.9.
v5.3.5
pytest 5.3.5 (2020-01-29)
Bug Fixes
- #6517: Fix regression in pytest 5.3.4 causing an INTERNALERROR due to a wrong assertion.
v5.3.4
pytest 5.3.4 (2020-01-20)
Bug Fixes
- #6496: Revert #6436: unfortunately this change has caused a number of regressions in many suites, so the team decided to revert this change and make a new release while we continue to look for a solution.
v5.3.3
pytest 5.3.3 (2020-01-16)
Bug Fixes
-
#2780: Captured
output during teardown is shown with
-rP
. -
#5971: Fix a
pytest-xdist
crash when dealing with exceptions raised in subprocesses created by themultiprocessing
module. - #6436: FixtureDef <_pytest.fixtures.FixtureDef> objects now properly register their finalizers with autouse and parameterized fixtures that execute before them in the fixture stack so they are torn down at the right times, and in the right order.
-
#6532: Fix
parsing of outcomes containing multiple errors with
testdir
results (regression in 5.3.0).
Trivial/Internal Changes
- #6350: Optimized automatic renaming of test parameter IDs.
v5.3.2
pytest 5.3.2 (2019-12-13)
Improvements
-
#4639: Revert "A warning is now issued when assertions are made for
None
".The warning proved to be less useful than initially expected and had quite a few false positive cases.
Bug Fixes
- #5430: junitxml: Logs for failed test are now passed to junit report in case the test fails during call phase.
-
#6290: The
supporting files in the
.pytest_cache
directory are kept with--cache-clear
, which only clears cached values now. -
#6301: Fix
assertion rewriting for egg-based distributions and
editable
installs (pip install --editable
).
v5.3.1
pytest 5.3.1 (2019-11-25)
Improvements
-
#6231: Improve check for misspelling of
pytest.mark.parametrize
. - #6257: Handle exit.Exception raised in notify_exception (via pytest_internalerror), e.g. when quitting pdb from post mortem.
Bug Fixes
-
#5914: pytester: fix
no_fnmatch_line
when used after positive matching. -
#6082: Fix line detection for doctest samples inside
property
docstrings, as a workaround to bpo-17446. - #6254: Fix compatibility with pytest-parallel (regression in pytest 5.3.0).
-
#6255: Clear the
sys.last_traceback
,sys.last_type
andsys.last_value
attributes by deleting them instead of setting them toNone
. This better matches the behaviour of the Python standard library.
v5.3.0
pytest 5.3.0 (2019-11-19)
Deprecations
-
#6179: The default value of
junit_family
option will change toxunit2
in pytest 6.0, given that this is the version supported by default in modern tools that manipulate this type of file.In order to smooth the transition, pytest will issue a warning in case the
--junitxml
option is given in the command line butjunit_family
is not explicitly configured inpytest.ini
.For more information, see the docs.
Features
-
#4488: The pytest team has created the pytest-reportlog plugin, which provides a new
--report-log=FILE
option that writes report logs into a file as the test session executes.Each line of the report log contains a self contained JSON object corresponding to a testing event, such as a collection or a test result report. The file is guaranteed to be flushed after writing each line, so systems can read and process events in real-time.
The plugin is meant to replace the
--resultlog
option, which is deprecated and meant to be removed in a future release. If you use--resultlog
, please try outpytest-reportlog
and provide feedback. -
#4730: When
sys.pycache_prefix
(Python 3.8+) is set, it will be used by pytest to cache test files changed by the assertion rewriting mechanism.This makes it easier to benefit of cached
.pyc
files even on file systems without permissions. -
#5515: Allow selective auto-indentation of multiline log messages.
Adds command line option
--log-auto-indent
, config optionlog_auto_indent
and support for per-entry configuration of indentation behavior on calls tologging.log()
.Alters the default for auto-indention from
on
tooff
. This restores the older behavior that existed prior to v4.6.0. This reversion to earlier behavior was done because it is better to activate new features that may lead to broken tests explicitly rather than implicitly. -
#5914:
pytester
learned two new functions, no_fnmatch_line and no_re_match_line.The functions are used to ensure the captured text does not match the given pattern.
The previous idiom was to use
re.match
:assert re.match(pat, result.stdout.str()) is None
Or the
in
operator:assert text in result.stdout.str()
But the new functions produce best output on failure.
-
#6057: Added tolerances to complex values when printing
pytest.approx
.For example,
repr(pytest.approx(3+4j))
returns(3+4j) ± 5e-06 ∠ ±180°
. This is polar notation indicating a circle around the expected value, with a radius of 5e-06. Forapprox
comparisons to returnTrue
, the actual value should fall within this circle. -
#6061: Added the pluginmanager as an argument to
pytest_addoption
so that hooks can be invoked when setting up command line options. This is useful for having one plugin communicate things to another plugin, such as default values or which set of command line options to add.
Improvements
-
#5061: Use multiple colors with terminal summary statistics.
-
#5630: Quitting from debuggers is now properly handled in
doctest
items. -
#5924: Improved verbose diff output with sequences.
Before:
E AssertionError: assert ['version', '...version_info'] == ['version', '...version', ...] E Right contains 3 more items, first extra item: ' ' E Full diff: E - ['version', 'version_info', 'sys.version', 'sys.version_info'] E + ['version', E + 'version_info', E + 'sys.version', E + 'sys.version_info', E + ' ', E + 'sys.version', E + 'sys.version_info']
After:
E AssertionError: assert ['version', '...version_info'] == ['version', '...version', ...] E Right contains 3 more items, first extra item: ' ' E Full diff: E [ E 'version', E 'version_info', E 'sys.version', E 'sys.version_info', E + ' ', E + 'sys.version', E + 'sys.version_info', E ]
-
#5936: Display untruncated assertion message with
-vv
. -
#5990: Fixed plurality mismatch in test summary (e.g. display "1 error" instead of "1 errors").
-
#6008:
Config.InvocationParams.args
is now always atuple
to better convey that it should be immutable and avoid accidental modifications. -
#6023:
pytest.main
now returns apytest.ExitCode
instance now, except for when custom exit codes are used (where it returnsint
then still). -
#6026: Align prefixes in output of pytester's
LineMatcher
. -
#6059: Collection errors are reported as errors (and not failures like before) in the terminal's short test summary.
-
#6069:
pytester.spawn
does not skip/xfail tests on FreeBSD anymore unconditionally. -
#6097: The "[XXX%]" indicator in the test summary is now colored according to the final (new) multi-colored line's main color.
-
#6116: Added
--co
as a synonym to--collect-only
. -
#6148:
atomicwrites
is now only used on Windows, fixing a performance regression with assertion rewriting on Unix. -
#6152: Now parametrization will use the
__name__
attribute of any object for the id, if present. Previously it would only use__name__
for functions and classes. -
#6176: Improved failure reporting with pytester's
Hookrecorder.assertoutcome
. -
#6181: The reason for a stopped session, e.g. with
--maxfail
/-x
, now gets reported in the test summary. -
#6206: Improved
cache.set
robustness and performance.
Bug Fixes
-
#2049: Fixed
--setup-plan
showing inaccurate information about fixture lifetimes. -
#2548: Fixed line offset mismatch of skipped tests in terminal summary.
-
#6039: The
PytestDoctestRunner
is now properly invalidated when unconfiguring the doctest plugin.This is important when used with
pytester
'srunpytest_inprocess
. -
#6047: BaseExceptions are now handled in
saferepr
, which includespytest.fail.Exception
etc. -
#6074: pytester: fixed order of arguments in
rm_rf
warning when cleaning up temporary directories, and do not emit warnings for errors withos.open
. -
#6189: Fixed result of
getmodpath
method.
Trivial/Internal Changes
-
#4901:
RunResult
frompytester
now displays the mnemonic of theret
attribute when it is a validpytest.ExitCode
value.
v5.2.4
pytest 5.2.4 (2019-11-15)
Bug Fixes
-
#6194: Fix
incorrect discovery of non-test
__init__.py
files. -
#6197: Revert
"The first test in a package (
__init__.py
) marked with@pytest.mark.skip
is now correctly skipped.".
v5.2.3
pytest 5.2.3 (2019-11-14)
Bug Fixes
-
#5830: The
first test in a package (
__init__.py
) marked with@pytest.mark.skip
is now correctly skipped. -
#6099: Fix
--trace
when used with parametrized functions. -
#6183: Using
request
as a parameter name in@pytest.mark.parametrize
now produces a more user-friendly error.
v5.2.2
pytest 5.2.2 (2019-10-24)
Bug Fixes
-
#5206: Fix
--nf
to not forget about known nodeids with partial test selection. -
#5906: Fix
crash with
KeyboardInterrupt
during--setup-show
. - #5946: Fixed issue when parametrizing fixtures with numpy arrays (and possibly other sequence-like types).
-
#6044: Properly
ignore
FileNotFoundError
exceptions when trying to remove old temporary directories, for instance when multiple processes try to remove the same directory (common withpytest-xdist
for example).
v5.2.1
pytest 5.2.1 (2019-10-06)
Bug Fixes
-
#5902: Fix
warnings about deprecated
cmp
attribute inattrs>=19.2
.
v5.2.0
pytest 5.2.0 (2019-09-28)
Deprecations
- #1682: Passing arguments to pytest.fixture() as positional arguments is deprecated - pass them as a keyword argument instead.
Features
-
#1682: The
scope
parameter of@pytest.fixture
can now be a callable that receives the fixture name and theconfig
object as keyword-only parameters. See the docs for more information. -
#5764: New
behavior of the
--pastebin
option: failures to connect to the pastebin server are reported, without failing the pytest run
Bug Fixes
-
#5806: Fix
"lexer" being used when uploading to bpaste.net from
--pastebin
to "text". -
#5884: Fix
--setup-only
and--setup-show
for custom pytest items.
Trivial/Internal Changes
-
#5056: The
HelpFormatter uses
py.io.get_terminal_width
for better width detection.
v5.1.3
pytest 5.1.3 (2019-09-18)
Bug Fixes
- #5807: Fix pypy3.6 (nightly) on windows.
-
#5811: Handle
--fulltrace
correctly withpytest.raises
. - #5819: Windows: Fix regression with conftest whose qualified name contains uppercase characters (introduced by #5792).
v5.1.2
pytest 5.1.2 (2019-08-30)
Bug Fixes
-
#2270: Fixed
self
reference in function-scoped fixtures defined plugin classes: previouslyself
would be a reference to a test class, not the plugin class. - #570: Fixed long standing issue where fixture scope was not respected when indirect fixtures were used during parametrization.
-
#5782: Fix
decoding error when printing an error response from
--pastebin
. -
#5786: Chained
exceptions in test and collection reports are now correctly
serialized, allowing plugins like
pytest-xdist
to display them properly. -
#5792: Windows:
Fix error that occurs in certain circumstances when loading
conftest.py
from a working directory that has casing other than the one stored in the filesystem (e.g.,c:\test
instead ofC:\test
).
v5.1.1
pytest 5.1.1 (2019-08-20)
Bug Fixes
-
#5751: Fixed
TypeError
when importing pytest on Python 3.5.0 and 3.5.1.
v5.1.0
pytest 5.1.0 (2019-08-15)
Removals
-
#5180: As per our policy, the following features have been deprecated in the 4.X series and are now removed:
-
Request.getfuncargvalue
: useRequest.getfixturevalue
instead. -
pytest.raises
andpytest.warns
no longer support strings as the second argument. -
message
parameter ofpytest.raises
. -
pytest.raises
,pytest.warns
andParameterSet.param
now use native keyword-only syntax. This might change the exception message from previous versions, but they still raiseTypeError
on unknown keyword arguments as before. -
pytest.config
global variable. -
tmpdir_factory.ensuretemp
method. -
pytest_logwarning
hook. -
RemovedInPytest4Warning
warning type. -
request
is now a reserved name for fixtures.
For more information consult Deprecations and Removals in the docs.
-
-
#5565: Removed unused support code for unittest2.
The
unittest2
backport module is no longer necessary since Python 3.3+, and the small amount of code in pytest to support it also doesn't seem to be used: after removed, all tests still pass unchanged.Although our policy is to introduce a deprecation period before removing any features or support for third party libraries, because this code is apparently not used at all (even if
unittest2
is used by a test suite executed by pytest), it was decided to remove it in this release.If you experience a regression because of this, please file an issue.
-
#5615:
pytest.fail
,pytest.xfail
andpytest.skip
no longer support bytes for the message argument.This was supported for Python 2 where it was tempting to use
"message"
instead ofu"message"
.Python 3 code is unlikely to pass
bytes
to these functions. If you do, please decode it to anstr
beforehand.
Features
-
#5564: New
Config.invocation_args
attribute containing the unchanged arguments passed topytest.main()
. - #5576: New NUMBER option for doctests to ignore irrelevant differences in floating-point numbers. Inspired by Sébastien Boisgérault's numtest extension for doctest.
Improvements
-
#5471: JUnit XML now includes a timestamp and hostname in the testsuite tag.
-
#5707: Time taken to run the test suite now includes a human-readable representation when it takes over 60 seconds, for example:
===== 2 failed in 102.70s (0:01:42) =====
Bug Fixes
- #4344: Fix RuntimeError/StopIteration when trying to collect package with "__init__.py" only.
-
#5115: Warnings
issued during
pytest_configure
are explicitly not treated as errors, even if configured as such, because it otherwise completely breaks pytest. -
#5477: The XML
file produced by
--junitxml
now correctly contain a<testsuites>
root element. -
#5523: Fixed
using multiple short options together in the command-line (for
example
-vs
) in Python 3.8+. -
#5524: Fix
issue where
tmp_path
andtmpdir
would not remove directories containing files marked as read-only, which could lead to pytest crashing when executed a second time with the--basetemp
option. -
#5537: Replace
importlib_metadata
backport withimportlib.metadata
from the standard library on Python 3.8+. -
#5578: Improve
type checking for some exception-raising functions (
pytest.xfail
,pytest.skip
, etc) so they provide better error messages when users meant to use marks (for example@pytest.xfail
instead of@pytest.mark.xfail
). -
#5606: Fixed
internal error when test functions were patched with objects that
cannot be compared for truth values against others, like
numpy
arrays. -
#5634:
pytest.exit
is now correctly handled inunittest
cases. This makesunittest
cases handlequit
from pytest's pdb correctly. - #5650: Improved output when parsing an ini configuration file fails.
-
#5701: Fix
collection of
staticmethod
objects defined withfunctools.partial
. -
#5734: Skip
async generator test functions, and update the warning message to
refer to
async def
functions.
Improved Documentation
-
#5669: Add
docstring for
Testdir.copy_example
.
Trivial/Internal Changes
-
#5095: XML
files of the
xunit2
family are now validated against the schema by pytest's own test suite to avoid future regressions. - #5516: Cache node splitting function which can improve collection performance in very large test suites.
-
#5603:
Simplified internal
SafeRepr
class and removed some dead code. -
#5664: When
invoking pytest's own testsuite with
PYTHONDONTWRITEBYTECODE=1
, thetest_xfail_handling
test no longer fails. -
#5684: Replace
manual handling of
OSError.errno
in the codebase by newOSError
subclasses (PermissionError
,FileNotFoundError
, etc.).
v5.0.1
pytest 5.0.1 (2019-07-04)
Bug Fixes
-
#5479: Improve
quoting in
raises
match failure message. -
#5523: Fixed
using multiple short options together in the command-line (for
example
-vs
) in Python 3.8+. -
#5547:
--step-wise
now handlesxfail(strict=True)
markers properly.
Improved Documentation
- #5517: Improve "Declaring new hooks" section in chapter "Writing Plugins"
v5.0.0
v4.6.11
Bug Fixes
-
#6334: Fix summary entries appearing twice when
f/F
ands/S
report chars were used at the same time in the-r
command-line option (for example-rFf
).The upper case variants were never documented and the preferred form should be the lower case.
-
#7310: Fix
UnboundLocalError: local variable 'letter' referenced before assignment
in_pytest.terminal.pytest_report_teststatus()
when plugins return report objects in an unconventional state.This was making
pytest_report_teststatus()
skip entering if-block branches that declare theletter
variable.The fix was to set the initial value of the
letter
before the if-block cascade so that it always has a value.
v4.6.10
-
#6870: New
Config.invocation_args
attribute containing the unchanged arguments passed topytest.main()
.Remark: while this is technically a new feature and according to our policy it should not have been backported, we have opened an exception in this particular case because it fixes a serious interaction with pytest-xdist, so it can also be considered a bugfix.
v4.6.9
pytest 4.6.9 (2020-01-04)
Bug Fixes
-
#6301: Fix
assertion rewriting for egg-based distributions and
editable
installs (pip install --editable
).
v4.6.8
pytest 4.6.8 (2019-12-19)
Features
- #5471: JUnit XML now includes a timestamp and hostname in the testsuite tag.
Bug Fixes
- #5430: junitxml: Logs for failed test are now passed to junit report in case the test fails during call phase.
Trivial/Internal Changes
-
#6345: Pin
colorama
to0.4.1
only for Python 3.4 so newer Python versions can still receive colorama updates.
v4.6.7
pytest 4.6.7 (2019-12-05)
Bug Fixes
-
#5477: The XML
file produced by
--junitxml
now correctly contain a<testsuites>
root element. -
#6044: Properly
ignore
FileNotFoundError
(OSError.errno == NOENT
in Python 2) exceptions when trying to remove old temporary directories, for instance when multiple processes try to remove the same directory (common withpytest-xdist
for example).
v4.6.6
pytest 4.6.6 (2019-10-11)
Bug Fixes
-
#5523: Fixed
using multiple short options together in the command-line (for
example
-vs
) in Python 3.8+. -
#5537: Replace
importlib_metadata
backport withimportlib.metadata
from the standard library on Python 3.8+. -
#5806: Fix
"lexer" being used when uploading to bpaste.net from
--pastebin
to "text". -
#5902: Fix
warnings about deprecated
cmp
attribute inattrs>=19.2
.
Trivial/Internal Changes
-
#5801: Fixes
python version checks (detected by
flake8-2020
) in case python4 becomes a thing.
v4.6.5
Bug Fixes
-
#4344: Fix RuntimeError/StopIteration when trying to collect package with "init.py" only.
-
#5478: Fix encode error when using unicode strings in exceptions with
pytest.raises
. -
#5524: Fix issue where
tmp_path
andtmpdir
would not remove directories containing files marked as read-only, which could lead to pytest crashing when executed a second time with the--basetemp
option. -
#5547:
--step-wise
now handlesxfail(strict=True)
markers properly. -
#5650: Improved output when parsing an ini configuration file fails.
v4.6.4
Configuration
-
If you want to rebase/retry this MR, click this checkbox.
This MR has been generated by Renovate Bot.