pytest

The pytest framework makes it easy to write small tests, yet scales to support complex functional testing for applications and libraries.

Version

5.3.5

Runtime

Python Versions

Supported Languages

Python

Official Documentation

https://docs.pytest.org/en/latest/

YAML Configuration

pytest:

  • input:

    Patterns to include in execution and reports.

  • ignore:

    Patterns to exclude from execution and reports.

  • auto-fix:

    N/A

  • config-file:

    Alias of -c option.

  • machine:

    • cpu:

      Amount of CPU. The default machine has 0.75 CPU with 2880 MiB RAM.

  • options:

    Below is the list of options that are supported:

    general:
      -k EXPRESSION         only run tests which match the given substring expression. An expression is a python evaluatable expression where all names are substring-matched against
                            test names and their parent classes. Example: -k 'test_method or test_other' matches all test functions and classes whose name contains 'test_method' or
                            'test_other', while -k 'not test_method' matches those that don't contain 'test_method' in their names. -k 'not test_method and not test_other' will
                            eliminate the matches. Additionally keywords are matched to classes and functions containing extra names in their 'extra_keyword_matches' set, as well as
                            functions which have names assigned directly to them.
      -m MARKEXPR           only run tests matching given mark expression. example: -m 'mark1 and not mark2'.
      -x, --exitfirst       exit instantly on first error or failed test.
      --maxfail=num         exit after first num failures or errors.
      --strict-markers, --strict
                            markers not registered in the `markers` section of the configuration file raise errors.
      -c file               load configuration from `file` instead of trying to locate one of the implicit configuration files.
      --continue-on-collection-errors
                            Force test execution even if collection errors occur.
      --rootdir=ROOTDIR     Define root directory for tests. Can be relative path: 'root_dir', './root_dir', 'root_dir/another_dir/'; absolute path: '/home/user/root_dir'; path with
                            variables: '$HOME/root_dir'.
      --import-mode={prepend,append}
                            prepend/append to sys.path when importing test modules, default is to prepend.
      --runxfail            report the results of xfail tests as if they were not marked
      --nf, --new-first     run tests from new files first, then the rest of the tests sorted by file mtime
      collection:
      --ignore=path         ignore path during collection (multi-allowed).
      --ignore-glob=path    ignore path pattern during collection (multi-allowed).
      --deselect=nodeid_prefix
                            deselect item during collection (multi-allowed).
      --confcutdir=dir      only load conftest.py's relative to specified dir.
      --noconftest          Don't load any conftest.py files.
      --keep-duplicates     Keep duplicate tests.
      --collect-in-virtualenv
                            Don't ignore tests in a local virtualenv directory
      --doctest-modules     run doctests in all .py modules
      --doctest-glob=pat    doctests file matching pattern, default: test*.txt
      --doctest-ignore-import-errors
                            ignore doctest ImportErrors
      --doctest-continue-on-failure
                            for a given doctest, continue to run after the first failure
      test session debugging and configuration:
      -p name               early-load given plugin module name or entry point (multi-allowed). To avoid loading of plugins, use the `no:` prefix, e.g. `no:doctest`.
      -o OVERRIDE_INI, --override-ini=OVERRIDE_INI
                            override ini option with "option=value" style, e.g. `-o xfail_strict=True -o cache_dir=cache`.
      pytest-warnings:
      -W PYTHONWARNINGS, --pythonwarnings=PYTHONWARNINGS
                            set which warnings to report, see -W option of python itself.
    
  • thresholds:

    • num-issues:

      No tool specific severity levels are available.

  • experimental:

    • incremental:

      N/A

      Note: The incremental analysis cannot be supported for pytest. This is because pytest requires dependencies to run tests and the results on unchanged files can be affected by changed files.

YAML Examples

  • With default options:

    inspecode:
      pytest: default
    
  • With custom machine:

    inspecode:
      pytest:
        machine:
          cpu: 1.5 # 1.5 CPU, 5760 MiB RAM
    
  • With custom options:

    inspecode:
      pytest:
        options:
          --maxfail: 10
    
  • With a configuration file:

    inspecode:
      pytest:
        config-file: ./my-pytest.ini
    

    The above configuration is equivalent to:

    inspecode:
      pytest:
        options:
          -c: ./my-pytest.ini
    

Configuration File Detection

If no configuration files are specified via config-file or options field, Inspecode walks the file tree in lexical order in your repository to find the following configuration files and executes pytest with -c option for each file found.

  • setup.cfg with [tool:pytest] section
  • tox.ini with [pytest] section
  • pytest.ini

Note: Inspecode ignores specific configuration files while searching. See Configuration > Tool Configuration > Field: config-file for detail.

If the configuration files detected by Inspecode are not the ones you want to use or you do not want to use any configuration files, you can disable Inspecode from detecting configuration files by explicitly setting an empty string to config-file field like below:

inspecode:
  pytest:
    config-file: ""

Resolving Dependencies

Inspecode automatically detects the following dependencies in your repository and tries to resolve them before running pytest:

  • Setuptools

    Inspecode searches your repository for setup.py files and runs pip install --user at each directory containing the file.

    Note: As of now, Inspecode does not install dependencies listed in extras-require.

  • Requirements Files

    Inspecode recursively searches your repository for requirements files and runs pip install --user -r at each directory containing the file.

    The names of the requirements files Inspecode searches for are as listed below:

    • requirements.txt
    • rrequirements.txt
    • requirements-txt
    • rrequirements-txt
    • requirements-dev.txt
    • requirements-dev
    • requirements-test.txt
    • requirements-test
    • requirements
    • devRequirements.txt
    • devRequirements
    • requirements.dev
    • requirements-dev
    • requirements.dev.txt
    • requirements-dev.txt
    • dev.requirements
    • dev-requirements
    • dev.requirements.txt
    • dev-requirements.txt
    • testRequirements.txt
    • testRequirements
    • requirements.test
    • requirements-test
    • requirements.test.txt
    • requirements-test.txt
    • test.requirements
    • test-requirements
    • test.requirements.txt
    • test-requirements.txt

results matching ""

    No results matching ""