The goal of lazytest is to save development time by helping you rerun only the tests that have failed during the last run. It integrates tightly with the testthat package and provides the lazytest_local() function, a drop-in replacement for testthat::test_local(), that
- memoizes which tests have failed;
- runs only those tests in subsequent runs.
If all active tests have succeeded, the entire test suite is run in a second pass.
Installation
The development version of lazytest can be installed from R-universe:
# Enable repository from cynkra
options(
repos = c(
cynkra = "https://cynkra.r-universe.dev",
CRAN = "https://cloud.r-project.org"
)
)
# Download and install lazytest in R
install.packages("lazytest")or from GitHub:
# install.packages("pak")
pak::pak("cynkra/lazytest")Usage
Call lazytest_local() instead of testthat::test_local() or devtools::test():
lazytest::lazytest_local()The package also provides RStudio add-ins that run the tests in a new terminal. Unfortunately, the “Test package” command is hard-wired to devtools::test(), and there seems to be no way to customize it or hook into it.
If you’re using RStudio, it is a good idea to remap the shortcut for running tests (default: Ctrl + Shift + T / Cmd + Shift + T). The add-in provides two commands:
Run Lazy Tests in New Terminal (recommended mapping: Ctrl + Shift + T / Cmd + Shift + T)
Reset And Run Lazy Tests in New Terminal (recommended mapping: Ctrl + T / Cmd + T)
This allows you to keep the workflows you’re accustomed to and to benefit immediately.
Example
Let’s create a package with two boilerplate tests.
withr::local_seed(42)
withr::local_options(usethis.quiet = TRUE)
pkg_parent_dir <- withr::local_tempdir()
pkg_dir <- file.path(pkg_parent_dir, "tea")
usethis::create_package(pkg_dir)
usethis::with_project(path = pkg_dir, {
usethis::use_testthat()
usethis::use_test("blop")
usethis::use_test("blip")
})If we run the tests, they all pass.
withr::with_dir(
pkg_dir,
lazytest::lazytest_local()
)
#> ℹ Testing all tests.
#> ✔ | F W S OK | Context
#>
#> ⠏ | 0 | blip
#> ✔ | 1 | blip
#>
#> ⠏ | 0 | blop
#> ✔ | 1 | blop
#>
#> ══ Results ═════════════════════════════════════════════════════════════════════
#> [ FAIL 0 | WARN 0 | SKIP 0 | PASS 2 ]
#> ✔ All tests passed.
#> → Testing all tests next time.Now if we replace one of the tests with a failing test,
brio::write_lines(
text = c(
'test_that("blop() works", {',
'expect_equal(2 * 2, 42)',
'})'
),
path = file.path(pkg_dir, "tests", "testthat", "test-blop.R")
)and then run the tests,
withr::with_dir(
pkg_dir,
lazytest::lazytest_local()
)
#> ℹ Testing all tests.
#> ✔ | F W S OK | Context
#>
#> ⠏ | 0 | blip
#> ✔ | 1 | blip
#>
#> ⠏ | 0 | blop
#> ⠋ | 1 0 | blop
#> ✖ | 1 0 | blop
#> --------------------------------------------------------------------------------
#> Failure ('test-blop.R:2:1'): blop() works
#> Expected `2 * 2` to equal 42.
#> Differences:
#> `actual`: 4.0
#> `expected`: 42.0
#>
#> --------------------------------------------------------------------------------
#>
#> ══ Results ═════════════════════════════════════════════════════════════════════
#> -- Failed tests ----------------------------------------------------------------
#> Failure ('test-blop.R:2:1'): blop() works
#> Expected `2 * 2` to equal 42.
#> Differences:
#> `actual`: 4.0
#> `expected`: 42.0
#>
#>
#> [ FAIL 1 | WARN 0 | SKIP 0 | PASS 1 ]
#> → Testing the following tests next time:
#> • blop
#> Error:
#> ! Test failuresa file is created with the failing test name:
brio::read_lines(file.path(pkg_dir, ".lazytest"))
#> [1] "blop"Next time we run the tests, only this test will be run, until it is fixed at which point all tests are run again to check no failure has been introduced elsewhere.
withr::with_dir(
pkg_dir,
lazytest::lazytest_local()
)
#> ℹ Testing only tests that failed last time:
#> • blop
#> ✔ | F W S OK | Context
#>
#> ⠏ | 0 | blop
#> ✖ | 1 0 | blop
#> --------------------------------------------------------------------------------
#> Failure ('test-blop.R:2:1'): blop() works
#> Expected `2 * 2` to equal 42.
#> Differences:
#> `actual`: 4.0
#> `expected`: 42.0
#>
#> --------------------------------------------------------------------------------
#>
#> ══ Results ═════════════════════════════════════════════════════════════════════
#> -- Failed tests ----------------------------------------------------------------
#> Failure ('test-blop.R:2:1'): blop() works
#> Expected `2 * 2` to equal 42.
#> Differences:
#> `actual`: 4.0
#> `expected`: 42.0
#>
#>
#> [ FAIL 1 | WARN 0 | SKIP 0 | PASS 0 ]
#> → Repeating the same tests next time.
#> Error:
#> ! Test failuresHow does it work?
testthat::test_local() returns an object from which the tests that have failed can be retrieved. lazytest_local() wraps this function. If tests have failed, a file named .lazytest is written in the package directory. In the next call, if .lazytest exists, it is consulted, and a suitable filter argument is constructed and passed to testthat::test_local().
When all tests have passed and not all tests were run, a second call to testthat::test_local() is initiated, to make sure that no failures have been introduced in the meantime.
The presence of a .lazytest file in the package source indicates that the last test run has failed somewhere. You should not need to gitignore or Rbuildignore it: you should fix the reason behind the test failure, then run tests again, before committing to your repository’s default branch for instance.
Getting help
If you encounter a clear bug, please file an issue with a minimal reproducible example on GitHub.