Skip to content

TST: add usage of pytest-rerunfailures for remote-tests#2427

Merged
bsipocz merged 1 commit into
astropy:mainfrom
bsipocz:TST_add_rerunfailures
Jun 2, 2022
Merged

TST: add usage of pytest-rerunfailures for remote-tests#2427
bsipocz merged 1 commit into
astropy:mainfrom
bsipocz:TST_add_rerunfailures

Conversation

@bsipocz

@bsipocz bsipocz commented Jun 1, 2022

Copy link
Copy Markdown
Member

This PR also adds online to tox to deal with remote-data dependencies and cmd options

The actual parameters may need some tuning, as well as keeping an eye on pytest-dev/pytest-rerunfailures#193 as it would be more optimal for us to come back and revisit our failing test at the end of the test run.

Note: second commit temporarily disables the cron conditional, it will need to be removed once we make sure the configs are picked up properly (the CI job will most certainly fail)

To close #2424

@bsipocz bsipocz added this to the v0.4.7 milestone Jun 1, 2022
@bsipocz bsipocz force-pushed the TST_add_rerunfailures branch 2 times, most recently from 593ce15 to 0e5d1a5 Compare June 2, 2022 00:39
@codecov

codecov Bot commented Jun 2, 2022

Copy link
Copy Markdown

Codecov Report

Merging #2427 (432f141) into main (be0938f) will not change coverage.
The diff coverage is n/a.

❗ Current head 432f141 differs from pull request most recent head c194d86. Consider uploading reports for the commit c194d86 to get more accurate results

@@           Coverage Diff           @@
##             main    #2427   +/-   ##
=======================================
  Coverage   62.97%   62.97%           
=======================================
  Files         133      133           
  Lines       17290    17290           
=======================================
  Hits        10888    10888           
  Misses       6402     6402           

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@bsipocz

bsipocz commented Jun 2, 2022

Copy link
Copy Markdown
Member Author

I've cancelled the CI once it demonstrated that all configs are picked up correctly:

../../.tox/py39-test-alldeps-devdeps-online/lib/python3.9/site-packages/astroquery/alma/tests/test_alma_remote.py::TestAlma::test_download_and_extract PASSED [  1%]
../../.tox/py39-test-alldeps-devdeps-online/lib/python3.9/site-packages/astroquery/alma/tests/test_alma_remote.py::TestAlma::test_doc_example RERUN [  1%]
../../.tox/py39-test-alldeps-devdeps-online/lib/python3.9/site-packages/astroquery/alma/tests/test_alma_remote.py::TestAlma::test_doc_example FAILED [  1%]

I go ahead and remove the temporary config, and go ahead and merge without waiting for CI as it only affects the cron jobs.

@bsipocz bsipocz force-pushed the TST_add_rerunfailures branch from 432f141 to c194d86 Compare June 2, 2022 00:58
@bsipocz bsipocz merged commit eceb776 into astropy:main Jun 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ENH: make remote tests more roboust by rerunning them

1 participant