Testing

Testing is done using Selenium WebDriver for automating browser tests. Tox is used to execute testing procedures.

Before running all tests locally, ChromeDriver (for testing on Chrome browser) and GeckoDriver (for testing on Firefox browser) are required.

Install Tox via pip:

pip install tox

Download and setup the ChromeDriver:

wget -N https://chromedriver.storage.googleapis.com/75.0.3770.8/chromedriver_linux64.zip -P ~/
unzip ~/chromedriver_linux64.zip -d ~/
rm ~/chromedriver_linux64.zip
sudo mv -f ~/chromedriver /usr/local/bin/
sudo chmod +x /usr/local/bin/chromedriver

Download and setup the GeckoDriver:

wget -N https://github.com/mozilla/geckodriver/releases/download/v0.24.0/geckodriver-v0.24.0-linux64.tar.gz -P ~/
tar xvzf ~/geckodriver-v0.24.0-linux64.tar.gz -C ~/
rm ~/geckodriver-v0.24.0-linux64.tar.gz
sudo mv -f ~/geckodriver /usr/local/bin/
sudo chmod +x /usr/local/bin/geckodriver

To run the full test suite against your changes, simply run Tox. Tox should return without any errors. You can run Tox against all of our environments and both browsers by running:

tox

To run tests with a specific environment and for both browsers:

tox -e py36-sphinx20

To run tests with a specific environment and for a specified browser:

tox -e py36-sphinx20 -- --driver Chrome  # run tests with Python 3.6 and Sphinx < 2.1 with Chrome browser
tox -e py36-sphinx20 -- --driver Firefox  # run tests with Python 3.6 and Sphinx < 2.1 with Firefox browser

To run tests against all environments but with a specified browser:

tox -- --driver Chrome  # run tests with all environments with Chrome browser
tox -- --driver Firefox  # run tests with all environments with Firefox browser

The tox configuration has the following environments configured. You can target a single environment to limit the test suite:

py36-sphinx18
    Run test suite with Python 3.6 and Sphinx<1.9

py36-sphinx20
    Run test suite with Python 3.6 and Sphinx<2.1

py36-sphinx21
    Run test suite with Python 3.6 and Sphinx<2.2

py37-sphinx18
    Run test suite with Python 3.7 and Sphinx <1.9

py37-sphinx20
    Run test suite with Python 3.7 and Sphinx<2.1

py37-sphinx21
    Run test suite with Python 3.7 and Sphinx<2.2

docs
    Test documentation compilation with Sphinx

Continuous Integration

For every push to GitHub, Travis CI runs all the tests. You can check the current build status: https://travis-ci.org/readthedocs/readthedocs-sphinx-search