Last Revised: January 26, 2022
Hosting companies can host hundreds of websites with WordPress, and that’s why it’s important that their configuration is as compatible as possible with the software.
To verify this compatibility, the WordPress Hosting Community team makes available a series of PHPUnit tests with which to check the operation of WordPress in any environment.
- Requirements
- Installing phpunit-test-runner
- Creating your bot for WordPress.org
- Automating the bot
- Full tests
- Improving configuration
- WordPress Community Hosting Team
- About this document
Requirements
The objective of these tests is to check and validate that the next WordPress updates work correctly in your environment, so the ideal is to have hosting as close as possible to the one you offer your customers.
This means that we will need, at least:
- A database (its contents will be created and deleted). You can use MySQL, MariaDB or any supported system.
- A folder to host the software (not public).
- A temporary folder where content will be generated and deleted.
In addition to this, extra, the following elements will be needed:
- npm (to use Node.js)
- Composer 2.x
- Git
Installing on Ubuntu (example)
npm on Ubuntu
apt-get -y install npm
Composer 2.x on Ubuntu
cd
curl -sS https://getcomposer.org/installer -o composer-setup.php
php composer-setup.php --install-dir=/usr/local/bin --filename=composer
Git on Ubuntu
apt-get -y install git
Installing phpunit-test-runner
The phpunit-test-runner is a small piece of PHPUnit tests specifically designed for hosting companies. There is a whole documentation about this tool. Also, if you want, you can make your test results appear on the WordPress Host Test Results page.
The tool can be run manually or through an automated system. To see the operation and purpose of this document we will do the tests manually, and at the end we will add an automation system.
Runner Installation
The first thing we will do is download and synchronize the tool.
cd /home/wordpressbot/
git clone https://github.com/WordPress/phpunit-test-runner.git wordpress-test
cd /home/wordpressbot/wordpress-test/
The next step will be to configure the environment. To do this we will first make a copy of the sample file and then configure it.
cp .env.default .env
vim .env
The content (in summary form) can be something like this:
export WPT_PREPARE_DIR=/tmp/wp-test-runner
export WPT_TEST_DIR=/tmp/wp-test-runner
export WPT_REPORT_API_KEY=
export WPT_REPORT_URL=
export WPT_DB_NAME=wordpressdatabase
export WPT_DB_USER=wordpressusername
export WPT_DB_PASSWORD=wordpresspassword
export WPT_DB_HOST=localhost
export WPT_TABLE_PREFIX=${WPT_TABLE_PREFIX-wptests_}
export WPT_PHP_EXECUTABLE=${WPT_PHP_EXECUTABLE-php}
export WPT_PHPUNIT_CMD=
export WPT_RM_TEST_DIR_CMD=
export WPT_SSH_CONNECT=
export WPT_SSH_OPTIONS=
export WPT_SSH_PRIVATE_KEY_BASE64=
export WPT_DEBUG=
We will configure the folder where the download tests of the WordPress software and the database accesses will be done to be able to prepare the tests.
For this example, we will use a system test folder such /tmp/wp-test-runner
as . You can use a temporary folder or an existing folder for that same user, for example. We will configure this complete route in:
export WPT_PREPARE_DIR=/tmp/wp-test-runner
export WPT_TEST_DIR=/tmp/wp-test-runner
The next thing will be to include the configuration of the database that we have previously created. For example, we can create it with this command:
CREATE DATABASE wordpressdatabase CHARACTER SET = utf8mb4 COLLATE = utf8mb4_general_ci;
GRANT ALL ON wordpressdatabase.* TO 'wordpressusername'@'localhost' IDENTIFIED BY 'wordpresspassword';
FLUSH PRIVILEGES;
This database should not be used for anything other than the use of these tests.
Subsequently, we will configure the data in the configuration file:
export WPT_DB_NAME=wordpressdatabase
export WPT_DB_USER=wordpressusername
export WPT_DB_PASSWORD=wordpresspassword
export WPT_DB_HOST=localhost
This should be enough for the execution of the tests.
Now that we have the configuration ready, we can do the first test to validate that everything works correctly.
Testing the test
Before doing the tests we will update everything necessary to the latest version and activate the configuration of the environment.
cd /home/wordpressbot/wordpress-test/
git pull
npm update
source .env
We will enter the folder where we have the testing software and update it by downloading the latest version. We will also update the npm in case there are updates, and finally, we will load the configuration.
From here, we only have to run the 4 files that do the set of tests.
The first of these is the system that verifies that the environment is ready to go.
php prepare.php
The second of them is the one that performs the tests itself.
php test.php
Once the tests are finished, the report can be generated. If you have set up a username and password it will be sent to WordPress.org.
php report.php
This system generates two files in which is the information of what will be sent as a response to the tests that are being analyzed. You can see the contents of the files easily.
cat /tmp/wp-test-runner/env.json
cat /tmp/wp-test-runner/junit.xml
The first of them includes the information of all the necessary elements for WordPress, such as PHP and its extensions and versions, and the database, with its version.
The second of the files is a summary of all the tests launched with their results and that, if they are sent to WordPress.org, those that have failed will be shown on the results page, in order to work on them, whether they are elements to be solved by the hosting, as if they are errors that affect development and the Core.
And, finally, once you have the system finished, we only have to clean the files and the database to be able to perform a next test at another time.
php cleanup.php
Creating your bot for WordPress.org
If you want the test results to appear on the WordPress.org page , you can create a user for it.
The first thing will be to create a user in WordPress.org. If your company is called, for example, Somos Hosting, S.L., you can call your user something like somoshostingbot
. Please note that the associated email account should be checked frequently, as emails will arrive regarding the possible operation of the tests.
Create an issue on the test page asking for the bot to be included on the results page as “Test Reporter request”, indicating the email account you have used with that user.
When the user has been created in the system you will be given access to a password so that you can configure that the system sends the information automatically. You will have to enter Users -> Your Profile and there generate the application password. Later you can modify the constant of the environment with something similar to export WPT_REPORT_API_KEY='somoshostingbot:ABCD 1234 abcd 4567 EFGH efgh'
.
Automating the bot
If you’re sending the information to WordPress.org you might want to automate the system to send reports when there’s news.
In this case we will use a series of scripts and systems for this automation, with the aim of reducing the number of executions, and only launch them when necessary.
Surely there are more optimized ways to perform this system, but this is the one I have tried and it works.
Have a copy of WordPress
cd /home/wordpressbot/
git clone https://github.com/WordPress/wordpress-develop.git wordpress-develop
cd /home/wordpressbot/wordpress-develop/
git pull
git log -n1 --format=format:"%H" > /home/wordpressbot/wordpress-develop.log
Inside our user’s folder we will make a copy of the WordPress repository that is used for development (and we will do a test to validate that it is downloaded and updated correctly).
Later (the last line) will generate a file in which the identifier of the last commit made will be included.
Control files
We will create two control files.
echo '' > /home/wordpressbot/wordpress-tested.log
echo 0 > /home/wordpressbot/wordpress-running.log
The first line creates a file without content in which we will save the version of the last test that has been launched.
The second line will save the binary state (0 or 1) to know if a test is running or not, as a traffic light.
Installing the runner
This is not necessary if you have done it before.
cd /home/wordpressbot/
git clone https://github.com/WordPress/phpunit-test-runner.git wordpress-test
cd /home/wordpressbot/wordpress-test/
git pull
We download the PHPUnit test runner, and validate that the updates are done well.
We configure the environment. To do this we will first make a copy of the sample file and then configure it.
cp .env.default .env
vim .env
The content (in summary form) can be something like this:
export WPT_PREPARE_DIR=/home/wordpressbot/wp-test-runner
export WPT_TEST_DIR=/home/wordpressbot/wp-test-runner
export WPT_REPORT_API_KEY=somoshostingbot:ABCD1234abcd4567EFGHefgh
export WPT_REPORT_URL=
export WPT_DB_NAME=wordpressdatabase
export WPT_DB_USER=wordpressusername
export WPT_DB_PASSWORD=wordpresspassword
export WPT_DB_HOST=localhost
export WPT_TABLE_PREFIX=${WPT_TABLE_PREFIX-wptests_}
export WPT_PHP_EXECUTABLE=${WPT_PHP_EXECUTABLE-php}
export WPT_PHPUNIT_CMD=
export WPT_RM_TEST_DIR_CMD=
export WPT_SSH_CONNECT=
export WPT_SSH_OPTIONS=
export WPT_SSH_PRIVATE_KEY_BASE64=
export WPT_DEBUG=
Unlike in the previous case, the configuration of the test path is also on the same user. In this way, when the tests are launched there will be 3 folders:
/home/wordpressbot/wordpress-develop/
: A copy of the WordPress development Git./home/wordpressbot/wordpress-test/
: A copy of the Git of the test tool./home/wordpressbot/wp-test-runner/
: WordPress and files where the tests will be carried out.
The script that runs everything
cat > /home/wordpressbot/wordpressbot.sh << EOF
#!/bin/bash
echo ''
echo '[*] Launching the WordPress testing tool for hosting.'
echo ''
echo '[+] Detecting changes.'
echo ''
cd /home/wordpressbot/wordpress-develop/
git pull
git log -n1 --format=format:"%H" > /home/wordpressbot/wordpress-develop.log
estado=`cat /home/wordpressbot/wordpress-running.log`
if [ "$estado" -eq 0 ]; then
echo ''
echo '[+] Preparing test.'
echo ''
cd /home/wordpressbot/wpsabot-test/
git pull
npm update
file1="/home/wordpressbot/wordpress-develop.log"
file2="/home/wordpressbot/wordpress-tested.log"
if cmp -s "$file1" "$file2"; then
echo ''
echo 'Nothing to do. The last test is the same.'
echo ''
else
echo ''
echo '[+] Starting test.'
echo ''
echo 1 > /home/wordpressbot/wordpress-running.log
cp /home/wordpressbot/wordpress-develop.log /home/wordpressbot/wordpress-tested.log
cd /home/wordpressbot/wpsabot-test/
source .env
php prepare.php
php test.php
php report.php
cat /home/wordpressbot/wp-test-runner/env.json
php cleanup.php
echo 0 > /home/wordpressbot/wordpress-running.log
fi
else
echo ''
echo '[+] Test already in progress.'
echo ''
fi
echo ''
echo '[*] The End.'
echo ''
EOF
First we will launch the file where we are going to save the script.
cat > /home/wordpressbot/wordpressbot.sh << EOF
We will download the latest wordpress development update. We will save, in the log file, the identifier of the last commit that has been executed.
cd /home/wordpressbot/wordpress-develop/
git pull
git log -n1 --format=format:"%H" > /home/wordpressbot/wordpress-develop.log
We’ll see if a test is running right now or not. In case a test is currently running, we won’t do anything (and wait for it to finish).
estado=`cat /home/wordpressbot/wordpress-running.log`
if [ "$estado" -eq 0 ]; then
...
else
echo ''
echo '[+] Test already in progress.'
echo ''
fi
If there is no test running, we will update the latest version of the test software.
echo ''
echo '[+] Preparing test.'
echo ''
cd /home/wordpressbot/wpsabot-test/
git pull
npm update
We will check if there are any updates in the development software with respect to the last test.
file1="/home/wordpressbot/wordpress-develop.log"
file2="/home/wordpressbot/wordpress-tested.log"
if cmp -s "$file1" "$file2"; then
...
fi
If the last commit of the development software is the same one on which we already did a test, we will not release it again.
echo ''
echo 'Nothing to do. The last test is the same.'
echo ''
If the test is new, then we will have to do the test. The first thing will be to set the configuration files. We will tell the system that we are running a test, and we will store the identifier of the test we are going to perform (commit).
echo ''
echo '[+] Starting test.'
echo ''
echo 1 > /home/wordpressbot/wordpress-running.log
cp /home/wordpressbot/wordpress-develop.log /home/wordpressbot/wordpress-tested.log
We will do the test.
cd /home/wordpressbot/wpsabot-test/
source .env
php prepare.php
php test.php
php report.php
cat /home/wordpressbot/wp-test-runner/env.json
php cleanup.php
And finally, we will tell the system that we have finished the test.
echo 0 > /home/wordpressbot/wordpress-running.log
And the test will be over.
echo ''
echo '[*] The End.'
echo ''
Now that we have the script, the first thing will be to launch it manually at least once to check that it works correctly.
bash /home/wordpressbot/wordpressbot.sh
Scheduling the tests (cron)
Now that we have a script that does it all, we can program a cron that, every minute, runs the entire system.
As the script has a traffic light system so as not to run more than once in parallel, it can be launched very frequently.
* * * * * bash /webs/wpsabot/wpsabot.sh >/dev/null 2>&1
Full tests
If you want to do a full test on a setup from scratch, you have a few options:
Improving configuration
Do not forget that the objective of this tool is the verification that our environment and infrastructure is the ooptima for WordPress to work, so, following the example of before, we could make several improvements such as the installation of the extension of bcmath
, , gd
, libsodium
, mod_xml
mcrypt
and imagick
or utilities such as ghostscript
and imagemagick
.
The goal? That there are no errors and have the green light for the perfect configuration.
WordPress Community Hosting Team
If you are a company that is somehow in the day to day of offering services related to WordPress I recommend that you remain attentive to the publications of the WordPress Community Hosting team, in addition to closely following the conversations of the Slack channel #hosting-community (see the meeting schedules of the team), where you can be in contact with those responsible for other hosting companies.
About this document
This document is regulated by the EUPL v1.2 license, published in WP SysAdmin and created by Javier Casares. Please, if you use this content in your website, your presentation or any material you distribute, remember to mention this site or its author, and having to put the material you create under EUPL license.