Writing Unit Tests For WordPress Plugins

Advertisement

When my WordPress plugin had only three users, it didn’t matter much if I broke it. By the time I reached 100,000 downloads, every new update made my palms sweat.

My first goal for the WordPress Editorial Calendar1 was to make it do anything useful. I was new to JavaScript and PHP and didn’t really know what I could pull off. In a few days I had a proof of concept. In a few more I had a working version and was asking friends to install it. The calendar worked… sort of.

I spent three times as much time fixing bugs as I did coding. Once the plugin worked, I wrote unit tests to make sure it kept working.

The unit tests for my calendar use QUnit2, but they really use just three functions: test, expect and ok. This article shows you how to integrate unit tests into your WordPress plugin, where to write unit tests and how they can help you.

QUnit Basics

Unit tests follow a basic pattern: do something, then check the results. (“Is this variable 4 when it should be 5?” “Does this line of my table show up where it’s supposed to?”)

function myTest() {
    test('My test run', function() {
        expect(2);
        ok(5 === 5, 'This test will always pass');
        ok(5 === 6, 'This test will always fail');
    });
}

This structure forces you to think of your code in simple units that return true or false. The test function starts a test run with two arguments: the title for this test run, and the function containing the tests. The expect function tells QUnit how many tests are in the run. If we call too few or too many, it causes an error.

The ok function performs the test of the expression. It takes two arguments: a boolean indicating whether the test was successful, and a message.

Test runs are added to a special list section, which shows the total number of tests, whether each test passed or failed, and how long the tests took.

A Real Unit Test

WordPress plugins complicate testing by using a combination of PHP and JavaScript. Even passing a simple value from PHP to JavaScript gets tricky.

The function below finds a WordPress preference with the get_option3 function and creates a JavaScript variable with the resulting value.

function getOption($myOption, $myVariable) {
    if (get_option($myOption) != "") {
        ?><script type="text/javascript">
            <?php echo($myVariable); ?> = <?php echo(get_option($myOption)); ?>;
        <php
    }
}

Now we’ll call it to get the name of the blog and set it to a variable named myBlogName.

getOption("blogname", "myBlogName");

Little helper functions like these glue your plugin together, but they always worry me. Did I access the JavaScript variable with the right name or did I mistype it? Did I use the same name twice? A simple unit test makes all of these worries go away.

function wpPrefTest() {
    test('Get access to WordPress preferences', function() {
        expect(1);
        ok(myBlogName, 'The variable (myBlogName) should be available.');
    });
}

This unit test checks whether the variable myBlogName exists. We could also look for a specific value or compare it to something else from the application.

Once you have this unit test, you’ll never need to worry about getting the blog’s name. It will always be there, and you’ll find out fast if you ever break it.

Integrating QUnit With WordPress

Testing in special development environments isn’t accurate. I wanted to add QUnit directly to my calendar, but I didn’t want to make the page’s size larger. The solution is a URL parameter and a little PHP to include QUnit only when I need it:

wp_enqueue_script( "qunit", path_join(WP_PLUGIN_URL, basename( dirname( __FILE__ ) )."/lib/qunit.js"), array( 'jquery' ) );
wp_enqueue_script( "edcal-test", path_join(WP_PLUGIN_URL, basename( dirname( __FILE__ ) )."/edcal_test.js"), array( 'jquery' ) );

This tells WordPress to include the QUnit JavaScript and my unit tests from edcal_test.js4. I could have just embedded the script’s reference directly on my page, but I might have run into trouble if other plugins on the same page use QUnit.

You can see the full source code here5.

The next step was to make sure that these scripts load only when I need them. To do this, I wrapped the code in a check for a URL parameter:

if ($_GET['qunit']) {
    wp_enqueue_script( "qunit", path_join(WP_PLUGIN_URL, basename( dirname( __FILE__ ) )."/lib/qunit.js"), array( 'jquery' ) );
    wp_enqueue_script( "edcal-test", path_join(WP_PLUGIN_URL, basename( dirname( __FILE__ ) )."/edcal_test.js"), array( 'jquery' ) );
}

This loads the scripts only if I’m running unit tests, and everything else in the plugin stays the same. You can run the unit tests at any time just by adding &qunit=true to the end of the URL. That’s a good thing because my unit tests actually change what’s going on in the blog.

You can run the Editorial Calendar’s unit tests in your browser right now6. Scroll down to see the unit test results at the bottom of the page.

The PHP makes sure that my scripts get to the browser. The last step is to call them from my JavaScript. Once again, I want to call them only if we’re in unit test mode. So, I add a small function to get the parameters from the URL:

getUrlVars: function() {
    var vars = [], hash;
    var hashes = window.location.href.slice(window.location.href.indexOf('?') + 1).split('&');
    for (var i = 0; i < hashes.length; i++) {
        hash = hashes[i].split('=');
        vars.push(hash[0]);
        vars[hash[0]] = hash[1];
    }
    return vars;
}

And then I call my unit tests if the QUnit parameter is there:

jQuery(document).ready(function() {
    if (edcal.getUrlVars().qunit) {
        edcal_test.runTests();
    }
});

This ensures that we call the unit tests only if they’re available.

The last step is to set up a space for the unit test’s output. QUnit writes its test results into specially designated tags on your HTML page. You could embed these tags directly in your HTML output, but because they need to be there only when QUnit is active, I create the HTML in JavaScript instead.

jQuery('head').append('<link>');
css = jQuery('head').children(':last');
css.attr({
    rel: 'stylesheet',
    type: 'text/css',
    href: '../wp-content/plugins/edcal/lib/qunit.css'
});

jQuery('#wpbody-content .wrap').append('<div id="edcal-qunit"></div>');

jQuery('#edcal-qunit').append(
    '<h1 id="qunit-header">WordPress Editorial Calendar Unit Tests</h1>' +
    '<h2 id="qunit-banner"></h2>' +
    '<div id="qunit-testrunner-toolbar"></div>' +
    '<h2 id="qunit-userAgent"></h2>' +
    '<ol id="qunit-tests"></ol>' +
    '<div id="qunit-fixture">test markup</div>');

QUnit needs a list tag, a couple of divs and a style sheet.

7

Now we’re ready to write our first test.

The First Unit Test

The first calendar unit tests scroll the calendar up and down and make sure we see the correct number of days.

moveTests: function() {
    var curSunday = edcal.nextStartOfWeek(Date.today()).add(-1).weeks();
    edcal.moveTo(Date.today());

    test('Move to today and check visible dates', function() {
        expect(2);
        ok(edcal_test.getFirstDate().equals(curSunday.clone()),
           'firstDate should match ' + curSunday);
        ok(edcal_test.getLastDate().equals(
           curSunday.clone().add(edcal.weeksPref).weeks().add(-1).days()),
           'lastDate should match ' + curSunday);
    });
}

Our first test moves the calendar to today and checks to see if the first and last days are what we expect. We set up a variable, move the calendar and start the test by calling the test function.

In this case we want to make sure the dates are correct, so we compare the date from the calendar to the one we expect and then pass the result to the ok function. The test succeeds if they match and fails if they don’t.

This test may seem simple, but a lot is going on under the hood. We’re testing date handling, drawing and the fundamental arithmetic of the calendar.

Unit tests can do anything. The WordPress Editorial Calendar unit tests8 automate the plugin like a robot. They cover everything a user could do with the calendar.

What To Unit Test

I write a lot more unit tests for JavaScript than I do for compiled languages. In Java and C++, the compiler catches many of my mistakes, whereas JavaScript lets you pass a string when you meant to pass a number and lets you call a function with two arguments when it needs three.

Here’s the basic list of areas I test in JavaScript applications:

  • Tests that make sure the program does what it is meant to do
    These tests ensure the basic functionality keeps working; they do not test interactions. (The calendar lets you drag and drop posts, but writing unit tests for dragging and dropping wouldn’t make sense; instead, we would focus on what happens after the drop event.)
  • Tests that make sure the program doesn’t do what it’s not meant to do
    These tests ensure the program fails properly when it gets garbage.
  • A unit test for every major bug I’ve found
    These tests ensure that none of these bugs creep back in.

APIs and other clear boundaries in the code lend themselves well to unit tests, as do utility functions that you call from many places in the application. With the calendar, this means testing calendar movements and testing how we create and modify posts in WordPress, like this:

  1. Move the calendar and check the dates;
  2. Create a post and make sure it was created properly;
  3. Edit the post we just created and make sure it saves properly;
  4. Move the post and make sure it shows up at the correct date;
  5. Move the post from two places at the same time and make sure we get the proper warning message;
  6. Delete the post from the server and make sure it isn’t there anymore.

The penultimate test covers an error condition in which two people move the same post at the same time. Unit tests work well for this kind of error, because the error type is difficult to test and manual testing is less likely to uncover problems.

For your own application, you should have at least one unit test for every data-changing operation that users can perform. I like to add them for all of the places where a user can get data, too. It might sound like a lot of work, but you can cut down on it by writing single tests that cover multiple areas.

Asynchronous Unit Tests

Many of these combination unit tests make AJAX calls. QUnit provides a special function for handling AJAX called asyncTest. This function works just like test, but it pauses the test run at the end of the function. The QUnit framework waits until your AJAX call completes and you call the start function before restarting the test run.

The asyncTest function handles all of the tests that edit posts from the calendar, including deleting the post at the end:

asyncTest('Delete the post created for testing', function() {
    expect(1);

    edcal.deletePost(edcal_test.post.id, function(res)
    {
        equals(jQuery('#post-' + res.post.id).length, 0,
               'The post should now be deleted from the calendar.');
        start();
    });
});

When you restart the test framework you can call more tests. Calling the next test function at the end of the previous one chains them together, and it supports calling all of your tests with just a call to the first function.

These tests, which call AJAX, ensure that the integration between the JavaScript on the client side and the PHP on the back end works properly.

That’s Not a Unit Test

When I first learned to write unit tests in C++ the rule was this: a single test should only call code in a single module or CPP file. That is, a unit test should test one unit of code.

Changing posts from unit tests violates that rule. Instead of just testing JavaScript, I’m really testing JavaScript, PHP, WordPress itself and MySQL all at once. That makes it an automated integration test.

Integration tests aren’t traditional unit tests, but they work well for WordPress plugins. When I create a post, I’ll know that the AJAX code in my plugin works as well as the JavaScript code. Covering a larger portion of the application with fewer tests makes it easier to focus on what I should be testing.

What Not To Unit Test

You could write unit tests forever, but some are more useful than others. Here are some guidelines.

  • Don’t unit test the UI.
    The test has to run by itself. It can’t wait for you to click a button or look at something to make sure it appears correctly.
  • Don’t unit test performance.
    Tests take a variable amount of time on different machines and browsers. Don’t write unit tests that depend on a function being returned in a set amount of time.
  • Don’t unit test code from other projects.
    Adding unit tests for WordPress or a jQuery plugin you depend on might be tempting, but it rarely pays off. If you want to contribute unit tests to WordPress.org that’s great, but your unit tests should check that your plugin works.

The Editorial Calendar has 26 unit tests, at about 3,500 lines of code. That may not sound like much, but they’ve saved many of my releases.

Unit Tests Saved My Butt

I didn’t add unit tests until the thirteenth release of my plugin. By then, the calendar had a couple of hundred users and was growing fast. My plugin was working, and I was getting close to release 1.0.

Instead of adding new features, I took in a new framework, added special code to load it, wrote 381 lines of unit tests and integrated all of this into the plugin. It seems like a lot of work, but it saved my butt.

Right before a release, I wrote some harmless-looking PHP code like the following to get the JSON data that represented a set of posts to show up in the calendar:

function edcal_postJSON($post) {
    setup_postdata($post);
    ?>
    {
        "date" : "<?php the_time('d') ?><?php the_time('m') ?><?php the_time('Y') ?>",
        "title" : <?php echo($this->edcal_json_encode(get_the_title())); ?>,
        "author" : <?php echo($this->edcal_json_encode(get_the_author())); ?>
    },
    <?php
}

function edcal_posts() {
    header("Content-Type: application/json");

    global $post;
    $args = array(
        'posts_per_page' => -1,
        'post_status' => "publish&future&draft",
        'post_parent' => null // any parent
    );

    $myposts = query_posts($args);

    ?>[
        <?php
        $size = sizeof($myposts);
        for($i = 0; $i < $size; $i++) {
            $post = $myposts[$i];
            edcal_postJSON($post, $i < $size - 1);
        }
    ?> ]
    <?php
}

I ran the code and everything worked. I was about to release the new version but ran my unit tests first to make sure. They failed. Can you spot the bug? I didn’t.

I was returning a JSON array, but the last item in the array had a trailing comma. That’s invalid JSON. Firefox accepts it, but Safari, Chrome and IE don’t. I almost shipped a broken plugin to over half of my users.

Now I run the unit tests on all major browsers whenever I release a new version. Any time WordPress releases a new version, I run the unit tests. WordPress 3.3 broke the calendar — and I found out exactly why in 15 seconds.

Popular Plugins Are Stable Plugins

Most WordPress plugins are free and open source, but free doesn’t always mean cheap. The total cost of ownership9 of unstable plugins is more than people would pay. That’s a fancy way of saying that users will run from your plugin if it’s a bug fest.

My plugin became popular because of its features, but stability kept it popular. People remember one buggy release for a long time. If the Editorial Calendar deletes or corrupts posts from just one blog, thousands of people would stop using it. And they’d be justified.

Unit tests add the stability you need to support the multitude of browsers, mobile devices and dark corners involved in any JavaScript application. They’re easy to write, and they pay you back: because you find the bugs, and your users don’t.

(al)

Footnotes

  1. 1 http://wordpress.org/extend/plugins/editorial-calendar/
  2. 2 http://docs.jquery.com/QUnit
  3. 3 http://codex.wordpress.org/Function_Reference/get_option
  4. 4 http://plugins.svn.wordpress.org/editorial-calendar/trunk/edcal_test.js
  5. 5 http://plugins.svn.wordpress.org/editorial-calendar/trunk/
  6. 6 http://www.zackgrossbart.com/extras/sandbox/wp-admin/edit.php?page=cal&qunit=true
  7. 7 http://www.zackgrossbart.com/extras/sandbox/wp-admin/edit.php?page=cal&qunit=true
  8. 8 http://www.zackgrossbart.com/extras/sandbox/wp-admin/edit.php?page=cal&qunit=true
  9. 9 http://en.wikipedia.org/wiki/Total_cost_of_ownership

↑ Back to topShare on Twitter

Zack Grossbart is an engineer, designer, and author. He's a founding member of the Spiffy UI project, the architect of the WordPress Editorial Calendar, and a Consulting Engineer with NetIQ. Zack began loading DOS from a floppy disk when he was five years old. He first worked professionally with computers when he was 15 and started his first software company when he was 16. Zack lives in Cambridge, Massachusetts with his wife and daughter.

Advertising

Note: Our rating-system has caused errors, so it's disabled at the moment. It will be back the moment the problem has been resolved. We're very sorry. Happy Holidays!

  1. 1

    Thanks for your insight Zack, a brilliant post.

  2. 2

    Very nice article.

    This could be a nice base to understand and write PHPUnit tests.

    I have a question, why do you include the tests in the plugin that goes public? You could run the tests on development/stage version and if they pass, build a production version of the plugin without the tests included.

    • 3

      Including the tests for the public version has two big advantages. The first is that I minimize differences between my staging and production version and make the tests more valid. The second is that I can have users of the plugin run the tests on their blogs when I’m supporting the plugin and helping them debug an issue.

  3. 4

    Forgive my ignorance, but why is this necessary?

    If you write your code with compatibility in mind (using functions that work across multiple versions of PHP, jQuery, etc.) and anonymize everything, shouldn’t it all just gel together as expected?

    • 5

      Damian Gostomski

      March 8, 2012 2:12 am

      Firstly, I’d like to say what a great article. Unit testing is something I’ve known I should do for a long time, but never really got round to it. Hopefully this is the push I’ve needed, especially as I now have 2 plugins in the WP repository

      As for why this is necessary, I see two main reasons:
      1. Regression testing – By writing a test, you no longer need to manually check each change doesn’t break previous working functionality. Let say a few months down the line, you change a variable name in a JSON object, or remove a property, and you don’t think it’s used anywhere else, a unit test will flag that error up.
      2. It covers mistakes that will slip through the human mind, such as in the example shown towards the end. It’s incredibly easy to make a mistake like that in a dynamic language, and even easier if it’s a browser specific one. Again, unit test help to automated the process of finding these, instead of relying on a manual testing process, which is not only slower, but more error prone

  4. 6

    I get the idea here but this seems like an awful lot of work for something that’s already been dealt with for both PHP and JavaScript.

    First, I’m confused as to why you’re outputting so much JavaScript directly like that. Use wp_localize_script. Then make sure you have xdebug on the server (wamp or mamp or xampp all have it), and turn on error reporting when you’re on your development server. If you write this with wp_localize_script then all your problems will be apparent in the PHP. It would have been much easier to avoid your JSON mistake by building the same object in PHP and passing it to the localize script function. If you had an extra comma you’d get an error.

    If you have ill formatted JavaScript it will show in any browser console. And using break points and stepping through the code will certainly kill any JavaScript bugs you have.

    And don’t take this as an attack on the author but If you’re a JAVA dev you should be well versed in debugging with breakpoints. These tools exists for JavaScript too!

    So while this is a good article on the importance of testing I argue that there are better ways to go about doing it. But, of course I’d argue that! What developer doesn’t argue that there’s always a better way.

  5. 7

    I use wp-unit, which is itself a plugin, to run tests and make them visible in the WordPress admin area.

  6. 8

    You should probably be using wp_localize_script to pass data to javascript

  7. 9

    Great article. Thanks for the insight!

  8. 10

    I’m not sure I like this or encourage it. When you are doing test-driven development, you’ll usually test every function and class you have. It’s advisable that you have a separation between the JavaScript and the PHP code. That is, write tests for the PHP code and write test for the JavaScript code. There are dedicated Unit Testing libraries for both.

    This will help also when you want to know the coverage of unit testing on your PHP/JavaScript code. For the User Interface, there is Selenium. It should test that your interface works as it should.

    Also, as Jonny has mentioned, you should use wp_localize_script to pass variable to your script.

    Still a nice way to put tests when you are in a hurry.

  9. 11

    Thanks to all of the people who mentioned wp_localize_script. It’s tricky to come up with a simple case of a bug since most bugs are complicated or you wouldn’t have them.

    There are always new functions to learn about and better ways to get something done, but that’s why unit tests are so useful. None of us are perfect.

  10. 12

    In dynamic typed languages unit tests are even more important, as they also check for missing variables.

    For everyone interested in writing tests for their plugin PHP part they should look for.It’s not as nicely integrated as in other frameworks such as Django, but after initial hassle it can be used for plugin testing.

    E.g. I emptied the whole wp-testcase folder (as I only want to test my plugins) and chip in a following test_myplugins.php:

    foreach (glob("wordpress/wp-content/plugins/*/tests/test_*.php") as $plugin_test) {
    include $plugin_test;
    }

    It’s really simple way to gather tests within your plugin directories.

↑ Back to top