Navigating the web programmatically with Phantom.js

Phantom.js is a headless webkit browser which let's you navigate the web in a programmatic manner from the command line. A JavaScript API allows to control the browser for example to load a web page, check its contents or run JavaScript within the page. Obviously this is a powerful tool for many purposes. Running test cases comes to mind quikly.
I created a first example to load a qUnit test case, run the test and, after a short delay, grab the test result from the DOM elements: 

var page = new WebPage(),
    url = "http://localhost:3000/repo/test/model/model.qunit",
    delayInMillis = 200;

page.onLoadFinished = function (status) {
   
    window.setTimeout(function() {
       
        var result = page.evaluate(function () {
            return {
                passed: document.querySelector("#qunit-testresult .passed").innerText,
                failed: document.querySelector("#qunit-testresult .failed").innerText,
                total: document.querySelector("#qunit-testresult .total").innerText
            };
        });
       
        console.log(JSON.stringify(result));
        phantom.exit();
       
    }, delayInMillis);
};

page.open(url);

The sample creates a WebPage object and registers a listener for the loadFinished event. The page is loaded by simply call page.open(url). As soon as the page has been loaded completely the handler is called. To get the results of the qUnit test a functions is evaluated. This function is called within the page just loaded so it can access the DOM to get the test result from the DOM as rendered by qUnit. To make sure all tests have finished at the time the script evaluates the DOM, a timeout is used to delay the action.

Once phantom.js is on the PATH the still naive script above can be called from the command line:

# phantomjs load-qunit.js

The ability to inject JavaScripts into pages is very powerful. It makes integration testing of a web application quite easy as we can inject scripts to emulate user inputs triggering events. Afterwards the resulting action can be asserted by checking the changes in the DOM. This adds another possible testing layer above unit and component testing which could be done with qUnit alone. It's not cross browser, but it's kind of lightweight and fast and requires less setup then Selenium or JsTestDriver. A good tool well suited to be run on both a developer workstation or an integration test server.

While being headless (there is still a dependency to X on Linux; see work around) Phantom.js is still able to render pages into a bitmap. This makes it easy to create a detailed test report especially when something went wrong. A rendered picture of the failing page can support debugging and complete other data which can be drawn from the DOM in case of a failing test.

It's dead easy:

var page = new WebPage(),
    url = "http://localhost:3000/repo/test/model/model.qunit",
    delayInMillis = 200,
    screenshotFile = "failedTest.png"
    viewportWith = 1024,
    viewportHeight = 1000;

page.viewportSize = {
    width: viewportWith,
    height: viewportHeight
};

page.onLoadFinished = function (status) {
   
    window.setTimeout(function() {
       
        var result = page.evaluate(function () {
            return {
                passed: document.querySelector("#qunit-testresult .passed").innerText,
                failed: document.querySelector("#qunit-testresult .failed").innerText,
                total: document.querySelector("#qunit-testresult .total").innerText
            };
        });
       
        console.log(JSON.stringify(result));

        if (result.failed > 0) {
            page.render(screenshotFile);
        }

        phantom.exit();
       
    }, delayInMillis);
};

page.open(url);

 As demonstrated Phantom.js allows for automating many tasks done with a webbrowser. Not only testing but many other possible applications of Phantom.js can be very useful for web developers. It's obvious that testing with Phantom.js is not enough for the mainstream web as here cross browser testing is a must and solutions like Selenium are available.


Installation on OSX was as easy as downloading and extracting an archive file. Examples shipped with the distribution and the official documentation allowed for a quick jump start. So Phantom.js is another usefule webkit application which brings the web and it's technology out of the usual browser environement. Try it out!

Screenshot taken with Phantom.js after qUnit test has run: