follow me on twitter

follow me on twitter.

Saturday, February 25, 2012

Navigating the web programmatically with Phantom.js

Phantom.js is a headless webkit browser which let's you navigate the web in a programmatic manner from the command line. A JavaScript API allows to control the browser for example to load a web page, check its contents or run JavaScript within the page. Obviously this is a powerful tool for many purposes. Running test cases comes to mind quikly.
I created a first example to load a qUnit test case, run the test and, after a short delay, grab the test result from the DOM elements: 


var page = new WebPage(),
    url = "http://localhost:3000/repo/test/model/model.qunit",
    delayInMillis = 200;

page.onLoadFinished = function (status) {
   
    window.setTimeout(function() {
       
        var result = page.evaluate(function () {
            return {
                passed: document.querySelector("#qunit-testresult .passed").innerText,
                failed: document.querySelector("#qunit-testresult .failed").innerText,
                total: document.querySelector("#qunit-testresult .total").innerText
            };
        });
       
        console.log(JSON.stringify(result));
        phantom.exit();
       
    }, delayInMillis);
};

page.open(url);

The sample creates a WebPage object and registers a listener for the loadFinished event. The page is loaded by simply call page.open(url). As soon as the page has been loaded completely the handler is called. To get the results of the qUnit test a functions is evaluated. This function is called within the page just loaded so it can access the DOM to get the test result from the DOM as rendered by qUnit. To make sure all tests have finished at the time the script evaluates the DOM, a timeout is used to delay the action.

Once phantom.js is on the PATH the still naive script above can be called from the command line:

# phantomjs load-qunit.js

The ability to inject JavaScripts into pages is very powerful. It makes integration testing of a web application quite easy as we can inject scripts to emulate user inputs triggering events. Afterwards the resulting action can be asserted by checking the changes in the DOM. This adds another possible testing layer above unit and component testing which could be done with qUnit alone. It's not cross browser, but it's kind of lightweight and fast and requires less setup then Selenium or JsTestDriver. A good tool well suited to be run on both a developer workstation or an integration test server.

While being headless (there is still a dependency to X on Linux; see work around) Phantom.js is still able to render pages into a bitmap. This makes it easy to create a detailed test report especially when something went wrong. A rendered picture of the failing page can support debugging and complete other data which can be drawn from the DOM in case of a failing test.

It's dead easy:

var page = new WebPage(),
    url = "http://localhost:3000/repo/test/model/model.qunit",
    delayInMillis = 200,

    screenshotFile = "failedTest.png"
    viewportWith = 1024,
    viewportHeight = 1000;

page.viewportSize = {
    width: viewportWith,
    height: viewportHeight
};


page.onLoadFinished = function (status) {
   
    window.setTimeout(function() {
       
        var result = page.evaluate(function () {
            return {
                passed: document.querySelector("#qunit-testresult .passed").innerText,
                failed: document.querySelector("#qunit-testresult .failed").innerText,
                total: document.querySelector("#qunit-testresult .total").innerText
            };
        });
       
        console.log(JSON.stringify(result));


        if (result.failed > 0) {
            page.render(screenshotFile);

        }

        phantom.exit();
       
    }, delayInMillis);
};

page.open(url);


 As demonstrated Phantom.js allows for automating many tasks done with a webbrowser. Not only testing but many other possible applications of Phantom.js can be very useful for web developers. It's obvious that testing with Phantom.js is not enough for the mainstream web as here cross browser testing is a must and solutions like Selenium are available.


Installation on OSX was as easy as downloading and extracting an archive file. Examples shipped with the distribution and the official documentation allowed for a quick jump start. So Phantom.js is another usefule webkit application which brings the web and it's technology out of the usual browser environement. Try it out!

Screenshot taken with Phantom.js after qUnit test has run:



Tuesday, February 14, 2012

Using jslint

JSLint and JSHint are tools which help to avoid bugs related to some JavaScript oddities. Linting my JavaScript code has become common for me and I found some issues in my code which I would not have found without it. It also helps refactoring code. For instance when moving code to a separate file, JSLint tells me about variables and functions I want to use in the script but are still in the old file. I don't have to run a test case to find the bug. The following option helps me out in this particular case:

undef: false #  true if variables and functions need not be declared before used.

On OSX I use Textmate for JavaScript code. JSLintMate is a bundle for Textmate which integrates JSLint and JSHint into the editor and fires it whenever I save a JavaScript file. So, customizing my lint options has become an issue quickly. At the end of this post you can find  my current JSLint options which are stored in the .jslinrc file in my home directory and hence are passed to JSLint by default.

Within a JavaScript file I can put additional instructions in comments. The jslint instruction can be used to pass options overriding or completing the default options:

/*jslint browser: true, devel: true */

The above tells JSLint to include global variables for the browser environment and allow calls to development tools like the console.

Other globals can be added by the global instruction:

/*global jQuery: true, craftjs: true */

This can be very useful to tell JSLint which globals are provided by third-party libraries as jQuery in the snippet above. Besides libraries there might also be globals created by my own scripts in other files which are linked into the page or prepended by a build tool like sprockets or craft.js.

Besides JSLint, JSHint becomes more and more popular. The mentioned craft.js I'm currently working on uses JSHint. Both roughly offer the same functionality. JSHint offers additional options and hence offers more control over the lint process. The JSLint instructions within the JavaScript comments are supported by JSHint as well. That's why JSLint and JSHint can be used  side by side without cluttering the source code even more.

Conclusion

Being used of using IDE features for languages like Java makes a JavaScript developer feel lonely. Not only can JavaScript IDEs not offer such a rich feature set like Java IDEs can. JavaScript even does not have a compiler which tells a developer about typos and other mistakes. JSLint/JSHint can help to some extend by providing instant feedback when writing JavaScript code and supports the endevour of delivering high-quality JavaScript code.

my current jslint default options

adsafe: true # true if ADsafe rules should be enforced. See http://www.ADsafe.org/
bitwise: false # true if bitwise operators should be allowed.
cap: false # true if upper case HTML should be allowed
css: false # true if CSS workarounds should be tolerated
debug: false # true if debugger statements should be allowed (set to false before going into production)
eqeq: false # true if the == and != operators should be tolerated.
es5: false # true if ECMAScript 5 syntax should be allowed
evil: false # true if eval should be allowed
forin: false # true if unfiltered 'for in' statements should be allowed
fragment: false # true if HTML fragments should be allowed
indent: 4 # Number of spaces that should be used for indentation - used only if 'white' option is set
maxerr: 50 # The maximum number of warnings reported (per file)
maxlen: 120 # Maximum line length
newcap: false # true if Initial Caps with constructor functions is optional.
nomen: true # true if names should not be checked for initial or trailing underbars.
on: false # true if HTML event handlers (e.g. onclick="...") should be allowed
passfail: false # true if the scan should stop on first error (per file)
plusplus: true # true if ++ and -- should be allowed
regexp: false # true if . and [^...] should be allowed in RegExp literals.
safe: false # true if the safe subset rules are enforced (used by ADsafe)
sloppy: true # true if the ES5 'use strict'; pragma is not required
sub: false # true if subscript notation may be used for expressions better expressed in dot notation
undef: false #  true if variables and functions need not be declared before used.
vars: false # true if multiple var statement per function should be allowed.
white: false # true if strict whitespace rules should be ignored.

predef: '' # Names of predefined global variables - comma-separated string or a YAML array

browser: false # true if the standard browser globals should be predefined
rhino: false # true if the Rhino environment globals should be predefined
windows: false # true if Windows-specific globals should be predefined
widget: false # true if the Yahoo Widgets globals should be predefined
devel: true # true if functions like alert, confirm, console, prompt etc. are predefined
node: false # true if the node.js environment globals should be predefined


Thursday, February 2, 2012

Leveraging node.js for unit testing in JavaScript development

I played around with node.js to find a way to properly unit test my JavaScripts. While doing that I stumbled upon a nice feature node.js offers by the vm module. The function

vm.runInNewContext(script, globalObject, filename);

allows to run a script and pass a context to it. This context is nothing else than the evil global object of JavaScript we all know about and work around. So the above code allows us to run a script like the one below which is stored in test.js and has two dependencies to the global object:

(function () {
    "use strict";
   
    var renderer = require("./renderer");
   
    assert.ok(renderer);
}());

By using runInNewContext we can now load the test.js script and run it with an empty context. This is done by test-runner.js:


(function () {
    "use strict";
    var vm = require("vm"), fs = require("fs"),
        filename = "app/test.js",
        testScript = fs.readFileSync(filename).toString(),
        globalObject = {};

    vm.runInNewContext(testScript, globalObject, filename);
}());

Now we can run the test runner with node:

#  node test-runner.js

To get an error saying:

ReferenceError: require is not defined

This is exactly what we expected because the globalObject passed to runInNewContext is empty and hence the context has no require to offer as we are used to in a node.js environment. However, being able to pass our own global object, we can mock dependencies the test requires. This is exactly what we need to mock away our collaboraters and unit test our modules in an isolated manner. In Java, libraries like EasyMock do some magic to provide mocks for interfaces and classes. A great and very useful tool for test driven development in Java. In JavaScript mocking a function is a snap compared to that. The dynamic type system and the functional approach with late binding makes it easy to mock a function or an entire object with literals. Hence the test-runner.js script needs to setup the globalObject properly to provide an appropriate global object on which test.js can rely on:

(function () {
    "use strict";
    var vm = require("vm"), fs = require("fs"),
        filename = "app/test.js",
        globalObject = {
          require: function(module) {
            return {};
          }

        },
        testScript = fs.readFileSync(filename).toString();

    vm.runInNewContext(testScript, globalObject, filename);
}());

The above script now offers a require function as a property of the global object. It simply returns an empty object which is enough to fulfill our test case. Now our test script fails a liitle bit later because it misses the assert function to check the imported module. In this case we don't want to reimplement the assert function node.js already offers. Instead we just pass the real implementation:

globalObject = {
     assert: require("assert"),
     require: function(module) {
            return {};
     }
}

Running the test runner passes now without complaint, because we successfully mocked all dependencies for our test case.

The exampled I showed here is very simplicistic. But it clearly shows the opportunity the runInNewContext function of the vm module offers for unit testing JavaScript code. We can not only mock the global require function, but also other global properties as for instance the document or window properties in a browser or a jQuery Ajax call $.ajax({}).

While it realy sounds like a big burden to mock away an entire browser environment, the impact of such an development style on JavaScript code would be huge and raise not only quality and testability but also leads to flexible reusable modules, objects and functions. While I have not applied such an development approach for JavaScript by now, my experiences with TDD in Java makes me sure this would be a good idea for JavaScript as well. Not having Eclipse and its great refactorings for Java (a kindom for Ctrl+1!!) makes it not that easy and powerful, but the benefits make it worth doing it.

As shown above, node.js in general and vm.runInContext in particular offer a nice starting point for such an endeavour.