Context Navigation

<< Home

Using Windmill IDE

The Windmill IDE is the graphical tool you can use to create and edit tests, monitor output and interact with Javascript in the browser.

Starting the IDE

windmill firefox http://www.example.com

(If you are on a Mac and prefer Safari, replace firefox above with safari)

You should then see Firefox launch and Windmill load.

When Windmill is done loading you should be presented with two windows, one containing the application you want to test ( http://www.example.com) and the Windmill IDE, shown below.

Recording Tests

The record feature lets you quickly build your test.

Navigate to the starting page of the site you would like to test.
In the IDE, press the record button (farthest to the left of the circular buttons in the top right of the IDE). The icon inside should change from a circle to a square, and the focus will be moved to the main browsing window.
Do your testing! (Navigate around custom essay the page as you normally would.)
When you're done, click the record button in the IDE again. The square will change back to a circle.
- Waits: You will need to add these when you go to a new page, or call an ajax request, to make sure that the page is ready for the next step in the list.
- Open: If you manually type an address into the URL bar, this won't be recorded, use the open action.
- Assertions: To test that the operations you've recorded do what they were supposed to.

Editing Tests

The editor is simply a graphical representation of a list of actions to execute against your test page. (You can also directly edit the test files.)
The drop down fields for a specific action allow you to provide the information your action needs.
The up and down arrows on each action will insert another action above or below that step.
The trash can will delete that action, without asking -- so be careful!

Playing the Tests

The play icon at the top will play all the actions in the IDE, starting at the top.
The play icon on each step will start playing from that step and cascade until there are no more tests left to run. (This can be changed in the settings.)
Steps go green/red as they are called and pass/fail.

Saving Tests

Click the save button next to a suite and it will be exported to Python markup in a popup window (you can select other markups like JavaScript in the IDE's settings panel). Use the browser's "save page" feature to save this to disk for later playback.

DOM Explorer

The DOM explorer allows you to navigate and explore the DOM in your testing application, and update actions in the IDE to target the selected node (by clicking the node).

Start the DOM Explorer by click in the button in the top right of the IDE with an X (second from the right).
Your testing window should now be in focus, move your mouse around the page and you should see the element you are hovering over.
If you have an action in the IDE the actions fields should be updated on the fly as you move your mouse around the page, until you find the node you want and click. This should turn the explorer off and give focus back the the remote for you to do any editing you would like.

Assertion Tool

The Assertion Tool is designed to allow you to quickly create asserts on the fly. This works initially like the DOM Explorer but has a very different use.

Turn on the Assertion Tool by clicking the far right icon in the top right with the letter A on it.
Your testing window should now have focus and allow you to move your mouse around the page displaying in the remote the node you are currently over.
When you find the node you would like to assert, you can click on it.
Your focus should now be back in the remote, and actions should have been generated to validate the state of the node you clicked on when you run this test in the future.

Firebug Lite Integration

Giving you the power of Firebug in every browser you are working in, the utility is endless.

Output & Performance Tabs

As you are working in the Windmill IDE all possible output is sent to the output tab allowing you to keep an eye on everything happening in Windmill. Anyone developing new functions or extensions for Windmill should remember to send all the possible output they can as well as any performance information to this tab.

Settings Dialog

There are some very helpful preferences available to be configured in the settings tab, customize them for your test building pleasure.

System

Throw JavaScript errors, full debug: Currently we capture JavaScript errors so we can display them to you in the output tab, however if you would like them thrown so they can be caught in firebug or another error console.

Break on Failure: When you are running a tests and you encounter a failure, you can configure if the whole test run stops or not.

Suppress Javascript Alerts (Can be done with the reWriteAlerts controller function): When implemented will capture all Javascript alerts in the testing window.
Details: http://trac.getwindmill.com/wiki/ControllerApi

Suite Save Formatting: Allows you to specify the format you would like generated when you click to save a suite.

IDE/Recorder

Auto Scroll IDE to bottom: When tests are running in the IDE you can decide if you want the continually be moved to see the latest action, or not.

Absolute click sensitivity when recording: When you are recording a test in your testing window many actions like doubleclick also constitute two single clicks, then the firing of the double click event. When this is selected you are recording every single event, if you would like a cleaner test disable this. However, if you find that the recorder is missing simple clicks on elements you would like to click then you should turn this back on.. temporarily anyway.

Action play button cascades: When you have a set of tests in the IDE you would like to play, the play button for each action starts playback there and cascades until you are out of actions to run. If you would like to step through the actions one at a time, turn this option off and it will only run the single action to which you clicked play.

Use XPath only for explorer/recorder: If you find yourself testing an application with multiple elements that share the same ID, or have a dynamic ID that isn't reliable, you may want to record this action using XPath. If this action has an ID, the recorder will use that unless you turn this option on.

Record only absolute XPath's (vs relative): This includes div ID's etc. in the XPath, is a bit more robust and descriptive, but hasn't proven to be 100% cross browser compliant.

Play action by hitting 'return/enter': Instead of having to click the play button, hitting return from within the text inputs in the action will play it.

Python Tests

Windmill provides an implementation of the controller that is used for windmill tests using functest.

# Generated by the windmill services transformer
from windmill.authoring import WindmillTestClient

def test():
    client = WindmillTestClient(__name__)

    client.click(id=u'viewNavCenterRight')
    client.waits.sleep(milliseconds=2000)
    client.doubleClick(id=u'hourDiv1-1200')
    client.waits.sleep(milliseconds=2000)
    client.type(text=u'Properties Test', id=u'noteTitle')
    client.type(text=u'9:00', id=u'startTime')
    client.type(text=u'4:00', id=u'endTime')
    client.radio(id=u'startMeridianAM')
    client.radio(id=u'endMeridianPM')
    client.select(id=u'eventStatus', option=u'Tentative')
    client.type(text=u'A description for the properties test', id=u'noteDescription')

By default WindmillTestClient will throw an assertion whenever a test has a failed result. This triggers a failure for that test method in functest.

You can turn off these assertions and do the assertion yourself. This is useful if you want to assert that a test failed.

from windmill.authoring import WindmillTestClient

def test():
    client = WindmillTestClient(__name__, assertions=False)

    client.waits.forElement(xpath=u'html/body/center/p/font', timeout=u'3000')
    assert not client.asserts.assertValue(validator=u'Google Search', name=u'btnG')['result']

Notice that the object returned from client.waits.forElement is not a simple boolean, it's actually the entire result object returned from the windmill IDE. It includes the starttime, endtime, and other debugging information about the test call.

Passing Variables from the command line

Anything passed to windmill that isn't a used argument is stuck into the functest registry, which you can then access from within Python inside your tests.

from windmill.authoring import WindmillTestClient
import functest

def setup_module(module):
    client = WindmillTestClient(__name__)
    
    client.click(id=u'email')
    client.type(text=functest.registry['email'], id=u'email')
    client.type(text=functest.registry['password'], id=u'pass')
    client.click(value=u"Login")
    client.waits.forPageLoad(timeout=u'60000')

functest.registry['email'] corresponds to the command line parameter you are passing in and will be substituted for the value when the test is run.

Example test run of the above:

windmill firefox http://www.facebook.com email=abc123@slide.com password=something

JavaScript Tests

JavaScript tests in Windmill consist of a directory tree full of .js files containing your tests. On the server-side, Windmill just recursively parses the directory tree and serves up the files inside. When you run your tests, Windmill evals the contents of your test .js files in the window scope of the application being tested. This just means that if you have someNamespace in your app, you can access it directly in your JavaScript tests without doing anything special.

The JavaScript test engine can call the same set of UI commands as Python tests, and also allows you to use all the same asserts available in both of the major JSUnit testing frameworks (see "Asserts," below).

Test Setup and Parsing

Windmill will search recursively through the tested applications's window object and register any objects that have a name beginning with "test_" ('test' plus underscore), "setup", or "teardown."

Test Phases

The "setup"/"test_"/"teardown" naming determines what phase the tests belong to: within the same level, tests named "setup" are run first, followed by any named "test_", followed by any named "teardown." This allows you to create a complex hierarchy of tests, and make sure that whatever setup is needed for a specific test happens exactly the same way every time.

When you're running tests, there's also a way to filter them down to just the ones in a particular phase (see "Test-Writing Workflow," below). This saves a lot of time when you're developing tests, by cutting out redundant steps, and can help in debugging by allowing you to home in on a single phase (e.g., "Why won't the stupid teardown work?").

Organizing Your Tests

The require command can be used in your JavaScript tests to share code between modules, or to resolve order-of-loading issues. windmill.jsTest.require forces the framework to load and eval a file immediately. You call it like this:

 windmill.jsTest.require('shared/test_login.js');

The path for the require is always relative to the location of the top-level directory of JavaScript tests you're running -- NOT to the file where you're doing the require. (This is a tradeoff -- it's not very flexible, but we don't have to think about how to map JavaScript namespaces to directory and file names.)

The require method works recursively -- you can require files that require other files. The framework will only eval the code in a particular file once during a test run, no matter how many times you require it.

The initialize.js file

Windmill JavaScript tests gives you one other important tool for the order-of-loading problem: before loading, evaling, and parsing test files, it looks for a file named initialize.js in the root directory of your tests, and runs the code in it first, before doing anything with any other files in your test directory.

This can be useful for setting up shortcuts to use in your tests, or creating namespaces for holding your test objects temporarily until you're ready to hang them on your test namespace hierarchy.

An initialize.js file might have stuff in it like this:

var BASE_PATH = '/some_path'
var ACCOUNT_PREFS = myApp.base.getPrefs();
var USERNAME = fleegix.cookie.get('username');
var LOGOUT_REDIRECT = 'http://subdomain.mydomain.org/thanks';

var fooNamespace = {};
var barNamespace = {};

Basically, just remember that any code in initialize.js will run before anything else in your test code.

Running Your Tests in a Specific Order

If you just drop a bunch of .js files in a test directory, and run your tests, there is NO GUARANTEE OF THE ORDER in which your files will be evaled and parsed.

The JavaScript test engine lets you specify the order you want your tests to run in, by calling the register command in your initialize.js file, and passing it an array of essay writing assistance the test namespaces you want to run, in your desired order, like so:

// Use this to set the order you want your tests to run
var registeredTests = [
  'test_jsonDom',
  'test_waitForXHR',
  'test_formBasics',
  'test_jumBasics',
  'test_scope',
  'test_timer'
];
 
// Register top-level test namespaces in the order
// we want to run them
windmill.jsTest.register(registeredTests);

The tests inside the namespace will run in the order you define them -- except for any setup or teardown, which run first and last, respectively.

Test Formats

JavaScript tests come in in two flavors:

Executable functions
Windmill controller API commands objects

You can write these tests to run individually, or you can put into an array and run them in aggregate. (You can also organize tests into namespaces -- see "Namespacing Tests," below.)

Single functions

Single-function tests are just JavaScript functions executed in the window scope of the application you're testing.

Here are a couple of examples:

function test_fooThing () {
  var foo = 'asdf';
  jum.assertEquals(foo, 'asdf');
  var bar = 'qwer';
  jum.assertNotEquals(bar, 'asdf');
}

var test_getData = function () {
  // Callback function -- sets temp flag to 'returned'
  var success = function (s) {
    var data = eval('('+s+')');
    myTestInitData.tempData = data;
  };
  var url = windmillMain.shared.util.getCurrentDir() + 'data.json';
  fleegix.xhr.get(success, url);
};

Controller API Command Objects

All the actions in the standard Windmill controller API are available from the JavaScript test framework.

The big difference in the JavaScript tests from Python tests is that there is no server round-trip for each action -- the JavaScript test engine simply passes the actions directly to the controller object, so test execution is generally faster. However this also means that reporting on success/failure doesn't occur until the entire test run completes.

Asserts

var test_albumNodeExists = { 
  method: "asserts.assertNode", 
  params: {
    id: 'album_9'
  } 
};
var test_albumNodeText =  {
  method: "asserts.assertText",
  params: {
    id: 'album_9',
    validator: 'Power Windows'
  }
};

Waits

test_hasNavigated = {
  method: "waits.forElement",
  params: { 
    id: "formPageHeader"
  } 
};

Warning: keep in mind that note that code containing these kinds of UI control objects is eval'd inline when the code for that .js test file is loaded. You can't reference objects in your command objects that don't exist on the page when the test code is eval'd.

Calling actions directly from JavaScript

Note that the UI command-object methods are also available as JavaScript method calls on the windmill.jsTest.actions object. This can be useful if you have to do some kind of dynamic lookup of a UI element, or need to drive pieces of UI that aren't on the page on initial load.

Here's an example:

var test_bazThing = function () {
  var nodeId = getSomeDomNodeDynamically();
  windmill.jsTest.actions.click({ id: nodeId });
}

The special "waits.forJS"

This is a wait that depends on the result of arbitrary JavaScript returning true. The JavaScript passed can be either a string, or a JavaScript function. Pass the string or function as the js property of the params object.

Here's an example:

test_TempData = { 
  method: "waits.forJS",
  params: { 
    js: function () { 
      return !!windmillMain.tempData;
    }
  }   
};

Grouping tests in arrays

If you have a bunch of actions or functions you want to run in aggregate, you can put them all into an array and the JavaScript test engine will execute them in sequence, but report on their passing/failure as a group. (On failure, execution of the items in the array stops, the failure is reported, and the test engine moves to the next test.)

var test_arrayBarThing = [
  // Click a button
  { method: "click",
    params: {
      id: "fooButton"
    }
  },
  // Wait around a bit for an element on the new page
  { method: "waits.forElement",
    id: {
      'zoomieNode'
    }
  },
  // Verify the Ajaxy update to the page happened
  function () {
    var node = document.getElementById('zoomieNode')
    jum.assertText(node, 'Howdy');
  }
];

JSUnit-Compatible Asserts

Windmill's JavaScript test framework has the full range of JSUnit-compatible asserts, accessible inside your JavaScript test functions from the "jum" object. Here are the currently available asserts:

assertTrue
assertFalse
assertEquals
assertNotEquals
assertLessThan
assertMoreThan
assertNull
assertNotNull
assertUndefined
assertNotUndefined
assertNaN
assertNotNaN
assertEvaluatesToTrue
assertEvaluatesToFalse
assertContains

The syntax is precisely the same as JSUnit, allowing an optional comment as a first parameter:

jum.assertNotNaN('Totally not a number!', 'Geddy Lee');
=> (Totally not a number!) assertNotNaN -- <Geddy Lee> (String) expected to be a number but was not a number (NaN).
jum.assertNotNaN('Neil Peart');
=> assertNotNaN -- <Neil Peart> (String) expected to be a number but was not a number (NaN).

Namespacing Tests

Windmill JavaScript tests can also be organized into namespaces, just like any other JavaScript code. Namespacing, along with setup/teardown can be used to create sophisticated trees of tests.

Here's a basic example to give you an idea of what a namespaced test object might look like. The namespace object has a name beginning with "test_", so the JavaScript test parser knows it's a container for tests:

windmill.jsTest.require('shared.js');

test_jsonDom = new function () {
  // Navigate to the main page
  this.setup = windmillMain.shared.test_navMain;

  this.test_getData = function () {
    // Callback function -- sets temp flag to 'returned'
    var success = function (s) {
      var data = eval('('+s+')');
      windmillMain.tempData = data;
    };
    var url = windmillMain.shared.util.getCurrentDir() + 'data.json';
    fleegix.xhr.get(success, url);
  };
  this.test_wait = windmillMain.shared.util.waitForTempData;
  this.test_verifyData = function () {
    var data = windmillMain.tempData;
    var albums = data.albums;
    jum.assertNotNaN(albums.length);
  };
  this.test_writeDom = function () {
    var data = windmillMain.tempData;
    var albums = data.albums;
    var c = $('content');
    for (var i = 0; i < albums.length; i++) {
      var d = $elem('div', {
        id: 'album_' + i,
        innerHTML: albums[i]
      });
      c.appendChild(d);
    }
    jum.assertTrue(c.hasChildNodes());
  };
  this.test_domNodesActionStyle = [
    { method: "asserts.assertNode", params: {id: 'album_9'} },
    { method: "asserts.assertText", params: {id: 'album_9',
      validator: 'Power Windows'} }
  ];
  this.teardown = function () {
    // Clear out the temp data
    windmillMain.tempData = null;
    // Clear the DOM
    $('content').innerHTML = '';
  };
};

The unit tests for the JavaScript test engine provide a lot of examples of namespaced JavaScript tests.

Running JavaScript Tests

To run your JavaScript tests you can use one of the following methods.

From the Python shell:

run_js_tests('dir/to/js/tests')

From the command line:

windmill firefox http://yourdomain.com jsdir=./js/test/dir

To exit the browser after the tests, include the exit option:

windmill firefox http://yourdomain.com jsdir=./js/test/dir exit

Developing Tests -- Running Specific Tests or Phases

When you're working on a test, you don't want to have to run the setup prerequisites for your new test over and over while you're trying to debug it. The JavaScript test engine lets you run only specific tests, or only specific phases (e.g., 'setup,' 'test,' or 'teardown') of a test.

Running only a specific test

To limit to a specific test, in the Windmill Python shell, add the desired test as a second parameter, like so:

run_js_tests("./fleegix_js/tests", 'test_fleegixEvent.test_foo')

Add the ns: prefix to limit your run to all the tests in a specific namespace:

run_js_tests("./fleegix_js/tests", 'ns:test_fleegixEvent')

Note: When you're working with a hierarchy of namespaced tests, setup and teardown phase for a test or test namespace include the setup and teardown for parent namespaces of the test.

Running only a specific phase of your test

To limit your test run to a specific phase, pass the desired phase (or phases in a comma-delimited string), as a third parameter to the run_js_tests command. This comes in handy when you're working on a specific test, and don't want to go through a full run of setup, test, and teardown every time.

First, get the test ready to run, by running just the setup:

run_js_tests("./fleegix_js/tests", 'ns:test_fleegixXhr', 'setup')

Then, run your the test only, as many times as you need:

run_js_tests("./fleegix_js/tests", 'ns:test_fleegixXhr', 'test')

When you want to run the whole thing, run the teardown once, and re-run the whole shebang:

run_js_tests("./fleegix_js/tests", 'ns:test_fleegixXhr', 'teardown')
run_js_tests("./fleegix_js/tests", 'ns:test_fleegixXhr', 'setup, test, teardown')

Directly from the command line

Limiting to a specific test or phase can also be done when running tests from the command line using the jsphase and jsfilter options, like so:

windmill firefox http://yourdomain.com jsdir=./js/test/dir jsfilter=ns:test_fleegixXhr jsphase=setup

More Advanced Demo Videos

Download in other formats:

Plain Text