Monday, May 19, 2008

Using FlexUnit for Stress Testing

I saw quite a few questions in the forums on how to stress
test a Flex application. I thought about it and came up with an idea
that I want to share here.




I think FlexUnit can be used for stress testing. It is not
that difficult. I simply add multiple test runners for each client
application and run all of them asynchronously. Here is the example of
the FlexUnit runner:




<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" xmlns="*"
xmlns:flexunit="flexunit.flexui.*"
creationComplete="onCreationComplete()">

<mx:Script>
<![CDATA[
import flexunit.framework.TestSuite;
import test.TemperatureConverterTest;
import test.ArrayUtilTest;
import mx.collections.ArrayCollection;
import flexunit.flexui.TestRunnerBase;

[Bindable]
public var testClients:ArrayCollection;

public var NUMBER_OF_TESTS:int = 100;

private function onCreationComplete():void
{
var clients:Array = new Array();
var i:int;
for (i = 0; i < NUMBER_OF_TESTS; i++) {
clients.push("test"+i);
}
testClients = new ArrayCollection(clients);
}

// Creates the test suite to run
private function createSuite():TestSuite {
var ts:TestSuite = new TestSuite();

ts.addTest( TemperatureConverterTest.suite() );
ts.addTest( ArrayUtilTest.suite() );

return ts;
}

private function startTests():void {
trace(" elements: " + testRepeater.numChildren);
var tests:Array = this.test as Array;
for each (var testRunner:TestRunnerBase in tests) {
testRunner.test = createSuite();
testRunner.startTest();
}
}

]]>
</mx:Script>

<mx:Button name="Run" label="Start Tests" click="startTests()" />

<mx:Panel layout="vertical" width="100%">
<mx:Repeater id="testRepeater" dataProvider="{testClients}">
<flexunit:TestRunnerBase id="test" width="100%" height="100%" />
</mx:Repeater>

</mx:Panel>
</mx:Application>






Here is a screen shot of the test with 100 test runners:





So, I guess once you have a client application that runs
hundreds of tests in parallel, it is easy to launch it on several
instances of the browser, or even on several PCs. All you need to do is
to add startTests() call to onCreationComplete() as a last
line of the method, deploy the app and simply open the URL in the
browsers.


The other nice thing about it is that this test client can be
used for profiling purposes. FlexBuilder has a very nice profiler. I
found it very helpful in testing my BlazeDS application.

7 comments:

Douglas McCarroll said...

Very interesting - and I have two questions:

1. I'd be interested in reading more details on what the unit tests you are running do. I'm thinking that if I were to use this approach it wouldn't be for actual "unit" testing - i.e. testing relatively small pieces of code - instead I'd be doing something more extensive - for example testing calls across the wire to my DB via BlazeDS. Is this what you do?

2. I'm assuming that when you talk about using the profiler, you mean that you're running the profiler when you're simultaneously running all the tests. Is this correct?

Thanks,

Douglas

PS I'm linking to this at my "testing-debugging-agile-methods" page: http://www.brightworks.com/technology/adobe_flex/testing_debugging_agile_methods.html

Mykola said...

Hi Douglas, Thank you for your comment. The example test cases in this post are pure unit tests. However, I agree with you, FlexUnit is a great tool for integration tests as well. I added a new post today that describes testing BlazeDS data services with FlexUnit. Here is a link: http://mdzyuba.blogspot.com/2008/05/testing-remote-data-services-with.html

The FlexUnit test runner is a regular Flex application. I.e. it can be profiled as any other Flex application. So, it is correct, I am running the profiler when I run tests. The profiling reveals more problems when the application is under load.

Thanks for the link to your site! It has lots of info on testing and tools.

Douglas McCarroll said...

Thanks for the response, Mykola, and for the new article. Interesting stuff. BTW, are you aware of dpUint (http://code.google.com/p/dpuint/wiki/Introduction)?

Jean-Christophe Turin said...

Hi!

Thanks for sharing about your way making stress testing.

I really like the way your post looks! Nice presentation of your thoughts!

CheerzZ!

Anonymous said...

I have a question about the FlexUnit running in Flash. It is my understanding (and please correct me if I'm wrong) that the FlashPlayer only has 1 thread. So wouldn't all your test cases actually be from 1 thread? So the load you can handle might be much different then what you're actually showing here?

Anonymous said...

Can u pls add the code for ArrayUtilTest too . Since i am new to FLEXUNIT i am not able to point out
how to write that one .

import test.ArrayUtilTest;
ts.addTest( ArrayUtilTest.suite() );

Mykola said...

package test
{
import flexunit.framework.TestCase;
import flexunit.framework.TestSuite;
import mx.utils.ArrayUtil;


public class ArrayUtilTest extends TestCase
{
public function ArrayUtilTest(methodName:String)
{
super( methodName );
}

public static function suite():TestSuite {
var ts:TestSuite = new TestSuite();

ts.addTest( new ArrayUtilTest( "testToArray" ) );
ts.addTest( new ArrayUtilTest( "testToArrayArray" ) );
ts.addTest( new ArrayUtilTest( "testToArrayNull" ) );
return ts;
}

/**
* If the Object is already an Array, it returns the object.
*/
public function testToArrayArray():void {
var o:Array = new Array();
o.push("This is a test");
o.push("Some string");
var result:Array = ArrayUtil.toArray(o);
assertTrue(result != null);
assertTrue(result.length == 2);
assertTrue(result[0] == o[0]);
assertTrue(result[1] == o[1]);
}

/**
* If the Object is already an Array, it returns the object.
* If the object is not an Array, it returns an Array in which the only element is the Object.
* As a special case, if the Object is null, it returns an empty Array.
*/
public function testToArray():void {
var o:Object = new String("Hello world");
var result:Array = ArrayUtil.toArray(o);
assertTrue(result != null);
assertTrue(result.length == 1);
assertTrue(result[0] == o);
}

/**
* As a special case, if the Object is null, it returns an empty Array.
*/
public function testToArrayNull():void {
var o:Object = null;
var result:Array = ArrayUtil.toArray(o);
assertTrue(result != null);
assertTrue(result.length == 0);
}

}
}