Skip to main content

Using FlexUnit for Stress Testing

I saw quite a few questions in the forums on how to stress
test a Flex application. I thought about it and came up with an idea
that I want to share here.




I think FlexUnit can be used for stress testing. It is not
that difficult. I simply add multiple test runners for each client
application and run all of them asynchronously. Here is the example of
the FlexUnit runner:




<?xml version="1.0" encoding="utf-8"?>
<mx:Application xmlns:mx="http://www.adobe.com/2006/mxml" xmlns="*"
xmlns:flexunit="flexunit.flexui.*"
creationComplete="onCreationComplete()">

<mx:Script>
<![CDATA[
import flexunit.framework.TestSuite;
import test.TemperatureConverterTest;
import test.ArrayUtilTest;
import mx.collections.ArrayCollection;
import flexunit.flexui.TestRunnerBase;

[Bindable]
public var testClients:ArrayCollection;

public var NUMBER_OF_TESTS:int = 100;

private function onCreationComplete():void
{
var clients:Array = new Array();
var i:int;
for (i = 0; i < NUMBER_OF_TESTS; i++) {
clients.push("test"+i);
}
testClients = new ArrayCollection(clients);
}

// Creates the test suite to run
private function createSuite():TestSuite {
var ts:TestSuite = new TestSuite();

ts.addTest( TemperatureConverterTest.suite() );
ts.addTest( ArrayUtilTest.suite() );

return ts;
}

private function startTests():void {
trace(" elements: " + testRepeater.numChildren);
var tests:Array = this.test as Array;
for each (var testRunner:TestRunnerBase in tests) {
testRunner.test = createSuite();
testRunner.startTest();
}
}

]]>
</mx:Script>

<mx:Button name="Run" label="Start Tests" click="startTests()" />

<mx:Panel layout="vertical" width="100%">
<mx:Repeater id="testRepeater" dataProvider="{testClients}">
<flexunit:TestRunnerBase id="test" width="100%" height="100%" />
</mx:Repeater>

</mx:Panel>
</mx:Application>






Here is a screen shot of the test with 100 test runners:





So, I guess once you have a client application that runs
hundreds of tests in parallel, it is easy to launch it on several
instances of the browser, or even on several PCs. All you need to do is
to add startTests() call to onCreationComplete() as a last
line of the method, deploy the app and simply open the URL in the
browsers.


The other nice thing about it is that this test client can be
used for profiling purposes. FlexBuilder has a very nice profiler. I
found it very helpful in testing my BlazeDS application.

Comments

Very interesting - and I have two questions:

1. I'd be interested in reading more details on what the unit tests you are running do. I'm thinking that if I were to use this approach it wouldn't be for actual "unit" testing - i.e. testing relatively small pieces of code - instead I'd be doing something more extensive - for example testing calls across the wire to my DB via BlazeDS. Is this what you do?

2. I'm assuming that when you talk about using the profiler, you mean that you're running the profiler when you're simultaneously running all the tests. Is this correct?

Thanks,

Douglas

PS I'm linking to this at my "testing-debugging-agile-methods" page: http://www.brightworks.com/technology/adobe_flex/testing_debugging_agile_methods.html
Mykola said…
Hi Douglas, Thank you for your comment. The example test cases in this post are pure unit tests. However, I agree with you, FlexUnit is a great tool for integration tests as well. I added a new post today that describes testing BlazeDS data services with FlexUnit. Here is a link: http://mdzyuba.blogspot.com/2008/05/testing-remote-data-services-with.html

The FlexUnit test runner is a regular Flex application. I.e. it can be profiled as any other Flex application. So, it is correct, I am running the profiler when I run tests. The profiling reveals more problems when the application is under load.

Thanks for the link to your site! It has lots of info on testing and tools.
Thanks for the response, Mykola, and for the new article. Interesting stuff. BTW, are you aware of dpUint (http://code.google.com/p/dpuint/wiki/Introduction)?
Hi!

Thanks for sharing about your way making stress testing.

I really like the way your post looks! Nice presentation of your thoughts!

CheerzZ!
Anonymous said…
I have a question about the FlexUnit running in Flash. It is my understanding (and please correct me if I'm wrong) that the FlashPlayer only has 1 thread. So wouldn't all your test cases actually be from 1 thread? So the load you can handle might be much different then what you're actually showing here?
Anonymous said…
Can u pls add the code for ArrayUtilTest too . Since i am new to FLEXUNIT i am not able to point out
how to write that one .

import test.ArrayUtilTest;
ts.addTest( ArrayUtilTest.suite() );
Mykola said…
package test
{
import flexunit.framework.TestCase;
import flexunit.framework.TestSuite;
import mx.utils.ArrayUtil;


public class ArrayUtilTest extends TestCase
{
public function ArrayUtilTest(methodName:String)
{
super( methodName );
}

public static function suite():TestSuite {
var ts:TestSuite = new TestSuite();

ts.addTest( new ArrayUtilTest( "testToArray" ) );
ts.addTest( new ArrayUtilTest( "testToArrayArray" ) );
ts.addTest( new ArrayUtilTest( "testToArrayNull" ) );
return ts;
}

/**
* If the Object is already an Array, it returns the object.
*/
public function testToArrayArray():void {
var o:Array = new Array();
o.push("This is a test");
o.push("Some string");
var result:Array = ArrayUtil.toArray(o);
assertTrue(result != null);
assertTrue(result.length == 2);
assertTrue(result[0] == o[0]);
assertTrue(result[1] == o[1]);
}

/**
* If the Object is already an Array, it returns the object.
* If the object is not an Array, it returns an Array in which the only element is the Object.
* As a special case, if the Object is null, it returns an empty Array.
*/
public function testToArray():void {
var o:Object = new String("Hello world");
var result:Array = ArrayUtil.toArray(o);
assertTrue(result != null);
assertTrue(result.length == 1);
assertTrue(result[0] == o);
}

/**
* As a special case, if the Object is null, it returns an empty Array.
*/
public function testToArrayNull():void {
var o:Object = null;
var result:Array = ArrayUtil.toArray(o);
assertTrue(result != null);
assertTrue(result.length == 0);
}

}
}

Popular posts from this blog

cucumber-jvm and Android Studio

During this Winter holidays, I've been exploring a Behavior Driven Development and experimented with a couple of testing frameworks: Calabash and cucumber-jvm.

In this post, I would like to describe setting up cucumber-jvm for Android Studio projects.

I've extended cucumber-jvm examples with a version for Android Studio. Here is a copy in my github branch: https://github.com/mdzyuba/cucumber-jvm/tree/add_android_studio_example/examples/android/android-studio/Cukeulator.

In addition to the example code, here is some details on how it was created.

Dependencies
The cucumber-jvm requires several libraries to be added to a project: https://github.com/cucumber/cucumber-jvm/tree/master/android

I've found it's easier to add those dependencies with the Android Studio project setup dialog because it takes care of the jar versions for you.

File > Project Structure > Modules > app > Dependencies > + > Library dependency
  > type "cucumber" in the s…

Flash on Google Sites

This is a brief how-to add a Flash on a Google Site.

Google Sites allows adding Flash embedded into a Google Gadget. I uploaded a swf file to my Google Site - FlexWebTools and found a gadget that allows displaying any SWF on a Google Site page. It is called AnyFlash. It is pretty easy to use and it worked fine in my Firefox 3.5.5 on Mac OS X 10.5.8. Then I send a link to my site to my friends and they discovered that the SWF was not displayed properly on IE6 and in Firefox on Linux. I reviewed AnyFlash source code and eventually found that there is a bug 939 in the Google Gadget API. Therefore, I created a new gadget that does not use the gadgets.flash.embedFlash() method and works well with Firefox, Safari and IE6 browsers on Mac, Windows and Linux.

While working on the gadget, I found a few different ways of embedding Flash into a gadget. Some of them are pretty straightforward like simply pointing to a swf in the context tag. Some are more sophisticated. For instance, they would …

How to debug SOAP on Mac with tcpdump

I’ve been using several tools to debug SOAP on Mac and Windows. I would like to share my experience in using some of the tools and show some examples.

So far I found four major categories of tools that are useful in debugging SOAP:
Interface listeners such as tcpdump and othersProxy tools such as TCPMonitorServlet filtersApplication server logging. For example, Weblogic has a special logging option that dumps SOAP requests and responses to the sever log.
All of these tools have their own advantages and disadvantages. In this article, I will describe the first type - an interface listener tool called tcpdump. I am planning to describe other three categories in the future posts.

The tcpdump is a tool that sniffs IP traffic and dumps it to a file or a standard output stream. It was originally developed by Van Jacobson, Craig Leres and Steven McCanne from the Lawrence Berkeley National Laboratory, University of California, Berkeley, CA. It is open source. The tcpdump documentation and sou…