Testing Flash and Flex Applications
I was on a call today with a client and their hosting vendor discussing the finer technical points of a new Flex 2/Flash 9 project that we are working on and the topic of load testing came up. The hosting vendor talked about load testing Flash as if it was something that was always done. I never really gave it much though before, normally I would just load test the server-side services and any other code that my Flash/Flex applications would connect to and figured that was all that could be done. Besides, does load testing even apply to code running in a browser? Where does the load come from if there is only one user? Where I work we would call this monkey clicking if anything. Monkey clicking is where the user clicks much faster than the developer would expect them to. Most sites built with Flash can be broken this way and is considered by my team the pinnacle of ActionScript development - to have a site that can survive monkey clicking.
So what are all the ways that a Flash/Flex application can be tested? First and foremost there is good old QA. There is no better substitute for a good QA a person, especially one that really understands how to properly test a Flash or Flex application. We were lucky enough to find someone who not only had previous experience QA'ing Flash, but was also an ex-game tester. Video games actually have a lot in common with Flash interfaces, something to consider when hiring a new QA person. The QA persons job is to test for monkey clicking, making sure links go to the right place, forms work properly with validation and tabbing, animations are doing what they are supposed to and so on.
Second is unit testing. Wikipedia describes the benefits of unit testing as: The goal of unit testing is to isolate each part of the program and show that the individual parts are correct. A unit test provides a strict, written contract that the piece of code must satisfy. As a result, it affords several benefits. This is a great way to go, especially if you use a class library in all of your projects. By using reusable code that has been throughly tested, you effectively reduce the amount of QA needed for any project that uses your library because you can have very reliable nut and bolts to build with. If you don't have your own application and/or architectural framework to build upon there are many to choose from, such as Cairngorm and As2lib.
Third is load or stress testing. There are many tools that can be used to test your PHP, .NET, JAVA or whatever your using, but what about ActionScript? A good load testing program will "record" a users actions and then play them back over and over to simulate multiple users at once. This might work for an HTML based site, maybe Silverlight will do better with this, but Flash is somewhat of a black box in this department. So far I have not found anything to simulate a user within a Flash application. Sure I could write a special testing application in ActionScript to make calls to the server in the same way that my application does, but is this cost effective? I think the problem lies in the fact that every Flash application is created with it's own custom logic, unlike HTML where everything is built with the same components. Links, buttons and forms will always work the same in HTML, but not true in Flash - maybe Flex. There is a load testing framework for Flex, but it mostly concentrates on load testing Flex Data Services and can be found on Adobe Labs. Even this uses a special application to simulate the calls to the server and not the actual application you created.
As in most interactive marketing based work, especially those that are built in Flash, things like QA, unit testing and load testing don't get done or gets very little time because of poor scoping, budget, project delays or little understanding for the importance of testing. It's a constant fight for us developers, sometimes I think we're the only ones who consider what we do is actually software development and not just brochureware. My previous boss (who was the GM of our office) actually ridiculed me in front of all the senior managers in our office because I tried to make this point, which ended up being the beginning of the end for me at that job. I realized that my methodologies on web development would never be taken seriously, hence I would not get the time and resources to do the job right, so I left and got a new job. Too bad for them, she was pissed when I left. I guess that makes my point as clear as possible.
So what are all the ways that a Flash/Flex application can be tested? First and foremost there is good old QA. There is no better substitute for a good QA a person, especially one that really understands how to properly test a Flash or Flex application. We were lucky enough to find someone who not only had previous experience QA'ing Flash, but was also an ex-game tester. Video games actually have a lot in common with Flash interfaces, something to consider when hiring a new QA person. The QA persons job is to test for monkey clicking, making sure links go to the right place, forms work properly with validation and tabbing, animations are doing what they are supposed to and so on.
Second is unit testing. Wikipedia describes the benefits of unit testing as: The goal of unit testing is to isolate each part of the program and show that the individual parts are correct. A unit test provides a strict, written contract that the piece of code must satisfy. As a result, it affords several benefits. This is a great way to go, especially if you use a class library in all of your projects. By using reusable code that has been throughly tested, you effectively reduce the amount of QA needed for any project that uses your library because you can have very reliable nut and bolts to build with. If you don't have your own application and/or architectural framework to build upon there are many to choose from, such as Cairngorm and As2lib.
Third is load or stress testing. There are many tools that can be used to test your PHP, .NET, JAVA or whatever your using, but what about ActionScript? A good load testing program will "record" a users actions and then play them back over and over to simulate multiple users at once. This might work for an HTML based site, maybe Silverlight will do better with this, but Flash is somewhat of a black box in this department. So far I have not found anything to simulate a user within a Flash application. Sure I could write a special testing application in ActionScript to make calls to the server in the same way that my application does, but is this cost effective? I think the problem lies in the fact that every Flash application is created with it's own custom logic, unlike HTML where everything is built with the same components. Links, buttons and forms will always work the same in HTML, but not true in Flash - maybe Flex. There is a load testing framework for Flex, but it mostly concentrates on load testing Flex Data Services and can be found on Adobe Labs. Even this uses a special application to simulate the calls to the server and not the actual application you created.
As in most interactive marketing based work, especially those that are built in Flash, things like QA, unit testing and load testing don't get done or gets very little time because of poor scoping, budget, project delays or little understanding for the importance of testing. It's a constant fight for us developers, sometimes I think we're the only ones who consider what we do is actually software development and not just brochureware. My previous boss (who was the GM of our office) actually ridiculed me in front of all the senior managers in our office because I tried to make this point, which ended up being the beginning of the end for me at that job. I realized that my methodologies on web development would never be taken seriously, hence I would not get the time and resources to do the job right, so I left and got a new job. Too bad for them, she was pissed when I left. I guess that makes my point as clear as possible.
