ISBN: 0-471-41405 -0
Download (direct link):
Expensive and hard to test on; might not be available when the application is being developed; exact scenarios not repeatable
the area where the developer is located. Even when the network is available, it is difficult to use the network for more than a final proof-of-concept test. With a real network, you cannot repeat the exact test scenario two times because of the many factors that affect the performance of a real network. Even if you had your own network in a laboratory, the radio waves travel very different paths if your position is just slightly different. Add to that factor the number of other users on a typical commercial network, and you will find that it is crucial to first optimize the application by testing it on an emulator and then verifying the performance on real networks.
With an emulator, it is easy to record a test scenario and then repeat that scenario later in order to determine whether the developer has managed to improve the performance. You can easily add advanced features such as interruptions, different operator settings, and high -end handsets to the tests in order to make sure that the application is robust and ready for a variety of networks (and not just for one or two networks that have associated handsets).
To summarize, you should perform the testing mostly by using emulators, but you should always verify the testing on real networks and real devices. Now, lets look more in detail at how we can perform these tests.
GUI and Usability Testing___________________________________________________________________________________________
While the majority of developers perform their GUI and usability testing in their own labs, some public test sites are emerging. This section describes some of the basic factors to consider when performing these tests (regardless of who creates them).
Although many quickly find a favorite emulator or a favorite device, you must be open to the needs of consumers. The more testing that you perform on a variety of emulators, the lower the risk of losing entire market segments that comes from the application being incompatible with some devices. The time consumption of such tests is, however, a significant obstacle in the stressed development process. We can remedy this concern by choosing a main development platform with an associated emulator and then performing tests over most available emulators at certain milestones in the development process. The style guides that usually accompany the emulators are a great help with getting the most from the platform in question. On the CD-ROM and Web site that accompanies this book, there are a number of good emulators available that cover a range of devices and that you can use as a starting point. All of the key players in the device and operating system business are keen on having cutting -edge developer tools and should always be important bookmarks for the dedicated developer.
While you are making the application actually work on the desired devices, you should know that this phase of the testing should also include basic usability tests. This testing is especially important for browser-based applications such as WAP. While users commonly use the Web browser for surfing, the mobile Internet involves straight-to-the-point information. How much information can the user access within 10 seconds on your site? When you think about such things, you must picture the user as someone who is totally unfamiliar with the application and not the experienced tester or developer within your company. A useful exercise is to draw a site map that shows the tree of the decks in the application, as shown in Figure 14.2. In the site map, we show the percent of the test users who access a certain deck.
Figure 14.2 illustrates the WAP site of a small movie theater. The main deck gives the user three choices: book tickets, look at movie information, and check out information about the theater. In this example, we can see how the current movies are fairly far down in the hierarchy, but lots of users still access them. As a result, the average user has to perform many navigational steps in order to reach the desired information. Here, it would be easy to move this current.wml deck to a higher level by making it a choice in the main deck, as shown in Figure 14.3. While this example is pretty small, it makes a lot of difference if we can make this kind of change to larger sites.
Figure 14.2 Example site map for a WAP application.
Figure 14.3 Site map for a modified WAP application.
In the modified site, we have moved the deck that shows the current movies to a higher level; consequently, it catches a lot of hits. This kind of testing improves the usability as well as the overall perceived performance.
We should investigate some generic GUI aspects, such as the following:
How much navigation does the user typically have to go through? Mobile users are more inpatient than their fixed-Internet counterparts.