Testing International Software

Testing International Software

Testing International Software

I have several friends who are STEs (Software Test Engineers) who will take exception to the placement of this chapter at the end, wondering why I am choosing to send testers to the "back of the bus," as it were. There are two very good reasons for this:

• Testing is obviously something that is usually done after a product is developed rather than before, which makes putting this chapter at the end sensible.
• The job of the tester overlaps so much of the information contained in this book that if I had not separated it there would have been a million "note to test:" references throughout the book.

This chapter, therefore, will serve as a central place that someone who is a tester but who has never had to handle i18N issues before might not want to skip. If you are a developer, you should also take a look, just in case your testers decide to buy this book and you want to keep down your bug count!

The best place to start is in deciding what to test.

Planning Out What to Test?

My first bullet point up above is a little glib because testing does not have to wait until the product is done; it starts at the beginning as the tester reviews specifications and makes estimates as to how long it will take to test the product that is being described. This is a crucial factor in determining what features should be cut and perhaps entertained for future versions when there is more time. This section discusses the issues to keep in mind.

There are many different configurations that tests can be run under, and I have tried to summarize them in Table 16.1.

Table 16.1 The Test Environments to Consider

Category	Operating System	Settings	Hardware
1	NT/Win2K	Regional Settings set to other locales	U.S./Any
2	NT/Win2K	Category 1 plus locale keyboard installed	U.S./Any
3	Win2K	Category 2 plus changed default system language	U.S./Any
4	All	Fully localized OS (on Win2K, swap language via MUI)	U.S./Any
5	All	Fully localized OS	Locale specific

Obviously, testing with the Category 5 in Table 16.1 will find the largest number of bugs related to i18N, but it is often not practical to have to install multiple language versions of the operating systems to do this. There are three points you will want to keep in mind:

• Testing under Windows 2000 is the smartest and most cost-effective way to test the behavior under your other application because you can change regional options and default system language so easily. You can further extend the ability to compare language with Windows 2000 terminal services.
• As you move through the four options in Table 16.1, the amount of work it will take to test the product will increase significantly (tenfold or more by the time you reach Category 5).
• The number of bugs you find that you would not have found with one of the configurations given earlier in the table will decrease significantly. (Typically, less than 1% of all defects that might exist are found under Category 5.)

With these points made, I will now look at each category individually. The decision for how much to cover is often not up to a tester directly, but the tester's input on how much work will be required to cover the product can be vital, so the information must be carefully considered and balanced in relation to other factors such as market requirements.

Category 1: Regional Settings

The most important bugs that will be found here relate to the basic i18N issues in Part I of the book: Can data be correctly input and is the data that is input subsequently handled and output correctly? Does this apply to the various problems and issues with date/time values, numeric values, and currencies? What happens when calendar settings change?

This testing should center on the control panel. Often, the best way to test your application is to have unique and bizarre language choices for your regional settings-perhaps even customizing formats at times. When you report bugs, make sure to accurately describe the settings you used because they can be very difficult to remember when you change them often. There is nothing more frustrating to a tester to have found a bug and then not be able to find it again!

In addition to the general testing of different settings, choosing to focus some of your time on the actual configurations that you know your users will have is also valuable. But do not think that the general testing is a waste of time; the best place to find such bugs is in locales that neither developer nor tester was really thinking about. It will also give you more confidence when new markets are considered.

Category 2: Input Locales and Data Storage

All the bugs in Category 1 are, of course, possible to find; in addition, you will be looking for bugs related to the input of native characters and their persistent storage. Generally speaking, the amount of work to do Category 2 is already being expended for Category 1, so it makes sense to consider fuller coverage for roughly the same cost in terms of planning.

Of course, the scenarios multiply as soon as you agree to cover more, so you want to make sure that you have people in your test organization who are ready and able to change their settings constantly-in possibly more disruptive ways than Category 1 would ever be.

Category 3: Default System Language

All the bugs in Categories 1 and 2 are possible to find, and a whole new array of bugs is possible as you look for bugs related to native characters on other code pages and how the application under test (AUT) behaves on these locales.

It makes sense to plan on testing specific families of languages, such as the following. (Each item in this list will include the types of bugs found in other families.)

• At least one language on the same code page that requires a different amount of space to be taken by text-German or Dutch is a good example if you are starting from the United States. The U.S. is a good example if you are on German or Dutch and want to make sure that dialogs are acceptable when less space is being utilized.
• At least one language on another code page-Greek, Russian, and Turkish are good examples if you are starting from the U.S. or Western Europe.
• At least one Asian language-Chinese, Japanese, and Korean are good examples, with Japanese being the most common choice.
• At least one bidirectional language-Arabic and Hebrew are good examples, although Arabic will often find more bugs, due to the regular nature of Hebrew characters with little to no shaping rules that affect the characters in the language.
• At least one complex scripts language-Thai and Vietnamese are good examples because they have some of the complex script shaping or line-breaking rules in which it is good to test the application's behavior.
• At least one Unicode-only language-Hindi is a good example.

Obviously, you do not necessarily need to pick all these categories. For example, unless you plan to invest a great deal of time in Part II of this book and in support for multinational applications, you need to spend as much time on the Unicode-only languages.(You can assume that they will simply not work!)

Category 4: Faux-Localized Operating System

This category can, of course, find all the bugs in Categories 1, 2, and 3 (although how much of Category 3 is covered depends on your language choices). Generally, the additional types of bugs that you will find relate to components (including common dialogs and localized components such as Internet Explorer or Microsoft Office) provided by the operating system, which can vary across languages.

Windows 2000 Multilangauge User Interface (MUI) and Office 2000 Language Packs are the ideal (and, in fact, only) way to test this category, but it is worth considering whether you have strong dependencies on such components in your AUT.

Category 5: Truly Localized Operating System

This category is the ultimate level of testing which can find bugs in all the other categories (how many Category 2 bugs are covered depends on your language choices). The resources required to test this scenario are often too large to realistically consider, although if you have a large enough product and are planning to automate the testing of it.

An MSDN Universal or Professional subscription is required here because it can get you localized versions of all the operating systems. Spending the time to learn how automated setups work is a good idea for languages that you do not know because you will not have to spend as much time worrying about error messages that you do not understand during setup (something I know from personal experience!).

Choosing Which Category from Table 16.1 to Support

The choice of which categories to support can be a complex one, and you will seldom find a more complex problem in need of cost/benefit analysis. Generally, doing the majority of testing in Category 2 with spot-checking in Category 3 provides the best balance. As you become more experienced with the developers and testers in your organization, you will not only become more sophisticated in testing the product, but your developers will better understand the issues and they will produce fewer i18N bugs.

Installation of the Application Under Test (AUT)

Although this step seems trite, I can assure you that it is not. It is important to install on a localized operating system and deal with situations such as the fact that common directories (for example, Program Files) might have localized names. This can underscore two different types of problems:

• The program might not be installing to the proper location.
• The program might have problems installing to locations that include native characters in the path.

These are the sorts of issues that you can test with every install.

For your test environment when setting up applications that use SQL Server, your test server should be case sensitive because this can often discover problems such as those with the Turkish I that I discuss in Chapter 4, "User Interface Issues," and Chapter 12, " Jet, SQL Server, and Other Databases." Choosing different sorts for the server can find problems as well. Proper set up of your test SQL Server can avoid many problems that would otherwise be found by your customers. Regularly reinstalling SQL Server with different sorts is also a good way to find bugs that would otherwise be missed.

Execution of Your International Test Cases

The subject of doing breakouts that list all the important scenarios to test (also known as a test breakout, which contains test cases) is obviously beyond the scope of this book. However, I will list many of the issues to keep in mind as you create the test cases for the AUT.

Things to Keep in Mind for i18N

For globalized applications, here is a sample of the types of issues you will want to consider:

• Check every place that you can enter a date, time, currency, or number in every place that accepts these values in the AUT. Make sure that you can enter them with the correct format. (See the Regional Settings dialog to find these items for a given locale.)
• Check places where the AUT is doing comparisons, validations, or calculations on dates, times, currency, or numbers. Accepting correctly formatted values doesn’t guarantee that the calculations on these values understand the format.
• Check every place that you can enter native characters into text fields. Be sure to enter a mix of non-English and English characters.
• Make sure that native characters can be displayed properly after they are entered. If the native characters require a special font (for example, Japanese), is your application installing this font on the machine? (Or are you guaranteed that it will be available for every Windows installation?) If the native characters need a special font to display, you need to be sure to check every location where text is displayed in your GUI. (Typically, each control has a font property that may be set individually.)
• Enter the maximum number of characters (using native characters) into each different “size” control in the GUI. Because Unicode and DBCS characters take up twice as much room, controls might appear to work for small strings, but are not allocating enough space for large strings.
• Look our for display problems relating to bad Unicode-to-ANSI conversions, which characteristically show a question mark (?) instead of a native character.
• Along the same line, look out for display problems relating to bad font choices, which usually show vertical bars or boxes instead of valid letters.
• Generally speaking, garbage text is due to incorrect conversions from ANSI to Unicode, and you should look out for this.
• To the extent that you can, it is important to make sure that data that should be sorted is, in fact, appropriately sorted.
• If your application has any persistent storage (files, database, and so forth), be sure to test that native characters can "round-trip"; that is, they can be entered, persisted to storage, reloaded, and displayed correctly in the GUI.
• If the bug also happens with pure English data on an English system, it is not an i18N problem, and you should not label it as such.

Things to Keep in Mind for L10N

In a localized application, there is a whole range of test cases that must also be considered, such as the following:

• Check each dialog, and each place where text appears for English strings.
• Check the accelerator key (the letter that has the underscore that enables a user to hit ALT+ to access that control) for each control and menu item on each dialog. (Frequently when localizing, several controls will be assigned the same letter, which effectively disables the accelerators for the duplicated controls.)
• In a more compact source language, check the each place where text appears to make sure that it’s not cut off. For example, most non-English words are much longer than their English counterparts.
• Bring up each error message to make sure that the strings are being handled correctly. If possible, have a native language speaker read the dialogs and ensure that they make sense. Many error messages are concatenated from several English strings (each of which might not be correctly translated to the locale), and are then presented to the user as one string (which most likely will not make sense in another language). I discuss this problem in more detail in Part III, "Localization."
• Review your application for pictures that substitute for words with a native speaker, if possible. Pictures typically aren’t localized, but might not connote the same thing or be appropriate for the given locale. This is discussed in Part III.

Things to Keep in Mind for Complex Scripts in General

• When dealing with languages that consider some character combinations to be illegal, look for a small dotted circle, which is the standard indication for a character that is expecting another character to be there.
• If you do not understand a language, be careful in trying to decide whether the way the language is handled in your application is incorrect. Someone who knows that language can be crucial here.
• Watch carefully for clipping of text both above and below the line.
• When dealing with languages such as Thai, make sure that you have someone who can help you understand whether word breaking is happening properly.
• Arabic scripts have a natural flow of a sentence such as أرسل كافة الشوائب، والأسئلة، والاقتراحات، والشكاوى، والتعليقاتإلى العنوان: (In English, this means "Send all bugs, questions, complaints, and comments to:".) You would not expect many breaks between letters in a word (much like cursive script in Latin text), so be on the lookout for words that seem mostly broken into individual letters or characters.
• Make sure that someone who knows the language can verify the sorting behavior for it.
• If you are on Windows 2000 and want to verify proper keyboard behavior with a Hindi keyboard and a Unicode control, you can try typing v+c+m+d+l+s. This will become the wordनमस्ते, which is the Hindi word Namaste, a common greeting. A similar test for Tamil can be accomplished by typing b+C+k+d+k+c+d. This will become the word வணக்கம, which is the Tamil word Vanakam, a similar greeting. If its letters or any of the shapes look different, complex script support is not working properly.
• You can get similar words in all languages you need to support from customers or localizers.

Where We Just Were...

This chapter discusses many of the issues that come up in testing international applications, issues that must be carefully considered if you are expanding into global markets. It is important for a tester to keep in mind that he or she is, in many ways, the most important person in a company that is expanding its market. Testers are the ones who can help realistically assess the markets that can be covered, and the success or failure in global markets will depend largely on how good a job the testers do in making sure that the developer did a good job.

This makes a great note on which to close the book-with a product that the tester signs off on and development is proud of, and that is embraced by customers around the world! Although the reality can sometimes fall short, by achieving a global perspective, you can guarantee that you will produce products that come much closer to your goal.