SOFTWARE TESTING COMMUNITY WORLD

Automation Testing

2008-09-26T21:36:00.001-07:00

Successful automation mandates a testing process. Just as a developer needs a system development process, testers need a testing process to successfully use test tools. The testing process provides the steps, guidelines and techniques that will ensure practical, successful automation. To achieve the testing and risk management goals of the project, a solid testing process is essential to focus the test automation effort where it can do the most good.

The difficulty in software testing stems from the complexity of software: we can not completely test a program with moderate complexity. Testing is more than just debugging. The purpose of testing can be quality assurance, verification and validation, or reliability estimation.

Testing can be used as a generic metric as well. Correctness testing and reliability testing are two major areas of testing. Software testing is a trade-off between budget, time and quality.

Unlike most physical systems, most of the defects in software are design errors, not manufacturing defects. Software does not suffer from corrosion, wear-and-tear — generally it will not change until upgrades, or until obsolescence. So once the software is shipped, the design defects — or bugs — will be buried in and remain latent until activation. Regardless of the limitations, testing is an integral part in software development. It is broadly deployed in every phase in the software development cycle. Typically, more than 50% percent of the development time is spent in testing. Testing is usually performed for the following purposes:

To improve quality
For Verification & Validation (V&V)
For reliability estimation

Software testing can be very costly. Automation is a good way to cut down time and cost. Software testing tools and techniques usually suffer from a lack of generic applicability and scalability. The reason is straight-forward. In order to automate the process, we have to have some ways to generate oracles from the specification, and generate test cases to test the target software against the oracles to decide their correctness. Today we still don’t have a full-scale system that has achieved this goal. In general, significant amount of human intervention is still needed in testing. The degree of automation remains at the automated test script level. Testing is potentially endless. We can not test till all the defects are unearthed and removed — it is simply impossible. At some point, we have to stop testing and ship the software. The question is when. Realistically, testing is a trade-off between budget, time and quality. It is driven by profit models. The pessimistic, and unfortunately most often used approach is to stop testing whenever some, or any of the allocated resources — time, budget, or test cases — are exhausted. The optimistic stopping rule is to stop testing when either reliability meets the requirement, or the benefit from continuing testing cannot justify the testing cost. This will usually require the use of reliability models to evaluate and predict reliability of the software under test. Each evaluation requires repeated running of the following cycle: failure data gathering — modeling — prediction. This method does not fit well for ultra-dependable systems, however, because the real field failure data will take too long to accumulate.

Manual Testing

2008-09-26T21:34:00.000-07:00

What is Testing?
* An examination of the behavior of a program by executing on sample data sets.
* Testing comprises of set of activities to detect defects in a produced material.
* To unearth & correct defects.
* To detect defects early & to reduce cost of defect fixing.
* To avoid user detecting problems.
* To ensure that product works as users expected it to.
Why Testing? 
* To unearth and correct defects.
* To detect defects early and to reduce cost of defect fixing.
* To ensure that product works as user expected it to.
* To avoid user detecting problems.
Testing Techniques
Black Box Testing -Testing of a function without knowing internal structure of the program.
White Box Testing -Testing of a function with knowing internal structure of the program.
Regression Testing -To ensure that the code changes have not had an adverse affect to the other modules or on existing functions.
Functional Testing:
* Study SRS
* Identify Unit Functions
* For each unit function
- Take each input function
- Identify Equivalence class
- Form Test cases
- Form Test cases for boundary values
- From Test cases for Error Guessing
* Form Unit function v/s Test cases, Cross Reference Matrix
* Find the coverage
Unit Testing:
* The most ‘micro’ scale of testing to test particular functions or code modules. Typically done by the programmer and not by testers .
* Unit - smallest testable piece of software.
* A unit can be compiled/ assembled/ linked/ loaded; and put under a test harness.
* Unit testing done to show that the unit does not satisfy the functional specification and/ or its implemented structure does not match the intended design structure.
Integration Testing: 
* Integration is a systematic approach to build the complete software structure specified in the design from unit-tested modules. There are two ways integration performed. It is called Pre-test and Pro-test.
* Pre-test: the testing performed in Module development area is called Pre-test. The Pre-test is required only if the development is done in module development area.
Alpha testing: 
* Testing of an application when development is nearing completion minor design changes may still be made as a result of such testing. Typically done by end-users or others, not by programmers or testers.
Beta testing: 
* Testing when development and testing are essentially completed and final bugs and problems need to be found before final release. Typically done by end-users or others, not by programmers.
System Testing: 
* A system is the big component.
* System testing is aimed at revealing bugs that cannot be attributed to a component as such, to inconsistencies between components or planned interactions between components.
* Concern: issues, behaviors that can only be exposed by testing the entire integrated system (e.g., performance, security, recovery).
Volume Testing: 
* The purpose of Volume Testing is to find weaknesses in the system with respect to its handling of large amounts of data during short time periods. For example, this kind of testing ensures that the system will process data across physical and logical boundaries such as across servers and across disk partitions on one server.
Stress testing: 
* This refers to testing system functionality while the system is under unusually heavy or peak load; it’s similar to the validation testing mentioned previously but is carried out in a “high-stress” environment. This requires that you make some predictions about expected load levels of your Web site.
Usability testing: 
* Usability means that systems are easy and fast to learn, efficient to use, easy to remember, cause no operating errors and offer a high degree of satisfaction for the user. Usability means bringing the usage perspective into focus, the side towards the user.
Security testing: 
* If your site requires firewalls, encryption, user authentication, financial transactions, or access to databases with sensitive data, you may need to test these and also test your site’s overall protection against unauthorized internal or external access.
Test Life Cycle
* Identify Test Candidates
* Test Plan
* Design Test Cases
* Execute Tests
* Evaluate Results
* Document Test Results
* Post Shipment Review
Test Plan: 
* A Test Plan is a detailed project plan for testing, covering the scope of testing, the methodology to be used, the tasks to be performed, resources, schedules, risks, and dependencies. A Test Plan is developed prior to the implementation of a project to provide a well defined and understood project roadmap.
Test Specification: 
* A Test Specification defines exactly what tests will be performed and what their scope and objectives will be. A Test Specification is produced as the first step in implementing a Test Plan, prior to the onset of manual testing and/or automated test suite development. It provides a repeatable, comprehensive definition of a testing campaign.

Approach for Test Automation

2008-09-09T22:24:00.000-07:00

Description

Approach for Test Automation is used for building a strategy for automation starting from the Requirement phase till the deployment phase. Approach for automation begins with finding out the conditions and business rules given by the client and grouping similar conditions, business rules together as Test cases or Test Scenarios.

Steps involved in Approach for Test Automation are as follows:

· Requirement gathering from the client

· Understanding and Analyzing the Requirement

· Grouping the requirement into Test cases

· Preparing the Design for Automation

· Building Scripts depending on the Design

· Review of Scripts at Offshore

· Delivery to the Client

For example in the Unilever project:

Since it is Project for Automation of SAP, Team was divided into two,

SAP Consultants.

Testing Team.

Requirement gathering from the client included knowledge transfer at onsite, where all the transactions including the customized transactions where explained to the SAP Consultants of satyam, Under standing of requirements was done at the client side, Depending on the flow of transactions, all the transactions where grouped together as different scenarios. While SAP Consultants where onsite for gathering Requirements, Work done at the offshore was to analyze on the appropriate version of the QTP Tool to be used and to get licenses for QTP, user accounts for SAP access, Installation of QTP and SAP.

All the Identified Scenarios which consists of different transactions where explained to Testing Team. Then common functionalities where identified, and grouped together as different Test cases.

After defining different Test cases, design for automation of scripts was done, where Reusability was one of the main aspects, where all the transactions, which are common in the scenarios, are identified as Reusable transactions.

Automation Framework:

For Quality deliverables, important things that should be followed are:

· Following common coding structure for all the scripts

· Consistent coding conventions

· Use of Reusability for making code efficient and minimal

· Efficient use of Object Repository

· Use of Error and Exception handling Functions

· Use of Data table object, Environment variables for using data

For example in the Unilever project:
Coding Structure used is that there will a main Action for all the scripts which is named as corresponding Test case name, This Action intern will call all the other actions (Each Transaction used in the script would be an action) in the script. Importing and Exporting of sheets is also done in this main action. Importing of sheets is done at the beginning before calling other actions and exporting of sheets is done at the last. Sheets corresponding to all the actions in the scripts are imported, so that the data in the fields of the Imported sheets are used as input data to the script, output data from the script is also collected and exported to the corresponding sheet.

There are many transactions which are reused in the scripts, similar transactions (Actions) are made as reusable so that its not required to record those transactions again and again, which ever transaction was common among scripts, were recorded in one script and is reused in all the scripts where the transaction is required.

Error handling in the unilever automation is done by using Recovery Scenario where each Recovery scenario calls a function which handles the recovery by exiting all the other transactions (actions) when recovery fires. Major types of Recovery used in this project where Popup Window, Object state and on Error Recoveries. In Error handling using Recovery, all the common types of exceptions or errors are given common Recovery scenarios using regular expressions, which minimized the number of Recovery scenarios and Functions used.

Design for Structure of Scripts used in Unilever Project:

Main Action, which calls all the other Called Actions

Actions (transactions) in the script

Calls

Ex: S014_001 (Action Name) Ex: ME21N (Action Name

Which is same as the

Transaction name in

SAP)

WinRunner testing process involves six main stages

2008-08-18T10:56:00.000-07:00

• Create GUI Map File so that WinRunner can recognize the GUI objects in the application being tested
• Create test scripts by recording, programming, or a combination of both. While recording tests, insert checkpoints where you want to check the response of the application being tested.
• Debug Test: run tests in Debug mode to make sure they run smoothly
• Run Tests: run tests in Verify mode to test your application.
• View Results: determines the success or failure of the tests.
• Report Defects: If a test run fails due to a defect in the application being tested, you can report information about the defect directly from the Test Results window.
2. What is Contained in the GUI   Map ?
WinRunner stores information it learns about a window or object in a GUI Map. When WinRunner runs a test, it uses the GUI map to locate objects. It reads an object’s description in the GUI map and then looks for an object with the same properties in the application being tested. Each of these objects in the GUI Map file will be having a logical name and a physical description.
There are 2 types of GUI Map files.
• Global GUI Map file: a single GUI Map file for the entire application
•     GUI Map File per Test: WinRunner automatically creates a GUI Map file for each test created.
3. How Does WinRunner recognize objects on the Application?
WinRunner uses the GUI Map file to recognize objects on the application. When WinRunner runs a test, it uses the GUI map to locate objects. It reads an object’s description in the GUI map and then looks for an object with the same properties in the application being tested.

4. How Does WinRunner evaluates test Results?

Following each test run, WinRunner displays the results in a report. The report details all the major events that occurred during the run, such as checkpoints, error messages, system messages, or user messages. If mismatches are detected at checkpoints during the test run, you can view the expected results and the actual results from the Test Results window.

5. Have You Created test Scripts and what is contained in the Test Scripts?

Yes I have created test scripts. It contains the statement in Mercury Interactive Test Script Language (TSL). These statements appear as a test script in a test window. You can then enhance your recorded test script, either by typing in additional TSL functions and programming elements or by using Win Runner’s visual programming tool, the Function Generator.

6. Have you performed debugging of the scripts?

Yes, I have performed debugging of scripts. We can debug the script by executing the script in the debug mode. We can also debug script using the Step, Step Into, Step out functionalities provided by the WinRunner.

7.  How do you run your test scripts?

We run tests in Verify mode to test your application. Each time WinRunner encounters a checkpoint in the test script, it compares the current data of the application being tested to the expected data captured earlier. If any mismatches are found, WinRunner captures them as actual results.

8.  How do you analyze the results and report the defects?

Following each test run, WinRunner displays the results in a report. The report details all the major events that occurred during the run, such as checkpoints, error messages, system messages, or user messages. If mismatches are detected at checkpoints during the test run, you can view the expected results and the actual results from the Test Results window. If a test run fails due to a defect in the application being tested, you can report information about the defect directly from the Test Results window. This information is sent via e-mail to the quality assurance manager, who tracks the defect until it is fixed.

9.  What is the use of Test Director Software?

Test Director is Mercury Interactive software test management tool. It helps quality assurance personnel plan and organize the testing process. With Test Director you can create a database of manual and automated tests, build test cycles, run tests, and report and track defects. You can also create reports and graphs to help review the progress of planning tests, running tests, and tracking defects before a software release.

10. How you integrated your automated scripts from Test Director?

When you work with WinRunner, you can choose to save your tests directly to your Test Director database or while creating a test case in the Test Director we can specify whether the script in automated or manual. And if it is automated script then Test Director will build a skeleton for the script that can be later modified into one which could be used to test the AUT.

11. What is the purpose of loading WinRunner Add-Ins?

Add-Ins are used in WinRunner to load functions specific to the particular add-in to the memory. While creating a script only those functions in the add-in selected will be listed in the function generator and while executing the script only those functions in the loaded add-in will be executed else WinRunner will give an error message saying it does not recognize the function. Add-Ins are used in WinRunner to load functions specific to the particular add-in to the memory. While creating a script only those functions in the add-in selected will be listed in the function generator and while executing the script only those functions in the loaded add-in will be executed else WinRunner will give an error message saying it does not recognize the function.

12. What are the reasons that WinRunner fails to identify an object on the GUI?

WinRunner fails to identify an object in a GUI due to various reasons.
• The object is not a standard windows object.
• If the browser used is not compatible with the WinRunner version, GUI Map Editor will not be able to learn any of the objects displayed in the browser window.
13. What do you mean by the logical name of the object?
An object’s logical name is determined by its class. In most cases, the logical name is the label that appears on an object.
14. If the object does not have a name then what will be the logical name?
If the object does not have a name then the logical name could be the attached text.
15. What is the different between GUI map and GUI map files?
1) The GUI map is actually the sum of one or more GUI map files. There are two modes for organizing GUI map files.
• Global GUI Map file: a single GUI Map file for the entire application
• GUI Map File per Test: WinRunner automatically creates a GUI Map file for each test created.
2) GUI Map file is a file which contains the windows and the objects learned by the WinRunner with its logical name and their physical description.
16. How do you view the contents of the GUI map?
GUI Map editor displays the content of a GUI Map. We can invoke GUI Map Editor from the Tools Menu in WinRunner. The GUI Map Editor displays the various GUI Map files created and the windows and objects learned in to them with their logical name and physical description.
17. When you create GUI map do you record all the objects of specific objects?
If we are learning a window then WinRunner automatically learns all the objects in the window else we will we identifying those object, which are to be learned in a window, since we will be working with only those objects while creating scripts.
18. What is the purpose of set_window command?
Set_Window command sets the focus to the specified window. We use this command to set the focus to the required window before executing tests on a particular window.

Syntax: set_window (, time);
The logical name is the logical name of the window and time is the time the execution has to wait till it gets the given window into focus.
19. How do you load GUI map? What is the disadvantage of loading the GUI maps through start up scripts?
We can load a GUI Map by using the GUI_load command.
Syntax: GUI_load();
• If we are using a single GUI Map file for the entire AUT then the memory used by the GUI Map may be much high.
• If there is any change in the object being learned then WinRunner will not be able to recognize the object, as it is not in the GUI Map file loaded in the memory. So we will have to learn the object again and update the GUI File and reload it.

20. How do you unload the GUI map? What actually happens when you load GUI map?  
We can use GUI_close to unload a specific GUI Map file or else we call use GUI_close_all command to unload all the GUI Map files loaded in the memory.
Syntax:  GUI_close();   or GUI_close_all;
When we load a GUI Map file, the information about the windows and the objects with their logical names and physical description are loaded into memory. So when the WinRunner executes a script on a particular window, it can identify the objects using this information loaded in the memory.
21. What is the purpose of the temp GUI map file? What is the extension of GUI map file?
While recording a script, WinRunner learns objects and windows by itself. This is actually stored into the temporary GUI Map file. We can specify whether we have to load this temporary GUI Map file should be loaded each time in the General Options.
The extension for a GUI Map file is “.gui”.
22. How do you find an object in an GUI map?
The GUI Map Editor is been provided with a Find and Show Buttons.
• To find a particular object in the GUI Map file in the application, select the object and click the Show window. This blinks the selected object.
• To find a particular object in a GUI Map file click the Find button, this gives the option to select the object. When the object is selected, if the object has been learned to the GUI Map file it will be focused in the GUI Map file.
23. What different actions are performed by find and show button?
• To find a particular object in the GUI Map file in the application, select the object and click the Show window. This blinks the selected object.
• To find a particular object in a GUI Map file click the Find button, this gives the option to select the object. When the object is selected, if the object has been learned to the GUI Map file it will be focused in the GUI Map file.
24. How do you identify which files are loaded in the GUI map?
The GUI Map Editor has a drop down “GUI File” displaying all the GUI Map files loaded into the memory.
25. How do you modify the logical name or the physical description of the objects in GUI map?
You can modify the logical name or the physical description of an object in a GUI map file using the GUI Map Editor.
26. When do you feel you need to modify the logical name?
Changing the logical name of an object is useful when the assigned logical name is not sufficiently descriptive or is too long.
27. When it is appropriate to change physical description?
Changing the physical description is necessary when the property value of an object changes.

28. How WinRunner handles varying window labels?

We can handle varying window labels using regular expressions. WinRunner uses two “hidden” properties in order to use regular expression in an object’s physical description. These properties are regexp_label and regexp_MSW_class.
• The regexp_label property is used for windows only. It operates “behind the scenes” to insert a regular expression into a window’s label description.
• The regexp_MSW_class property inserts a regular expression into an object’s MSW_class. It is obligatory for all types of windows and for the object class object.
29. What is the purpose of regexp_label property and regexp_MSW_class property?
• The regexp_label property is used for windows only. It operates “behind the scenes” to insert a regular expression into a window’s label description.
• The regexp_MSW_class property inserts a regular expression into an object’s MSW_class. It is obligatory for all types of windows and for the object class object.
30. How do you suppress a regular expression?
We can suppress the regular expression of a window by replacing the regexp_label property with label property
31. How do you copy and move objects between different GUI map files?
We can copy and move objects between different GUI Map files using the GUI Map Editor. The steps to be followed are:
• Choose Tools > GUI Map Editor to open the GUI Map Editor.
• Choose View > GUI Files.
• Click Expand in the GUI Map Editor. The dialog box expands to display two GUI map files simultaneously.
• View a different GUI map file on each side of the dialog box by clicking the file names in the GUI File lists.
• In one file, select the objects you want to copy or move. Use the Shift key and/or Control key to select multiple objects. To select all objects in a GUI map file, choose Edit > Select All.
• Click Copy or Move.
• To restore the GUI Map Editor to its original size, click Collapse
32. How do you select multiple objects during merging the files?
Use the Shift key and/or Control key to select multiple objects. To select all objects in a GUI map file, choose Edit > Select All
33. How do you clear a GUI map files? How do you filter the objects in the GUI map?
We can clear a GUI Map file using the “Clear All” option in the GUI Map Editor
GUI Map Editor has a Filter option. This provides for filtering with 3 different types of options.
• Logical name displays only objects with the specified logical name.
• Physical description displays only objects matching the specified physical description. Use any substring belonging to the physical description.
• Class displays only objects of the specified class, such as all the push buttons.
34. What is the purpose of GUI map configuration? How do you make the configuration and mappings permanent?
GUI Map configuration is used to map a custom object to a standard object
The mapping and the configuration you set are valid only for the current WinRunner session. To make the mapping and the configuration permanent, you must add configuration statements to your startup test script
35. What is the purpose of GUI spy?
Using the GUI Spy, you can view the properties of any GUI object on your desktop. You use the Spy pointer to point to an object, and the GUI Spy displays the properties and their values in the GUI Spy dialog box. You can choose to view all the properties of an object, or only the selected set of properties that WinRunner learns.
36. What is the purpose of obligatory and optional properties of the objects?
For each class, WinRunner learns a set of default properties. Each default property is classified “obligatory” or “optional”.
• An obligatory property is always learned (if it exists).
• An optional property is used only if the obligatory properties do not provide unique identification of an object. These optional properties are stored in a list. WinRunner selects the minimum number of properties from this list that are necessary to identify the object. It begins with the first property in the list, and continues, if necessary, to add properties to the description until it obtains unique identification for the object.
37. When the optional properties are learned?
An optional property is used only if the obligatory properties do not provide unique identification of an object.
38. What is the purpose of location indicator and index indicator in GUI map configuration?
In cases where the obligatory and optional properties do not uniquely identify an object, WinRunner uses a selector to differentiate between them. Two types of selectors are available:
• A location selector uses the spatial position of objects.
The location selector uses the spatial order of objects within the window, from the top left to the bottom right corners, to differentiate among objects with the same description.
• An index selector uses a unique number to identify the object in a window.The index selector uses numbers assigned at the time of creation of objects to identify the object in a window. Use this selector if the location of objects with the same description may change within a window.
39. How do you handle custom objects?
A custom object is any GUI object not belonging to one of the standard classes used by WinRunner. WinRunner learns such objects under the generic “object” class. WinRunner records operations on custom objects using obj_mouse_ statements.
If a custom object is similar to a standard object, you can map it to one of the standard classes. You can also configure the properties WinRunner uses to identify a custom object during Context Sensitive testing.
40. What is the name of custom class in WinRunner and what methods it applies on the custom objects?
WinRunner learns custom class objects under the generic “object” class. WinRunner records operations on custom objects using obj_ statements.
41. In a situation when obligatory and optional both the properties cannot uniquely identify an object what method WinRunner applies?
In cases where the obligatory and optional properties do not uniquely identify an object, WinRunner uses a selector to differentiate between them. Two types of selectors are available:
• A location selector uses the spatial position of objects.
• An index selector uses a unique number to identify the object in a window.
42. What is the purpose of different record methods 1) Record 2) Pass up 3) As Object 4) Ignore. ?
• Record instructs WinRunner to record all operations performed on a GUI object. This is the default record method for all classes. (The only exception is the static class (static text), for which the default is Pass Up.)
• Pass Up instructs WinRunner to record an operation performed on this class as an operation performed on the element containing the object. Usually this element is a window, and the operation is recorded as win_mouse_click.
• As Object instructs WinRunner to record all operations performed on a GUI object as though its class were “object” class.
• Ignore instructs WinRunner to disregard all operations performed on the class.
43. How do you find out which is the start up file in WinRunner?
The test script name in the Startup Test box in the Environment tab in the General Options dialog box is the start up file in WinRunner.
44. What are the virtual objects and how do you learn them?
• Applications may contain bitmaps that look and behave like GUI objects. WinRunner records operations on these bitmaps using win_mouse_click statements. By defining a bitmap as a virtual object, you can instruct WinRunner to treat it like a GUI object such as a push button, when you record and run tests.
• Using the Virtual Object wizard, you can assign a bitmap to a standard object class, define the coordinates of that object, and assign it a logical name.

To define a virtual object using the Virtual Object wizard:
• Choose Tools > Virtual Object Wizard. The Virtual Object wizard opens. Click Next.
• In the Class list, select a class for the new virtual object. If rows that are displayed in the window. For a table class, select the number of visible rows and columns. Click Next.
• Click Mark Object. Use the crosshairs pointer to select the area of the virtual object. You can use the arrow keys to make precise adjustments to the area you define with the crosshairs. Press Enter or click the right mouse button to display the virtual object’s coordinates in the wizard. If the object marked is visible on the screen, you can click the Highlight button to view it. Click Next.
• Assign a logical name to the virtual object. This is the name that appears in the test script when you record on the virtual object. If the object contains text that WinRunner can read, the wizard suggests using this text for the logical name. Otherwise, WinRunner suggests virtual_object, virtual_push_button, virtual_list, etc.
45. What are the two modes of recording?
• Context Sensitive recording records the operations you perform on your application by identifying Graphical User Interface (GUI) objects.
• Analog recording records keyboard input, mouse clicks, and the precise x- and y-coordinates traveled by the mouse pointer across the screen.
46. What is a checkpoint and what are different types of checkpoints?
Checkpoints allow you to compare the current behavior of the application being tested to its behavior in an earlier version.

You can add four types of checkpoints to your test scripts:
  
• GUI checkpoints verify information about GUI objects. For example, you can check that a button is enabled or see which item is selected in a list.
• Bitmap checkpoints take a “snapshot” of a window or area of your application and compare this to an image captured in an earlier version.
• Text checkpoints read text in GUI objects and in bitmaps and enable you to verify their contents.
• Database checkpoints check the contents and the number of rows and columns of a result set, which is based on a query you create on your database
47. What are data driven tests?
When you test your application, you may want to check how it performs the same operations with multiple sets of data. You can create a data-driven test with a loop that runs ten times: each time the loop runs, it is driven by a different set of data. In order for WinRunner to use data to drive the test, you must link the data to the test script which it drives. This is called parameterizing your test. The data is stored in a data table. You can perform these operations manually, or you can use the DataDriver Wizard to parameterize your test and store the data in a data table.
48. What are the synchronization points?
• Synchronization points enable you to solve anticipated timing problems between the test and your application. For example, if you create a test that opens a database application, you can add a synchronization point that causes the test to wait until the database records are loaded on the screen.
• For Analog testing, you can also use a synchronization point to ensure that WinRunner repositions a window at a specific location. When you run a test, the mouse cursor travels along exact coordinates. Repositioning the window enables the mouse pointer to make contact with the correct elements in the window.
49.  What is parameterizing?

In order for WinRunner to use data to drive the test, you must link the data to the test script which it drives. This is called parameterizing your test. The data is stored in a data table.

50. How do you maintain the document information of the test scripts?

Before creating a test, you can document information about the test in the General and Description tabs of the Test Properties dialog box. You can enter the name of the test author, the type of functionality tested, a detailed description of the test, and a reference to the relevant functional specifications document.

51. What do you verify with the GUI checkpoint for single property and what command it generates, explain syntax?

You can check a single property of a GUI object. For example, you can check whether a button is enabled or disabled or whether an item in a list is selected. To create a GUI checkpoint for a property value, use the Check Property dialog box to add one of the following functions to the test script:
• button_check_info
• scroll_check_info
• edit_check_info
• static_check_info
• list_check_info
• win_check_info
• obj_check_info

Syntax:
button_check_info (button, property, property_value );
edit_check_info ( edit, property, property_value );

52. What do you verify with the GUI checkpoint for object/window and what command it generates, explain syntax?
• You can create a GUI checkpoint to check a single object in the application being tested. You can either check the object with its default properties or you can specify which properties to check.
• Creating a GUI Checkpoint using the Default Checks
• You can create a GUI checkpoint that performs a default check on the property recommended by WinRunner. For example, if you create a GUI checkpoint that checks a push button, the default check verifies that the push button is enabled.
• To create a GUI checkpoint using default checks:
• Choose Create > GUI Checkpoint > For Object/Window, or click the GUI Checkpoint for Object/Window button on the User toolbar. If you are recording in Analog mode, press the CHECK GUI FOR OBJECT/WINDOW soft key in order to avoid extraneous mouse movements. Note that you can press the CHECK GUI FOR OBJECT/WINDOW soft key in Context Sensitive mode as well. The WinRunner window is minimized, the mouse pointer becomes a pointing hand, and a help window opens on the screen.
• Click an object.
• WinRunner captures the current value of the property of the GUI object being checked and stores it in the test’s expected results folder. The WinRunner window is restored and a GUI checkpoint is inserted in the test script as an obj_check_gui statement
Syntax: win_check_gui ( window, checklist, expected_results_file, time );
• Creating a GUI Checkpoint by Specifying which Properties to Check
• You can specify which properties to check for an object. For example, if you create a checkpoint that checks a push button, you can choose to verify that it is in focus, instead of enabled.
• To create a GUI checkpoint by specifying which properties to check:
• Choose Create > GUI Checkpoint > For Object/Window, or click the GUI Checkpoint for Object/Window button on the User toolbar. If you are recording in Analog mode, press the CHECK GUI FOR OBJECT/WINDOW soft key in order to avoid extraneous mouse movements. Note that you can press the CHECK GUI FOR OBJECT/WINDOW soft key in Context Sensitive mode as well. The WinRunner window is minimized, the mouse pointer becomes a pointing hand, and a help window opens on the screen.
• Double-click the object or window. The Check GUI dialog box opens.
• Click an object name in the Objects pane. The Properties pane lists all the properties for the selected object.
• Select the properties you want to check.
• To edit the expected value of a property, first select it. Next, either click the Edit Expected Value button, or double-click the value in the Expected Value column to edit it.
• To add a check in which you specify arguments, first select the property for which you want to specify arguments. Next, either click the Specify Arguments button, or double-click in the Arguments column. Note that if an ellipsis (three dots) appears in the Arguments column, then you must specify arguments for a check on this property. (You do not need to specify arguments if a default argument is specified.) When checking standard objects, you only specify arguments for certain properties of edit and static text objects. You also specify arguments for checks on certain properties of nonstandard objects.
• To change the viewing options for the properties of an object, use the Show Properties buttons.
• Click OK to close the Check GUI dialog box. WinRunner captures the GUI information and stores it in the test’s expected results folder. The WinRunner window is restored and a GUI checkpoint is inserted in the test script as an obj_check_gui or a win_check_gui statement.
• Syntax: win_check_gui ( window, checklist, expected_results_file, time );
      obj_check_gui ( object, checklist, expected results file, time );

53. What information is contained in the checklist file and in which file expected results are stored?
• The checklist file contains information about the objects and the properties of the object we are verifying.
• The gui*.chk file contains the expected results which is stored in the exp folder
54. What do you verify with the GUI checkpoint for multiple objects and what command it generates, explain syntax?
To create a GUI checkpoint for two or more objects:
• Choose Create > GUI Checkpoint > For Multiple Objects or click the GUI Checkpoint for Multiple Objects button on the User toolbar. If you are recording in Analog mode, press the CHECK GUI FOR MULTIPLE OBJECTS softkey in order to avoid extraneous mouse movements. The Create GUI Checkpoint dialog box opens.
• Click the Add button. The mouse pointer becomes a pointing hand and a help window opens.
• To add an object, click it once. If you click a window title bar or menu bar, a help window prompts you to check all the objects in the window.
• The pointing hand remains active. You can continue to choose objects by repeating step 3 above for each object you want to check.
• Click the right mouse button to stop the selection process and to restore the mouse pointer to its original shape. The Create GUI Checkpoint dialog box reopens.
• The Objects pane contains the name of the window and objects included in the GUI checkpoint. To specify which objects to check, click an object name in the Objects pane. The Properties pane lists all the properties of the object. The default properties are selected.
• To edit the expected value of a property, first select it. Next, either click the Edit Expected Value button, or double-click the value in the Expected Value column to edit it.
• To add a check in which you specify arguments, first select the property for which you want to specify arguments. Next, either click the Specify Arguments button, or double-click in the Arguments column. Note that if an ellipsis appears in the Arguments column, then you must specify arguments for a check on this property. (You do not need to specify arguments if a default argument is specified.) When checking standard objects, you only specify arguments for certain properties of edit and static text objects. You also specify arguments for checks on certain properties of nonstandard objects.
• To change the viewing options for the properties of an object, use the Show Properties buttons.
• To save the checklist and close the Create GUI Checkpoint dialog box, click OK. WinRunner captures the current property values of the selected GUI objects and stores it in the expected results folder. A win_check_gui statement is inserted in the test script.
Syntax: win_check_gui ( window, checklist, expected_results_file, time );
    obj_check_gui ( object, checklist, expected results file, time );
55. What do you verify with the bitmap check point for object/window and what command it generates, explain syntax?
• You can check an object, a window, or an area of a screen in your application as a bitmap. While creating a test, you indicate what you want to check. WinRunner captures the specified bitmap, stores it in the expected results folder (exp) of the test, and inserts a checkpoint in the test script. When you run the test, WinRunner compares the bitmap currently displayed in the application being tested with the expected bitmap stored earlier. In the event of a mismatch, WinRunner captures the current actual bitmap and generates a difference bitmap. By comparing the three bitmaps (expected, actual, and difference), you can identify the nature of the discrepancy.
• When working in Context Sensitive mode, you can capture a bitmap of a window, object, or of a specified area of a screen. WinRunner inserts a checkpoint in the test script in the form of either a win_check_bitmap or obj_check_bitmap statement.
• Note that when you record a test in Analog mode, you should press the CHECK BITMAP OF WINDOW soft key or the CHECK BITMAP OF SCREEN AREA soft key to create a bitmap checkpoint. This prevents WinRunner from recording extraneous mouse movements. If you are programming a test, you can also use the Analog function check_window to check a bitmap.
• To capture a window or object as a bitmap:
• Choose Create > Bitmap Checkpoint > For Object/Window or click the Bitmap Checkpoint for Object/Window button on the User toolbar. Alternatively, if you are recording in Analog mode, press the CHECK BITMAP OF OBJECT/WINDOW soft key. The WinRunner window is minimized, the mouse pointer becomes a pointing hand, and a help window opens.
• Point to the object or window and click it. WinRunner captures the bitmap and generates a win_check_bitmap or obj_check_bitmap statement in the script. The TSL statement generated for a window bitmap has the following syntax:
win_check_bitmap ( object, bitmap, time );
• For an object bitmap, the syntax is: 
     obj_check_bitmap ( object, bitmap, time );
• For example, when you click the title bar of the main window of the Flight Reservation application, the resulting statement might be:
win_check_bitmap ("Flight Reservation", "Img2", 1);
• However, if you click the Date of Flight box in the same window, the statement might be:
• obj_check_bitmap ("Date of Flight:", "Img1", 1);
Syntax: obj_check_bitmap ( object, bitmap, time [, x, y, width, height] );
56. What do you verify with the bitmap checkpoint for screen area and what command it generates, explain syntax?
• You can define any rectangular area of the screen and capture it as a bitmap for comparison. The area can be any size: it can be part of a single window, or it can intersect several windows. The rectangle is identified by the coordinates of its upper left and lower right corners, relative to the upper left corner of the window in which the area is located. If the area intersects several windows or is part of a window with no title (for example, a popup window), its coordinates are relative to the entire screen (the root window).
• To capture an area of the screen as a bitmap:
• Choose Create > Bitmap Checkpoint > For Screen Area or click the Bitmap Checkpoint for Screen Area button. Alternatively, if you are recording in Analog mode, press the CHECK BITMAP OF SCREEN AREA softkey. The WinRunner window is minimized, the mouse pointer becomes a crosshairs pointer, and a help window opens.
• Mark the area to be captured: press the left mouse button and drag the mouse pointer until a rectangle encloses the area; then release the mouse button.
• Press the right mouse button to complete the operation. WinRunner captures the area and generates a win_check_bitmap statement in your script.
• The win_check_bitmap statement for an area of the screen has the following syntax:
 win_check_bitmap ( window, bitmap, time, x, y, width, height );
57. What do you verify with the database checkpoint default and what command it generates, explain syntax?
• By adding runtime database record checkpoints you can compare the information in your application during a test run with the corresponding record in your database. By adding standard database checkpoints to your test scripts, you can check the contents of databases in different versions of your application.
• When you create database checkpoints, you define a query on your database, and your database checkpoint checks the values contained in the result set. The result set is set of values retrieved from the results of the query.
• You can create runtime database record checkpoints in order to compare the values displayed in your application during the test run with the corresponding values in the database. If the comparison does not meet the success criteria you
• Specify for the checkpoint, the checkpoint fails. You can define a successful runtime database record checkpoint as one where one or more matching records were found, exactly one matching record was found, or where no matching records are found.
• You can create standard database checkpoints to compare the current values of the properties of the result set during the test run to the expected values captured during recording or otherwise set before the test run. If the expected results and the current results do not match, the database checkpoint fails. Standard database checkpoints are useful when the expected results can be established before the test run.
Syntax: db_check(, );
• You can add a runtime database record checkpoint to your test in order to compare information that appears in your application during a test run with the current value(s) in the corresponding record(s) in your database. You add runtime database record checkpoints by running the Runtime Record Checkpoint wizard. When you are finished, the wizard inserts the appropriate db_record_check statement into your script.

Syntax:
db_record_check(ChecklistFileName,SuccessConditions,RecordNumber );

ChecklistFileName A file created by WinRunner and saved in the test's checklist folder. The file contains information about the data to be captured during the test run and its corresponding field in the database. The file is created based on the information entered in the Runtime Record Verification wizard.
Success Conditions   Contains one of the following values:
1. DVR_ONE_OR_MORE_MATCH - The checkpoint passes if one or more matching database records are found.
2. DVR_ONE_MATCH - The checkpoint passes if exactly one matching database record is found.
3. DVR_NO_MATCH - The checkpoint passes if no matching database records are found.
RecordNumber An out parameter returning the number of records in the database.
58. How do you handle dynamically changing area of the window in the bitmap checkpoints?
The difference between bitmaps option in the Run Tab of the general options defines the minimum number of pixels that constitute a bitmap mismatch
59. What do you verify with the database check point custom and what command it generates, explain syntax?
• When you create a custom check on a database, you create a standard database checkpoint in which you can specify which properties to check on a result set.
• You can create a custom check on a database in order to:
• check the contents of part or the entire result set
• edit the expected results of the contents of the result set
• count the rows in the result set
• count the columns in the result set
• You can create a custom check on a database using ODBC, Microsoft Query or Data Junction.
60. What do you verify with the sync point for object/window property and what command it generates, explain syntax?
• Synchronization compensates for inconsistencies in the performance of your application during a test run. By inserting a synchronization point in your test script, you can instruct WinRunner to suspend the test run and wait for a cue before continuing the test.
• You can a synchronization point that instructs WinRunner to wait for a specified object or window to appear. For example, you can tell WinRunner to wait for a window to open before performing an operation within that window, or you may want WinRunner to wait for an object to appear in order to perform an operation on that object.
• You use the obj_exists function to create an object synchronization point, and you use the win_exists function to create a window synchronization point. These functions have the following syntax:
Syntax:
obj_exists ( object [, time ] );
win_exists ( window [, time ] );
61. What do you verify with the sync point for object/window bitmap and what command it generates, explain syntax?
• You can create a bitmap synchronization point that waits for the bitmap of an object or a window to appear in the application being tested.
• During a test run, WinRunner suspends test execution until the specified bitmap is redrawn, and then compares the current bitmap with the expected one captured earlier. If the bitmaps match, then WinRunner continues the test.
• Syntax:
obj_wait_bitmap ( object, image, time );
win_wait_bitmap ( window, image, time );
62. What do you verify with the sync point for screen area and what command it generates, explain syntax?
• For screen area verification we actually capture the screen area into a bitmap and verify the application screen area with the bitmap file during execution
• Syntax: obj_wait_bitmap(object, image, time, x, y, width, height);
63. How do you edit checklist file and when do you need to edit the checklist file?
WinRunner has an edit checklist file option under the create menu. Select the “Edit GUI Checklist” to modify GUI checklist file and “Edit Database Checklist” to edit database checklist file. This brings up a dialog box that gives you option to select the checklist file to modify. There is also an option to select the scope of the checklist file, whether it is Test specific or a shared one. Select the checklist file, click OK which opens up the window to edit the properties of the objects.
64. How do you edit the expected value of an object?
We can modify the expected value of the object by executing the script in the Update mode. We can also manually edit the gui*.chk file which contains the expected values which come under the exp folder to change the values.
65. How do you modify the expected results of a GUI checkpoint?
We can modify the expected results of a GUI checkpoint be running the script containing the checkpoint in the update mode.
66. How do you handle ActiveX and Visual basic objects?
WinRunner provides with add-ins for ActiveX and Visual basic objects. When loading WinRunner, select those add-ins and these add-ins provide with a set of functions to work on ActiveX and VB objects.
67. How do you create ODBC query?
We can create ODBC query using the database checkpoint wizard. It provides with option to create an SQL file that uses an ODBC DSN to connect to the database. The SQL File will contain the connection string and the SQL statement.
68. How do you record a data driven test?
We can create a data-driven testing using data from a flat file, data table or a database.
• Using Flat File: we actually store the data to be used in a required format in the file. We access the file using the File manipulation commands, reads data from the file and assign the variables with data.
• Data Table: It is an excel file. We can store test data in these files and manipulate them. We use the ‘ddt_*’ functions to manipulate data in the data table.
• Database: we store test data in the database and access these data using ‘db_*’ functions.
69. How do you parameterize database check points?
When you create a standard database checkpoint using ODBC (Microsoft Query), you can add parameters to an SQL statement to parameterize the checkpoint. This is useful if you want to create a database checkpoint with a query in which the SQL statement defining your query changes
70. How do you convert a database file to a text file?
You can use Data Junction to create a conversion file which converts a database to a target text file.

71. How do you create parameterize SQL commands?
• A parameterized query is a query in which at least one of the fields of the WHERE clause is parameterized, i.e., the value of the field is specified by a question mark symbol ( ? ). For example, the following SQL statement is based on a query on the database in the sample Flight Reservation application:

SELECT Flights.Departure, Flights.Flight_Number, Flights.Day_Of_Week FROM Flights WHERE (Flights.Departure=?) AND (Flights.Day_Of_Week=?)

    SELECT defines the columns to include in the query.
    FROM specifies the path of the database.
WHERE (optional) specifies the conditions, or filters to use in the query.
Departure is the parameter that represents the departure point of a flight.
Day_Of_Week is the parameter that represents the day of the week of a flight.
• When creating a database checkpoint, you insert a db_check statement into your test script. When you parameterize the SQL statement in your checkpoint, the db_check function has a fourth, optional, argument: the parameter_array argument. A statement similar to the following is inserted into your test script:

db_check("list1.cdl", "dbvf1", NO_LIMIT, dbvf1_params);

The parameter_array argument will contain the values to substitute for the parameters in the parameterized checkpoint.

72.  Explain the following WinRunner Commands?
• db_connect - to connect to a database
db_connect(, );
• db_execute_query - to execute a query
db_execute_query ( session_name, SQL, record_number );
[record_number  is the out value]
• db_get_field_value - returns the value of a single field in the specified row_index and column in the session_name database session.
db_get_field_value ( session_name, row_index, column );
• db_get_headers - returns the number of column headers in a query and the content of the column headers, concatenated and delimited by tabs.
db_get_headers ( session_name, header_count, header_content );

• db_get_row - returns the content of the row, concatenated and delimited by tabs.
db_get_row ( session_name, row_index, row_content );
• db_write_records - writes the record set into a text file delimited by tabs.
db_write_records ( session_name, output_file [ , headers [ , record_limit ] ] );
• db_get_last_error - returns the last error message of the last ODBC or Data Junction operation in the session_name database session.
db_get_last_error ( session_name, error );
• db_disconnect - disconnects from the database and ends the database session.
db_disconnect ( session_name );
• db_dj_convert - runs the djs_file Data Junction export file. When you run this file, the Data Junction Engine converts data from one spoke (source) to another (target). The optional parameters enable you to override the settings in the Data Junction export file.
db_dj_convert ( djs_file [ , output_file [ , headers [ , record_limit ] ] ] );

73. What check points you will use to read and check text on the GUI and explain its syntax?
You can use text checkpoints in your test scripts to read and check text in GUI objects and in areas of the screen. While creating a test you point to an object or a window containing text. WinRunner reads the text and writes a TSL statement to the test script. You may then add simple programming elements to your test scripts to verify the contents of the text.
You can use a text checkpoint to:
1. Read text from a GUI object or window in your application, using obj_get_text and win_get_text
2. Search for text in an object or window, using win_find_text and obj_find_text
3. Move the mouse pointer to text in an object or window, using obj_move_locator_text and win_move_locator_text
4. Click on text in an object or window, using obj_click_on_text and win_click_on_text.
74. Explain Get Text checkpoint from object/window with syntax?
• We use obj_get_text (, ) function to get the text from an object
• We use win_get_text (window, out_text [, x1, y1, x2, y2]) function to get the text from a window.
75. Explain Get Text checkpoint from screen area with syntax?
 We use win_get_text (window, out_text [, x1, y1, x2, y2]) function to get the text from a window.
76. Explain Get Text checkpoint from selection (web only) with syntax?

Returns a text string from an object.
web_obj_get_text (object, table_row, table_column, out_text [, text_before, text_after, index]);
• object     The logical name of the object.
• table_row If the object is a table, it specifies the location of the row within a table. The string is preceded by the # character.
• table_column If the object is a table, it specifies the location of the column within a table. The string is preceded by the # character.
• out_text     The output variable that stores the text string.
• text_before     Defines the start of the search area for a particular text string.
• text_after     Defines the end of the search area for  a particular text string.
• index     The occurrence number to locate. (The default parameter number is numbered 1).

WinRunner Coding Standards

2008-08-18T10:53:00.000-07:00

1) No hard coded paths.

A WinRunner test should be able to be copied from one machine to another and run without any problems. Anything the test depends on (gui maps, text files, compiled modules, dll's) should be in the same parent folder as the test.

Exception:

Paths to permanent files on the K drive can be hard coded if absolutely necessary. (Warning – you may run into problems with tests running simultaneously on different machines accessing the same files on the K drive)

Wrong:

reload("C:\\WR_TESTS\\Acceptance_6\\acceptance_functions");

Right:

reload(getvar("testname") & "\\..\acceptance_functions");

2) Indent blocks of code for readability

Wrong:

for(counter = count - 24; counter < count - 1; counter++)
{
list_get_item("ListBox",counter,item);
str = str & item & "\r\n";
}

Right:

for(counter = count - 24; counter < count - 1; counter++)
{
list_get_item("ListBox",counter,item);
str = str & item & "\r\n";
}

Right:

for(counter = count - 24; counter < count - 1; counter++){
list_get_item("ListBox",counter,item);
str = str & item & "\r\n";
}

3) Avoid hard coding testing environment dependencies

Do not hardcode information which my change depending on testing environment. These include installation directories, DSN names, names of database servers, database usernames, and database passwords. It's better to define these in variables at the beginning of the test, so you do not have to make multiple changes throughout the script to implement an environmental change.

Wrong:

set_window("SQL Server Login",10);
edit_set("Login ID:", "sa");
edit_type("Password:", "password");

Right:

db_username = "sa";
db_password = "password";

set_window("SQL Server Login",10);
edit_set("Login ID:", db_username);
edit_type("Password:", db_password);

Wrong (also violates coding standard 1):

invoke_application("C:\\iAvenue\\Windows\\UAdmin.exe","","c:\\Power_db",SW_SHOW);

Right:

install_dir = "c:\\iAvenue\\Windows";

invoke_application(install_dir & "\\UAdmin.exe","",getvar("testname") & "\\..\Power_db",SW_SHOW);

4) Use text recognition as a last resort

Text recognition takes a lot of memory, can be unreliable, and can have varying results on different operating systems. It should therefore only be used if there is no other way to get the information from an object. Unfortunately, this is often the case, especially when the object is not recognized (class: object). In the following example, assume that "Assign Date" is class edit:

Wrong:

obj_get_text("Assign Date", text);

Right:

edit_get_text("Assign Date",text);

5) Do not use excessive wait statements. Try to use synchronization functions when waiting is required.

Wrong:

wait(40);

Right:

statusbar_wait_info("Status Bar","value","Sites processed = 20",40);

6) Do not use report_msg as a substitute for tl_step.

Nobody wants to read every line of the test results looking for a failure. It's much easier to look for green or red. It's OK to have a tl_step failure without a tl_step pass.

Wrong:

if(win_exists("Active Information Manager",1) == 0)
{
set_window("Active Information Manager", 1);
obj_get_text("AfxWnd42", text);
my_gui_checkpoint(text,"AIM.log");
}
else
report_msg("AIM failure! Window absent at startup");

Right:

if(win_exists("Active Information Manager",1) == 0)
{
set_window("Active Information Manager", 1);
obj_get_text("AfxWnd42", text);
my_gui_checkpoint(text,"AIM.log");
}
else
tl_step("AimReportRuns",FAIL,"AIM window absent at startup");

7) Use regular expressions to avoid multiple window instances in the gui map.

Wrong:

7) Use regular expressions to avoid multiple window instances in the gui map (continued)

Right:

HTML Validators

2008-07-28T08:23:00.001-07:00

RealValidator - Shareware HTML validator based on SGML parser by Liam Quinn. Unicode-enabled, supports documents in virtually any language; supports XHTML 1.0, HTML 4.01, HTML 4.0, HTML 3.2, HTML 3.0, and HTML 2.0 ; extensible - add proprietary HTML DTDs or change the existing ones; fetches external DTDs by HTTP and caches them for faster validation; HTML 3.2 and HTML 4.0 references included as HTML Help. For Windows.

CSE 3310 HTML Validator - HTML syntax checker for Windows from AI Internet Solutions. Supports wide variety of standards; accessibility (508) checking; uppercase/lowercase converter. Free 'lite' version. For Windows.

Link Checking Tools

2008-07-28T08:22:00.001-07:00

SiteAnalysis - Hosted service from Webmetrics, used to test and validate critical website components, such as internal and external links, domain names, DNS servers and SSL certificates. Runs as often as every hour, or as infrequent as once a week. Ideal for dynamic sites requiring frequent link checking.

HiSoftware Link Validation Utility - Link validation tool; available as part of the AccVerify Product Line.

ChangeAgent - Link checking and repair tool from Expandable Language. Identifies orphan files and broken links when browsing files; employs a simple, familiar interface for managing files; previews files when fixing broken links and before orphan removal; updates links to moved and renamed files; fixes broken links with an easy, 3-click process; provides multiple-level undo/redo for all operations; replaces links but does not reformat or restructure HTML code. For Windows.

Link Checker Pro - Link check tool from KyoSoft; can also produce a graphical site map of entire web site. Handles HTTP, HTTPS, and FTP protocols; several report formats available. For Windows platforms.

Web Link Validator - Link checker from REL Software checks links for accuracy and availability, finds broken links or paths and links with syntactic errors. Export to text, HTML, CSV, RTF, Excel. Freeware 'REL Link Checker Lite' version available for small sites. For Windows.

Site Audit - Low-cost on-the-web link-checking service from Blossom Software.

Xenu's Link Sleuth - Freeware link checker by Tilman Hausherr; supports SSL websites; partial testing of ftp and gopher sites; detects and reports redirected URL; Site Map; for Windows.

Linkalarm - Low cost on-the-web link checker from Link Alarm Inc.; free trial period available. Automatically-scheduled reporting by e-mail.

Alert Linkrunner - Link check tool from Viable Software Alternatives; evaluation version available. For Windows.

InfoLink - Link checker program from BiggByte Software; can be automatically scheduled; includes FTP link checking; multiple page list and site list capabilities; customizable reports; changed-link checking; results can be exported to database. For Windows. Discontinued, but old versions still available as freeware.

LinkScan - Electronic Software Publishing Co.'s link checker/site mapping tool; capabilities include automated retesting of problem links, randomized order checking; can check for bad links due to specified problems such as server-not-found, unauthorized-access, doc-not-found, relocations, timeouts. Includes capabilities for central management of large multiple intranet/internet sites. Results stored in database, allowing for customizable queries and reports. Validates hyperlinks for all major protocols; HTML syntax error checking. For all UNIX flavors, Windows, Mac.

CyberSpyder Link Test - Shareware link checker by Aman Software; capabilities include specified URL exclusions, ID/Password entries, test resumption at interruption point, page size analysis, 'what's new' reporting. For Windows.

Java Test Tools

2008-07-28T08:19:00.000-07:00

EMMA - Open-source toolkit, written in pure Java, for measuring and reporting Java code coverage. Targets support for large-scale enterprise software development while keeping individual developer's work fast and iterative. Can instrument classes for coverage either offline or on the fly (using an instrumenting application classloader); supported coverage types: class, method, line, basic block; can detect when a single source code line is covered only partially; coverage stats are aggregated at method, class, package, and "all classes" levels. Reports support drill-down, to user-controlled detail depth; HTML reports support source code linking. Does not require access to the source code; can instrument individial .class files or entire .jars (in place, if desired). Runtime overhead of added instrumentation is small (5-20%); memory overhead is a few hundred bytes per Java class.

PMD - Open source static analyzer scans java source for problems static analyzer. Capabilities include scanning for: Empty try/catch/finally/switch statements; Dead code - unused local variables, parameters and private methods; Suboptimal code - wasteful String/StringBuffer usage; Overcomplicated expressions - unnecessary if statements, for loops that could be while loops; Duplicate code - copied/pasted code - could indicate copied/pasted bugs.

Hammurapi - Code review tool for Java (and other languages with latest version) released under the GNU Lesser General Public License. Utilizes a rules engine to infer violations in source code. Doesn't fail on source files with errors, or if some inspectors throw exceptions. Parts of tool can be independently extended or replaced. Can review sources in multiple programming languages, perform cross-language inspections, and generate a consolidated report.

TestNG - A testing framework inspired from JUnit and NUnit; supports JDK 5 Annotations, data-driven testing (with @DataProvider), parameters, distribution of tests on slave machines, plug-ins (Eclipse, IDEA, Maven, etc); embeds BeanShell for further flexibility; default JDK functions for runtime and logging (no dependencies).

Concordian - An open source testing framework for Java developed by David Peterson. Utilizes requirements in plain English using paragraphs, tables and proper punctuation in HTML. Developers instrument the concrete examples in each specification with commands (e.g. "set", "execute", "assertEquals") that allow test scenarios to be checked against the system to be tested. The instrumentation is invisible to a browser, but is processed by a Java fixture class that accompanies the specification. The fixture is also a JUnit test case. Results are exported with the usual green and red indicating successes and failures. Site includes info re similarities and diffs from Fitnesse.

DBUnit - Open source JUnit extension (also usable with Ant) targeted for database-driven projects that, among other things, puts a database into a known state between test runs. Enables avoidance of problems that can occur when one test case corrupts the database and causes subsequent tests to fail or exacerbate the damage. Has the ability to export and import database data to and from XML datasets. Can work with very large datasets when used in streaming mode, and can help verify that database data matches expected sets of values.

StrutsTestCase - Open source Unit extension of the standard JUnit TestCase class that provides facilities for testing code based on the Struts framework, including validation methods. Provides both a Mock Object approach and a Cactus approach to actually run the Struts ActionServlet, allowing testing of Struts code with or without a running servlet engine. Uses the ActionServlet controller to test code, enabling testing of the implementation of Action objects, as well as mappings, form beans, and forwards declarations.

DDSteps - A JUnit extension for building data driven test cases. Enables user to parameterize test cases, and run them more than once using different data. Uses external test data in Excel which is injected into test cases using standard JavaBeans properties. Test cases run once for each row of data, so adding new tests is just a matter of adding a row of data in Excel.

JKool - A light weight performance measurement and monitoring tool from Nastel Inc. for live J2EE, Web and Web service-based applications. It provides timing information for web sessions, including JSP/servlets, JDBC, JMS and Java method calls, to measure performance, detect bottlenecks and failures. Probes include a Web probe (JSP, Servlets), a Java probe (Byte Code Instrumentation), a JMS probe, and a JDBC probe.

StrutsTestCase for JUnit - Open source extension of the standard JUnit TestCase class that provides facilities for testing code based on the Struts framework. Provides both a Mock Object approach and a Cactus approach to actually run the Struts ActionServlet, allowing testing Struts code with or without a running servlet engine. Because it uses the ActionServlet controller to test code, can test not only the implementation of Action objects, but also mappings, form beans, and forwards declarations. Since it already provides validation methods, it's quick and easy to write unit test cases.

JavaNCSS - A free Source Measurement Suite for Java by Clemens Lee. A simple command line utility which collects various source code metrics for Java. The metrics are collected globally, for each class and/or for each function.

Open Source Profilers for Java - Listing of about 25 open source code profilers for Java from 2006 from the Manageability.org web site.

SofCheck Inspector - Tool from SofCheck Inc. for analysis of Java for logic flaws and vulnerabilities. Exlpores all possible paths in byte code and detects flaws and vulnerabilities in areas such as: array index out of bounds, buffer overflows, race conditions, null pointer dereference, dead code, etc. Provides 100% path coverage and can report on values required for 100% unit test coverage. Patented precondition, postcondition and presumption reporting can help detect Malware code insertion.

WindowTester - Test automation tool from Instantiations Inc. for Swing or SWT UI's. Quickly and easily record GUI tests by interacting with the application as you normally would; WindowTester watches your actions and generates a test case automatically. Generated tests are pure Java and easily customized using the power available in the Java language. Provides a rich GUI Test Library, hiding the complexities and threading issues of GUI test execution; test cases are based on the JUnit standard

Squish for Java - Automated Java GUI testing tool for Java Swing, AWT, SWT and RCP/Eclipse applications. Record or create/modify scripts using Tcl, Python, JavaScript. Automatic identification of GUI objects of the AUT; inspect AUT's objects, properties and methods on run-time using the Squish Spy. Can be run via a GUI front-end or via command line tools. Can execute tests in a debugger allowing setting breakpoints and stepping through test scripts.

Klocwork K7 - Static analysis technology for Java, C, C++, analyzes defects & security vulnerabilities, architecture & header file anomalies, metrics. Developers can run Klocwork in Eclipse or various other IDE's. Users can select scope of reporting as needed by selecting software component, defect type, and defect state/status.

Coverity Prevent - Tool from Coverity Inc. for analysis of Java source code for security issues. Explores all possible paths in source code and detects security vulnerabilities and defects in multiple areas: memory leaks, memory corruption, and illegal pointer accesses, buffer overruns, format string errors and SQL injections vulnerabilities, multi-threaded programming concurrency errors, etc.

GUIDancer - Eclipse-based tool from Bredex GmbH for automated testing of Java/Swing GUI's, Tests are specified, not programmed - no code or script is produced. Test specification is initially separate from the AUT, allowing test creation before the software is fully functional or available. Specification occurs interactively; components and actions are selected from menus, or by working with the AUT in an advanced "observation mode". Test results and errors viewable in a results view, can be saved as html or xml file.

CMTJava - Complexity measurement tool from Verifysoft GmbH. Includes McCabe cyclomatic complexity, lines-of-code metrics, Halstead metrics, maintainability index.

JavaCov - A J2SE/J2EE Coverage testing tool from Alvicom; specializes in testing to MC/DC (Modified Condition/Decision Coverage) depth. Capabilities include: Eclipse plugin; report generation into HTML and XML; Apache Ant integration and support for test automation.

Jameleon - Open source automated testing harness for acceptance-level and integration testing, written in Java. Separates applications into features and allows those features to be tied together independently, in XML, creating self-documenting automated test cases. These test-cases can then be data-driven and executed against different environments. Easily extensible via plug-ins; includes support for web applications and database testing.

Agitator - Automated java unit testing tool from Agitar Software. Creates instances of classes being exercised, calling each method with selected, dynamically created sets of input data, and analyzing results. Stores all information in XML files; works with Eclipse and a variety of IDEs

PMD - Open source tool scans Java code for potential bugs, dead code, duplicate code, etc. - works with a variety of configurable and modifiable rulesets. Integrates with a wide variety of IDE's.

JLint - Open source static analysis tool will check Java code and find bugs, inconsistencies and synchronization problems by doing data flow analysis and building the lock graph.

Lint4j - A static Java source and byte code analyzer that detects locking and threading issues, performance and scalability problems, and checks complex contracts such as Java serialization by performing type, data flow, and lock graph analysis. Eclipse, Ant and Maven plugins available.

FindBugs - Open source static analysis tool to inspect Java bytecode for occurrences of bug patterns, such as difficult language features, misunderstood API methods, misunderstood invariants when code is modified during maintenance, garden variety mistakes such as typos, use of the wrong boolean, etc. Can report false warnings, generally less than 50%.

CheckStyle - Open source tool for checking code layout issues, class design problems, duplicate code, bug patterns, and much more.

Java Development Tools - Java coverage, metrics, profiler, and clone detection tools from Semantic Designs.

AppPerfect Test Studio - Suite of testing, tuning, and monitoring products for java development from AppPerfect Corp. Includes: Unit Tester, Code Analyzer, Java/J2EE Profiler and other modules.

GJTester - Java unit, regression, and contract (black box) test tool from TreborSoft. Enables test case and test script development without programming. Test private and protected functions, and server application's modules, without implementing test clients, regression testing for JAVA VM upgrades. Useful for testing CORBA, RMI, and other server technologies as well. GUI interface emphasizing ease of use.

QFTest - A cross-platform system and load testing tool from Quality First Software with support for for Java GUI test automation (Swing, Eclipse/SWT, Webstart, Applets, ULC). Includes small-scale test management capabilities, capture/replay mechanism, intuitive user interface and extensive documentation, reliable component recognition and can handle complex and custom GUI objects, integrated test debugger and customizable reporting.

Cactus - A simple open-source test framework for unit testing server-side java code (Servlets, EJBs, Tag Libs, Filters, etc.). Intent is to allow fine-grained continuous testing of all files making up an application: source code but also meta-data files (such as deployment descriptors, etc) through an in-container approach. It uses JUnit and extends it. Typically use within your IDE, or from the command line, using Ant. From Apache Software Foundation.

JUnitPerf - Allows performance testing to be dynamically added to existing JUnit tests. Enables quick composition of a performance test suite, which can then be run automatically and independent of other JUnit tests. Intended for use where there are performance/scalability requirements that need re-checking while refactoring code. By Mike Clark/Clarkware Consulting, licensed under the BSD License.

Koalog Code Coverage - Code coverage analyzer for Java applications from Koalog SARL. Includes: in-process or remote coverage computation, capability of working directly on Java method binaries (no recompilation), predefined (XML, HTML, LaTex, CSV, TEXT) or custom report generation, and session merging to allow compilation of overall results for distinct executions. Integrates with Ant and JUnit.

Abbot Java GUI Test Framework - Testing framework by Timothy Wall provides automated event generation and validation of Java GUI components, improving upon the very basic functions provided by the java.awt.Robot class. (Abbot = "A Better 'Bot'). The framework may be invoked directly from Java code or accessed without programming through the use of scripts via 'Costello', a script editor/recorder. Suitable for use both by developers for unit tests and QA for functional testing. Free - available under the GNU Lesser General Public License

JUnit - Framework to write repeatable java unit tests - a regression testing framework written by Erich Gamma and Kent Beck. For use by developers implementing unit tests in Java. Free Open Source Software released under the IBM Public License and hosted on SourceForge. Site includes a large collection of extensions and documentation.

jfcUnit - Framework for developing automated testing of Java Swing-based applications at the UI layer (as opposed to testing at lower layers, for which JUnit may be sufficient). Provides recording and playback capabilities. Also available as plugins for JBuilder and Eclipse. Free Open Source Software from SourceForge site.

JBench - Freeware Java benchmarking framework to compare algorithms, virtual machines, etc. for speed. Available as binary distribution (including documentation), source distribution, or jar file.

Clover - Code coverage tool for Java from Cenqua. Fully integrated plugin for NetBeans, JBuilder, and other IDE's. Seamless integration with projects using Apache ANT. View coverage data in XML, HTML, PDF, or via a Swing GUI.

JCover - Java code test coverage analysis tool from Codework Limited. Works with source or compiled files. Gathers coverage measures of branches, statements, methods, classes, file, package and produces reports in multiple formats. Coverage difference comparison between runs. Coverage API provided.

Structure101 - Java source code visualization tool from Headway Software. Lets user understand, measure, and control architecture, design, composition, and dependencies of code base. Analyzes byte code and shows all dependencies, at all levels and between all levels; method, class, package, application. Measures code complexity using a measurement framework called XS. For Windows, Linux and Mac OS X.

Java Tool Suite from Man Machine Systems - Includes JStyle, a Java source analyzer to generate code comments and metrics such as inheritance depth, Cyclomatic Number, Halstead Measures, etc; JPretty reformats Java code according to specified options; JCover test coverage analyzer; JVerify Java class/API testing tool uses an invasive testing model allowing access to internals of Java objects from within a test script and utilizes a proprietary OO scripting language; JMSAssert, a tool and technique for writing reliable software; JEvolve, an intelligent Java code evolution analyzer that automatically analyzes multiple versions of a Java program and shows how various classes have evolved across versions; can 'reason' about selective need for regression testing Java classes; JBrowser class browser; JSynTest, a syntax testing tool that automatically builds a Java-based test data generator.

JProbe Suite - Collection of Java debugging tools from Quest Software; includes JProbe Profiler and JProbe Memory Debugger for finding performance bottlenecks and memory leaks, LProbe Coverage code coverage tool, and JProbe Threadalyzer for finding deadlocks, stalls, and race conditions. JProfiler freeware version available.

Krakatau Professional for Java - Software metrics tool from Power Software includes more than 70 OO, procedural, complexity, and size metrics related to reusability, maintainability, testability, and clarity. Includes Cyclomatic Complexity, Enhanced Cyclomatic Complexity, Halstead Software Science metrics, LOC metrics and MOOD metrics. Has online advisor for quality improvement.

OptimizeIt - Profiler, thread debugger, and code coverage tool suite from Borland (formerly from VMGear).

Jtest - ParaSoft's Jtest is an integrated, automatic unit testing and standards compliance tool for Java. It automatically generates and executes JUnit tests and checks whether code follows 400 coding standards and can automatically correct for many.

DevPartner Java Edition - Compuware's (formerly NuMega) debugging/productivity tool to detect and diagnose Java bugs and memory and performance problems; thread and event analysis, coverage analysis. Integrates with several Java IDE's.

VTune - Intel's performance tuning tool for applications running on Intel processors; includes Java support. Includes suggestions for optimization techniques.

TCAT for Java - Part of Software Research's TestWorks suite of test tools; code coverage analyzer and code analysis for Java; written in Java.

Open Source code analyzers listing - A listing of open source Java code analysis tools written in Java.

Open Source code coverage tools listing - A listing of open source Java code analysis tools written in Java.

Open Source Java test tools listing - A listing of open source tools and frameworks for Java testing, written in Java.

Open Source web test tools listing - A listing of open source web test tools and frameworks written in Java.

(Note: some other tools in these listings also handle testing, management, or load testing of java applets, servlets, and applications, or are planning to add such capabilities. Check listed web sites for current information.)

Load and Performance Test Tools

2008-07-28T08:15:00.000-07:00

Hammerhead - Open source tool by Geoff Wong for generating concurrent http loads. Define test cases in an XML file - specify requests - url, method, body/payloads etc - and verifications. Verification is by matching content to regular expressions and with HTTP status codes. HTTP and HTTPS (SSL) support. Monitor and execute test suites from GUI (wxPython), and adjust load, number of agents, request intervals, rampup time, test duration. Real-time stats and error reporting are displayed.

Pylot - Open source tool by Corey Goldberg for generating concurrent http loads. Define test cases in an XML file - specify requests - url, method, body/payload, etc - and verifications. Verification is by matching content to regular expressions and with HTTP status codes. HTTP and HTTPS (SSL) support. Monitor and execute test suites from GUI (wxPython), and adjust load, number of agents, request intervals, rampup time, test duration. Real-time stats and error reporting are displayed.

JCrawler - An open-source stress-testing tool for web apps; includes crawling/exploratory features. User can give JCrawler a set of starting URLs and it will begin crawling from that point onwards, going through any URLs it can find on its way and generating load on the web application. Load parameters (hits/sec) are configurable via central XML file; fires up as many threads as needed to keep load constant; includes self-testing unit tests. Handles http redirects and cookies; platform independent.

vPerformer - Performance and load testing tool from Verisium Inc. to assess the performance and scalability of web apps. Use recorded scripts or customized scripts using Javascript. Targeted platforms: Windows

Curl-Loader - Open-source tool written in 'C', simulating application load and behavior of tens of thousand HTTP/HTTPS and FTP/FTPS clients, each with its own source IP-address. In contrast to other tools curl-loader is using real C-written client protocol stacks, namely, HTTP and FTP stacks of libcurl and TLS/SSL of openssl. Activities of each virtual client are logged and collected statistics include information about: resolving, connection establishment, sending of requests, receiving responses, headers and data received/sent, errors from network, TLS/SSL and application (HTTP, FTP) level events and errors.

RealityLoad XF On-Demand Load Testing - An on-demand load testing service (no licenses) from Gomez.com. Leverages Gomez' peer panel, which consists of over 15,000 end-user desktop testing locations distributed across the world, to provide distributed load tests that accurately reproduce the network and latency characteristics encountered by real users in a live environment.

OpNet LoadScaler - Load test tool from OpNet Technologies Inc. Create tests without programming; generate loads against web applications, and other services including Web Services, FTP, and Email. Record end-user browser activity in the OPNET TestCreatorTM authoring environment to automatically generate test scripts in industry-standard JavaScript. Modify, extend and debug tests with the included JavaScript editor. Alternatively, drag and drop icons onto the test script tree. No knowledge of a scripting language is required to customize test scripts.

StressTester - Enterprise load and performance testing tool for web applications from Reflective Solutions Ltd. Advanced user journey modeling, scalable load, system resources monitors and results analysis. No scripting required. Suitable for any Web, JMS, IP or SQL Application. OS independent.

The Grinder - A Java load-testing framework freely available under a BSD-style open-source license. Orchestrate activities of a test script in many processes across many machines, using a graphical console application. Test scripts make use of client code embodied in Java plug-ins. Most users do not write plug-ins themselves, instead using one of the supplied plug-ins. Comes with a mature plug-in for testing HTTP services, as well as a tool which allows HTTP scripts to be automatically recorded.

Proxy Sniffer - Web load and stress testing tool from from Ingenieurbüro David Fischer GmbH Capabilities include: HTTP/S Web Session Recorder that can be used with any web browser; recordings can then be used to automatically create optimized Java-based load test programs; automatic protection from "false positive" results by examining actual web page content; detailed Error Analysis using saved error snapshots; real-time statistics.

Testing Master - Load test tool from Novosoft, capabilities include IP spoofing, multiple simultaneous test cases and website testing features for sites with dynamic content and secure HTTPS pages.

Funkload - Web load testing, stress testing, and functional testing tool written in Python and distributed as free software under the GNU GPL. Emulates a web browser (single-threaded) using webunit; https support; produces detailed reports in ReST, HTML, or PDF.

Avalanche - Load-testing appliance from Spirent Communications, designed to stress-test security, network, and Web application infrastructures by generating large quantities of user and network traffic. Simulates as many as two million concurrently-connected users with unique IP addresses, emulates multiple Web browsers, supports Web Services testing Supports HTTP 1.0/1.1, SSL, FTP, RTSP/ RTP, MS Win Media, SMTP, POP3, DNS, Telnet, and Video on Demand over Multicast protocols.

Loadea - Stress testing tool runs on WinXP; free evaluation version for two virtual users. Capture module provides a development environment, utilizes C# scripting and XML based data. Control module defines, schedules, and deploys tests, defines number of virtual users, etc. Analysis module analyzes results and provides reporting capabilities.

LoadManager - Load, Stress, Stability and Performance testing tool from Alvicom. Runs on all platforms supported by Eclipse and Java such as Linux, Windows, HP Unix, and others.

QEngine Web Performance Testing - Automated testing tool from AdventNet for performance testing (load and stress testing) of web applications and web services. For Linux and Windows.

NeoLoad - Load testing tool for web applications from Neotys with clear and intuitive graphical interface, no scripting/fast learning curve, clear and comprehensive reports and test results. Can design complex scenarios to handle real world applications. Features include data replacement, data extraction, SOAP support, system monitoring (Windows, Linux, IIS, Apache, WebLogic, Websphere...), SSL recording, PDF/HTML/Word reporting, IP spoofing, and more. Multi-platform: Windows, Linux, Solaris.

webStress - Load and stress testing service from MoniForce BV. Includes recommendations on how to fix performance-related problems.

Test Complete Enterprise - Automated test tool from AutomatedQA Corp. includes web load testing capabilities.

WebPartner Test and Performance Center - Test tool from WebPartner for stress tests, load performance testing, transaction diagnostics and website monitoring of HTTP/HTTPS web transactions and XML/SOAP/WSDL web services.

QTest - Web load testing tool from Quotium Technologies SA. Capabilities include: cookies managed natively, making the script modelling phase shorter; HTML and XML parser, allowing display and retrieval of any element from a HTML page or an XML flux in test scripts; option of developing custom monitors using supplied APIs; more.

LoadDriver - Load test tool from Inforsolutions emphasizes ease of use; directly drives multiple instances of MSIE, rather than simulating browsers. Supports browser-side scripts/objects, HTTP 1.0/1.1, HTTPS, cookies, cache, Windows authentication. Tests can be scriptlessly parameterized with data from text files or custom ODBC data source, for individual userID, password, page to start, think times, data to enter, links to click, cache, initial cache state, etc.

Test Perspective Load Test - Do-it-yourself load testing service from Keynote Systems for Web applications. Utilizes Keynote's load-generating infrastructure on the Internet; conduct realistic outside-the-firewall load and stress tests to validate performance of entire Web application infrastructure.

SiteTester1 - Load test tool from Pilot Software Ltd. Allows definition of requests, jobs, procedures and tests, HTTP1.0/1.1 compatible requests, POST/GET methods, cookies, running in multi-threaded or single-threaded mode, generates various reports in HTML format, keeps and reads XML formatted files for test definitions and test logs. Requires JDK1.2 or higher.

httperf - Web server performance/benchmarking tool from HP Research Labs. Provides a flexible facility for generating various HTTP workloads and measuring server performance. Focus is not on implementing one particular benchmark but on providing a robust, high-performance, extensible tool. Available free as source code.

NetworkTester - Tool (formerly called 'NetPressure') from Agilent Technologies uses real user traffic, including DNS, HTTP, FTP, NNTP, streaming media, POP3, SMTP, NFS, CIFS, IM, etc. - through access authentication systems such as PPPOE, DHCP, 802.1X, IPsec, as necessary. Unlimited scalability; GUI-driven management station; no scripting; open API. Errors isolated and identified in real-time; traffic monitored at every step in a protocol exchange (such as time of DNS lookup, time to logon to server, etc.). All transactions logged, and detailed reporting available.

WAPT - Web load and stress testing tool from SoftLogica LLC. Handles dynamic content and HTTPS/SSL; easy to use; support for redirects and all types of proxies; clear reports and graphs.

Visual Studio Team System 2008 Test Edition - A suite of testing tools for Web applications and services that are integrated into the Microsoft Visual Studio environment. These enable testers to author, execute, and manage tests and related work items all from within Visual Studio.

OpenLoad - Affordable and completely web-based load testing tool from OpenDemand; knowledge of scripting languages not required - web-based recorder can capture and translate any user action from any website or web application. Generate up to 1000 simultaneous users with minimum hardware.

Apache JMeter - Java desktop application from the Apache Software Foundation designed to load test functional behavior and measure performance. Originally designed for testing Web Applications but has since expanded to other test functions; may be used to test performance both on static and dynamic resources (files, Servlets, Perl scripts, Java Objects, Data Bases and Queries, FTP Servers and more). Can be used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types; can make a graphical analysis of performance or test server/script/object behavior under heavy concurrent load.

TestMaker - Free open source utility maintained by PushToTest.com and Frank Cohen, for performance, scalability, and functional testing of Web application. A framework and utility to build and run intelligent test agents that implement user behaviors and drive the system as users would. Features an XML-based scripting language and library of test objects to create test agents. Includes capability to check and monitor email systems using SMTP, POP3, IMAP protocols. Java-based tool - runs on any platform.

SiteStress - Remote, consultative load testing service by Webmetrics. Simulates end-user activity against designated websites for performance and infrastructure reliability testing. Can generate an infinitely scalable user load from GlobalWatch Network, and provide performance reporting, analysis, and optimization recommendations.

e-Load - Load test tool from Empirix Software; for use in conjunction with test scripts from their e-Tester functional test tool. Allows on-the-fly changes and has real-time reporting capabilities. Includes script editor with advanced debugging and maintenance capabilities. Works with a wide variety of platforms. (Note: (Note: in March 2008 it was announced that Oracle aquired Empirix's e-TEST suite product line and will incorporate it into the Oracle Enterprise Manager and Oracle Real Application Testing products.)

Siege - Open source stress/regression test and benchmark utility; supports basic authentication, cookies, HTTP and HTTPS protocols. Enables testing a web server with a configurable number of concurrent simulated users. Stress a single URL with a specified number of simulated users or stress multiple URL's simultaneously. Reports total number of transactions, elapsed time, bytes transferred, response time, transaction rate, concurrency, and server response. Developed by Jeffrey Fulmer, modeled in part after Lincoln Stein's torture.pl, but allows stressing many URLs simultaneously. Distributed under terms of the GPL; written in C; for UNIX and related platforms.

JBlitz - Load, performance and functional test tool from Clan Productions. Runs multiple concurrent virtual users.to simulate heavy load. Validates each response using plain text or regular expression searches, or by calling out to your own custom code. Full Java API. For testing and 'bullet-proofing' server side software - ASPs, JSPs, servlets, EJBs, Perl / PHP / C / C++ / CGI scripts etc.

WebServer Stress Tool - Web stress test tool from Paessler AG handles proxies, passwords, user agents, cookies, AAL.

Web Polygraph - Freely available benchmarking tool for caching proxies, origin server accelerators, L4/7 switches, and other Web intermediaries. Other features: for high-performance HTTP clients and servers, realistic traffic generation and content simulation, ready-to-use standard workloads, powerful domain-specific configuration language, and portable open-source implementation. C++ source available; binaries avail for Windows.

OpenSTA - 'Open System Testing Architecture' is a free, open source web load/stress testing application, licensed under the Gnu GPL. Utilizes a distributed software architecture based on CORBA. OpenSTA binaries available for Windows.

PureLoad - Java-based multi-platform performance testing and analysis tool from Minq Software. Includes 'Comparer' and 'Recorder' capabilities, dynamic input data, scenario editor/debugger, load generation for single or distributed sources.

ApacheBench - Perl API for Apache benchmarking and regression testing. Intended as foundation for a complete benchmarking and regression testing suite for transaction-based mod_perl sites. For stress-testing server while verifying correct HTTP responses. Based on the Apache 1.3.12 ab code. Available via CPAN as .tar.gz file.

Torture - Bare-bones Perl script by Lincoln Stein for testing web server speed and responsiveness and test stability and reliability of a particular Web server. Can send large amounts of random data to a server to measure speed and response time of servers, CGI scripts, etc.

WebSpray - Low-cost load testing tool from CAI Networks; includes link testing capabilities; can simulate up to 1,000 clients from a single IP address; also supports multiple IP addresses with or without aliases. For Windows.

eValid LoadTest - Web test tool from Software Research, Inc that uses a 'Test Enabled Web Browser' test engine that provides browser based 100% client side quality checking, dynamic testing, content validation, page performance tuning, and webserver loading and capacity analysis.

WebPerformance Load Tester - Load test tool emphasizing ease-of-use, from WebPerformance Inc. Supports all browsers and web servers; records and allows viewing of exact bytes flowing between browser and server; no scripting required. Modem simulation allows each virtual user to be bandwidth limited. Can automatically handle variations in session-specific items such as cookies, usernames, passwords, IP addresses, and any other parameter to simulate multiple virtual users. For Windows, Linux, Solaris, most UNIX variants.

WebSuite - A collection of load testing, capture/playback, and related tools from Technovations for performance testing of web sites. Modules include WebCorder, Load Director, Report Generator, Batch, Manager, and others. WebSizr load testing tool supports authentication, SSL, cookies, redirects. Recorded scripts can be modified manually. For Windows.

FORECAST - Load testing tool from Facilita Software for web, client-server, network, and database systems. Capabilities include proprietary, Java, or C++ scripting; windows browser or network recording/playback. Network traces can also be taken from over 15 third party tracing tools. Virtual user data can be parameterized. Works with a wide variety of platforms.

http-Load - Free load test application from ACME Labs to generate web server loads, from ACME Software. Handles HTTP and HTTPS; for Unix.

QA Center Performance Edition - Compuware's tool for load/stress testing of web, database, and character-based systems. Integration with other Compuware tools provides an in-depth view by monitoring its operating system, database and network components, as well as the application itself. Works with a variety of databases, middleware, ERP.

Microsoft WCAT load test tool - Web load test tool from Microsoft for load testing of MS IIS servers; other MS stress tools also listed.

IBM Rational Performance Tester - Performance testing tool from IBM/Rational; has optional extensions to Seibel applications and SAP Solutions. Supports Windows, Linux and z/OS as distributed controller agents; provides high-level and detailed views of tests.

SilkPerformer - Enterprise-class load-testing tool from Borland (formerly Segue). Can simulate thousands of users working with multiple protocols and computing environments. Allows prediction of behavior of e-business environment before it is deployed, regardless of size and complexity.

Radview's WebLoad - Load testing tool from Radview Software, also available as part of their TestView web testing suite. Basic version available as open source, add-ons available for purchase. Capabilities include over 75 Performance Metrics; can view global or detailed account of transaction successes/failures on individual Virtual Client level, assisting in capturing intermittent errors; allows comparing of running test vs. past test metrics. Test scripting via visual tool or Javascript. Wizard for automating non-GUI-based services testing; DoS security testing.

Loadrunner - HP's (formerly Mercury's) load/stress testing tool for web and other applications; supports a wide variety of application environments, platforms, and databases. Large suite of network/app/server monitors to enable performance measurement of each tier/server/component and tracing of bottlenecks.

ISTQB Certified Tester Foundation Level Section 2 Practice Quiz

2008-07-28T07:42:00.001-07:00

The _____ method is the classic approach to project management, especially to the management of large projects.

agile
waterfall
code and fix
iterative/evolutionary

…life-cycle models can be considered evolutionary or incremental in the sense that a given release grows up around a core of features that are delivered, often on dynamic dates.

True
False

Testing throughout the project in a three-dimensional sense refers to the following dimensions:

Time, Resources, and Risk
Verification, Validation, and Defect Reporting
Time, Organizational, and Cultural
None of the above

What are appraisal costs?

All testing costs and the costs of everything else the company does to look for errors.
Everything the company spends to prevent software and documentation errors.
All costs of coping with errors discovered during development and testing.
All costs of coping with errors discovered, typically by your customers, after the product is released.

The _____ starts testing during the “Fragments Coded: first functionality” development stage?

programmer
end user
tester, programmer’s assistant, someone
A & C

During the component or subsystem testing, testers focus on bugs in constituent pieces of the system.

True
False

Select a reason that does not support the idea of incremental testing:

It is easier to pin down the cause of an error.
The programmer can see internal boundaries in the code that are completely invisible to the outside tester.
Testing of only one module, any errors are either in that module or in a simple program wrote to test it.
The programmer focuses on each module individually, which yields to better test coverage.

Integration testing:

involves testers look for various types of bugs in the entire system, fully integrated.
involves testers looking for bugs in the relationships and interfaces between pairs and components of groups of components in the system under test.
occurs often in a staged fashion.
B & C
A & B

System testing tends toward the _____ area of the testing spectrum.

behavioral
functional
performance
reliable

A test run during System Testing is:

Error recovery.
Terminal handling.
Installation.
Keyboard handling.

Acceptance testing is:

validating end-to-end functionality based upon requirements.
checking whether the software is stable enough to be tested.
testing usability of the systems interfaces.
None of the above.

Acceptance testing is the demonstration that a system meets requirements.

True
False

Which is not indicative of Black Box testing?

Performed by the testing group
Identifies how previous fixes have caused side effects.
Identifies how previous fixes failed.
Fails to identify masked bugs because it skips regression testing.

Functional tests are:

sometimes has the same meaning as behavioral tests.
simultaneously designing, developing, and executing tests.
must be augmented with other test approaches to deal with potentially important quality risks such as performance, load, capacity, and volume.
A & C.

The difference between volume tests and stress tests is:

Testing the audio components of an application vs. testing the breaking point of the system under load.
The study of the program’s ability to deal with the largest task vs. the study of the program’s response to peak bursts of activity.
The ability of a system to sustain load vs. the ability of a system to handle imported fields a large size.
None of the Above

Structural testing is not:

glass box testing.
proper selection of program or subprogram paths.
feeding the component input and examining the output.
exercised during the battery of tests.

As a result of a change in the system under test, a new revision of the system, Sn+1, contains a defect not present in revision Sn, the quality of the system is has progressed:

True
False

In order to reduce the time needed for regression testing you can apply the following method(s):

Automate if you can.
Combine test cases.
Designate some tests for periodic testing.
Narrow focus of your testing scope.

Maintenance testing of the retirement of a system may include:

Operations tests of the changed software.
The testing of data migration.
Archiving if long data-retention periods are required.
B or C.

Acceptance testing may occur at more than just a single test level. With the exception of:

A COTS software product may be acceptance tested when it is installed or integrated.
Acceptance testing of the usability of a component may be done during component testing.
Acceptance testing after a change has been released to the user community.
Acceptance testing of a new functional enhancement may come before system testing.

ISTQB Certified Tester Foundation Level Section 1 Practice Quiz

2008-07-28T07:36:00.000-07:00

Test granularity refers to:

Any way of determining the expected result for a test case.
A quality improvement idea common in software development.
The fineness or coarseness of a test’s focus.
The impact of a bug on the system under test.

The prime benefit of testing is that it results in improved defects

True
False

A bug report is a:

A collection of independent, reusable test cases.
A technical document that describes the various symptoms or failure modes associated with a single bug.
A deliverable that details the strategic approach to a testing effort
A & B

A software error can be described as:

A description of the relationship between two or more variables or set members in which the value of one does not influence the values of others.
Any ill-advised, substandard, or temporary fix applied to an urgent problem in the (often misguided) belief that doing so will keep a project moving forward.
The process in which developers determine the root cause of a bug and identify possible fixes.
A mismatch between the program and its specification.

Select a reason that does not agree with the fact that complete testing is impossible:

The domain of possible inputs is too large to test.
Limited financial resources.
There are too many possible paths through the program to test.
The user interface issues (and thus the design issues) are too complex to completely test.

Testing looks for situations in which a product fails to meet the developer’s expectations in specific areas.

True
False

Select a reason that does not support the idea of using separate test plans for test subprojects that are distinct in one or more ways:

Different resources
Different time periods
Different methodologies
Different objectives
Different audiences

The testing effort begins with:

Test planning
Test case design
Test execution
B & C
A & B

Testing during the design stage involves:

Examining the design documents
Reading drafts of the planning documents
Acceptance or qualification testing
None of the above

A well-designed test system promotes:

Principles
Actions
Resources
Accountability

When testing operating systems or applications, the first step of testing a new build should consist of :

Notifying test lead
Updating requirements
Testing the upgrade/installation procedures
A & B

The general rule of test execution is that you must always create a test procedure that will force the program to use the data you’ve entered and to prove that it is using your data correctly.

True
False

Which is not a goal of writing effective Problem/Bug reports?

Illustrate how to fix the problem
Explain how to reproduce the problem
Analyze the error so you can describe it in a minimum number of steps
Write a report that is complete, easy to understand, and non-antagonistic

Which of the following displays an exit criterion for the test team?

All software released to the test team is accompanied by release notes
The test team has executed the entire planned tests against the application under test.
Twice-weekly bug review meetings (under the Change Control Board) occur until System Test Phase Exit to manage the open bug backlog and bug closure times.
The Development teams have unit-tested all features and bug fixes scheduled for release.

The daily closure period refers to:

The average for all closed bugs, including the current day and all previous days
The amount of bugs opened over a 24 hour period
The average number of days between the opening of a bug and its resolution for all bugs closed on the same day.
None of the Above

Integrity testing involves:

The testing of pseudo code
Performance testing
Alpha testing
The final phase of testing prior to deployment

Testing literature reflects and promotes a strongly held belief that product reliability will not be better if testing is done by a fully independent test agency.

True
False

Select the item(s) that are general testing principles:

Testing shows a presence of defects
Exhaustive testing is impossible
Automation tools can be a great strategy
Absence-of-errors fallacy

Which is not a major task of test implementation and execution:

Develop and prioritizing test cases, creating test data, writing test procedures and optionally, preparing test harness and writing automated test scripts.
Logging the outcome of test execution and recording the identities and versions of the software under test, test tools and testware.
Checking test logs against the exit criteria specified in test planning.
Verifying that the test environment has been set up correctly.

Select the item(s) that compose test objectives:

Finding defects
Gaining confidence about the level of quality and providing information
Preventing tools
Utilization of testware

Download QTP 9.2 with Tutorial

2008-07-22T07:54:00.000-07:00

Dear Friends Click Here and Download

http://rapidshare.com/files/71379877/Quicktest_Professional_v9.2.part1.rar
http://rapidshare.com/files/71389900/Quicktest_Professional_v9.2.part2.rar
http://rapidshare.com/files/71423634/Quicktest_Professional_v9.2.part3.rar
http://rapidshare.com/files/71300660/Quicktest_Professional_v9.2.part4.rar

QTP Tutorial : Learning Recording

2008-07-22T07:53:00.000-07:00

Lets start recording with the mercury sample application FLIGHT.

We will begin with a simple test like the "Hello World" program with which we start learning any programming language.

1) First of all click on Record toolbar button ( or Automation menu --> Record or press F3). when we click on Record, "Record and Run Settings" window opens up. Go to "Windows Applications" tab and choose first option "Record and run test on any open windows based application." and click on ok.

2) Open Flight application (Start -->All programs -->QuickTest Professional --> Sample Applications --> Flight)

3) Type Agent name as "Shree" (You can enter any, but must be 4 characters or more) and enter password as "mercury".

4) In the flight reservation window that opens up:

a) In the Flight Schedule area, in Date of Flight enter tomorrow's date in mm/dd/yyyy format.( I used tabs to move to next fields)
b)In Fly From enter Denver
c)In Fly To enter London
d) Click on Flights button which is on the right hand side of "Fly To".
Let the default option be selected in the Flights Table.
e) Click ok
f)In the name field enter your name. It will fill rest of the required information by itself.
g) Click on insert order and after the order is inserted
h) Click on File --> exit.

5) Now you have recorded your first script. You can click on the expert view tab to see the script which QTP has recorded for you automatically.

6) Click on run from Automation Menu (or press F5) to open up run dialog box.

7) Go to the "Results Location" tab, and below "write run results to", in "new run results folder" radio button- let it be the default option --C:\Program Files\Mercury Interactive\QuickTest Professional\Tests\Test1\Res1(you can also change it)

8) click on ok.

Testing Technology Glossary-L

2008-07-21T10:31:00.002-07:00

Length. Maurice Halstead defined the length of a program to be

Line mask. An EXDIFF statement that permits masking a line or group of lines.
Linear histogram. A dynamically updated linear-style histogram showing accumulating C1 or S1 coverage for a selected module.
Log file. An established or default SMARTS file where all test information is automatically accumulated.
Logical branch. The set of statements in a module which are executed as the result of the evaluation of some predicate (conditional) within the module. The logical branch should be thought of as including the outcome of a conditional operation and the subsequent statement execution up to and including the computation of the value of the next predicate, but not including its evaluation in determining program flow.
Logical units. A logical unit is a concept used for synchronization when differencing two files with the EXDIFF system. A logical unit may be a line of text, a page of text, a CAPBAK screen dump, or the keys (or responses) between marker records in a keysave file.
Loop. A sequence of logical branches in a program that repeats at least one node. See Cycle.

(M,N)-cycle. An M-entry, N-exit cycle in a flow graph. A program is perfectly structured ("pure-structured") if it is composed of loops that involve only (1,1)-cycles. Most real-world programs contain many multi-exit cycles, however. Some studies show that over 99% of programs are non-pure-structured

Testing Technology Glossary-K

2008-07-21T10:31:00.001-07:00

Keysave file. A test script file automatically generated during the CAPBAK's recording session. A keysave file contains a sequence of event statements (including keystrokes, mouse movements and screen captures), which represent user input directed to the AU. When a test is played back, the event statements in the keysave file are regenerated and the AUT executes the previously recorded statements exactly as before.

Kiviat chart. A graphical depiction of the metric results from the METRIC Summary report, where each metric is represented by an axis and results are plotted with reference to user-definable upper and lower bounds. The Kiviat chart quickly identifies the metrics to focus on for a particular program.

Testing Technology Glossary-J

2008-07-21T10:30:00.000-07:00

Java. A programming language, not unlike C++, that is used to program applets that are interpretively executed by Java applet viewers associated with several InterNet browsers. Some say that Java := ((C++)--)++.
Junction node. A junction node within a program directed graph is a node, which has an in-degree of two or greater, and an out-degree of exactly one.

Testing Technology Glossary-I

2008-07-21T10:29:00.002-07:00

Image synchronization. The process by which a playback (e.g. from CAPBAK/X and CAPBAK/MSW) holds back execution until the image or window is redrawn or found.
In-degree. In a directed graph, the number of in-ways for a node.
Incompatible logical branch. Two segments in one program are said to be incompatible if there is no logically feasible execution of the program which will permit both of them to be executed in the same test. See also Essential logical branch.
Independent logical branch pair. A pair of logical branches is (sequentially) independent when there are no assignment actions along the first branch. This changes any of the variables used in the predicate of the second statement.
Infeasible path. A logical branch sequence is logically impossible if there is no collection of setting of the input space relative to the first branch in the sequence, which permits the sequence to execute.
Inspection/review. A process of systematically studying and inspecting programs in order to identify certain types of errors, usually accomplished by human rather than mechanical means.
Instrumentation. The first step in analyzing test coverage, is to instrument the source code. Instrumentation modifies the source code so that special markers are positioned at every logical branch or call-pair or path. Later, during program execution of the instrumented source code, these markers will be tracked and counted to provide data for coverage reports.
Integration Testing. Exposes faults during the process of integration of software components or software units and it is specifically aimed at exposing faults in their interactions. The integration approach could be either bottom-up (using drivers), top-down (using stubs) or a mixture of the two. The bottom up is the recommended approach.
Interface. The informational boundary between two software systems, software system components, elements, or modules.
Invocation point. The invocation point of a module is normally the first statement in the module.
Invocation structure. The tree-like hierarchy that contains a link for invocation of one module by another within a software system.
Iteration level. The level of iteration relative to the invocation of a module. A zero-level iteration characterizes flows with no iteration. A one-level iteration characterizes program flow which involves repetition of a zero-level flow.

Testing Technology Glossary-H

2008-07-21T10:29:00.001-07:00

Halstead metric. A measure of the complexity of computer software that is computed as
History report. A SMARTS summary report which indicates all test outcomes maintained in the log file, providing an overview of test regression throughout the testing process.
Hit report. This report is used by TCAT and S-TCAT to identifies all of the logical branches, call-pairs or paths which were exercised in the present and past tests. It analyzes both the trace file and archive file.
Hotkey window. When recording a test session with CAPBAK/X or CAPBAK/MSW, this window pops up when the hotkey function key is pressed (defaulted to F1). It allows you to issue commands, including inserting comments into the keysave file, to save an image or window for synchronization, to capture an image, a window, or the entire screen, to resume, and to end a recording session.

Testing Technology Glossary-G

2008-07-21T10:28:00.001-07:00

GUI (Graphical User Interface). A interface system, e.g. X11 or Windows '95 that communicates between a user and an application.

Testing Technology Glossary-F

2008-07-21T10:27:00.000-07:00

Filter. A stage in a software process that attempts to identify defects so they can be removed.
Filter Efficiency. The percentage of state-detectable defects vs. the actual average number of defects detected. Typical filter efficiencies range from 10% (not often of much practical use) to 90% (nothing is perfect)>
Flow control. When a terminal emulation program establishes communications with a mainframe application, it establishes flow control to prevent characters being lost. In some cases the mainframe application (or cluster controller) locks out the keyboard. This prevents the user from typing ahead; however, when CAPBAK is being used to record terminal sessions, the user is expected to wait for a response from the mainframe. The user, thus, imposes manual flow control to prevent data from being lost in cases where the keyboard is not locked. When CAPBAK is being run in terminal emulation mode, a record of the manual flow control is stored in the keysave and response files. When CAPBAK is transmitting keys in playback, flow control is maintained by using the information saved in these files. See also Automatic flow control.
Flow graph. The oriented diagram, composed with nodes and edges with arrows, the shows the flow of control in a program. Also called a flow chart or a directed graph.
Formal parameter. For an invocable element of program text, the set of variable names which are assigned value or meaning outside of the program text.
Full report. A METRIC report which indicates a set of metrics for each of the modules in a given source file. See also Complexity report.
Function call. A reference by one program to another through the use of an independent procedure-call or functional-call method. Each function call is the ``tail'' of a caller-callee call-pair.
Function Keys. During a recording session with CAPBAK/X or CAPBAK/MSW, you can issue commands via function keys (i.e. your F1 to F10 keyboard function keys). During a recording session, you can use the function keys to bring up the Hotkey window (see Hotkeywindow), add comments to the keysave file, select an image or window for or the entire screen, pause, resume or terminate the session. CAPBAK/X also has additional function keys that allow you to synchronize on a character string or extract characters froman image or the entire screen. During playback, function keys can be used to slow or to quicken the speed of playback, to insert or to append new keysave records into a keysave file, to pause, to resume or to terminate a playback session.
Functional specifications. A set of behavioral and performance requirements which, in aggregate, determine the functional properties of a software system.
Functional test cases. A set of test case data sets for software which are derived from structural test cases.

Testing Technology Glossary-E

2008-07-21T10:26:00.000-07:00

Edge. In a directed graph, the oriented connection between two nodes.
End-to-end testing. Test activity aimed at proving the correct implementation of a required function at a level where the entire hardware/software chain involved in the execution of the function is available.
Entry logical branch. An entry logical branch is one which has no predecessors, a situation which can occur only at the entrance (i.e., invocation point) of a module.
Entry node. In a program directed graph, a node which has more than one out-way and zero in-ways. An entry node has an in-degree of zero and a non-zero out-degree.
Environment Clause. A clause in the SMARTS ATS file that defines local environment variables that can be used as variables in the activation and evaluation clauses.
Error. A difference between program behavior and specification that renders the program results unacceptable. See Defect.
Essential edges. The set of paths that first include each edge which is on only one of the original set of paths.
Essential logical branch. A logical branch of a program that exists on only one path. Hence, execution of an essential logical branch is required to obtain complete segment (branch) coverage.
Essential paths. The set of paths that include one essential edge, that is an edge that lies on no other path.
Evaluation clause. A clause in the SMARTS' ATS file that specifies how to assess the correctness of a test.
Event synchronization. The process by which a playback (e.g. from CAPBAK/X or CAPBAK/MSW) is forced to wait until an menu opening is completed.
Exception report. A METRIC report which identifies source code procedures that exceed a user-defined metric threshold.
EXDIFF. The extended differencing system, a component of STW/Regression. EXDIFF compares two files and reports the difference between them, and it ignores difference that lie within a user-defined masked area.
Executable statement. A statement in a module which is executable in the sense that it produces object code instructions. A non-executable statement is not the opposite: it may be a declaration. Only comments can be left out without affective program behaviour.
Execution verifier. A system to analyze the execution-time behavior of a test object in terms of the level of testing coverage attained.
Exit logical branch. An exit logical branch is one for which there are no successor logical branches. This occurs only when the consequence of the logical branch is an exit from the module.
Exit node. In a directed graph, a node which has more than one in-way, but has zero out-ways. An exit node has an out-degree of zero and a non-zero in-degree.
Exit structure. The exit structure of a program directed graph is the set of logical branches which, if executed, lead unalterably to termination of program flow without involving subsequent repetition of any logical branches.
Explicit predicate. A program predicate whose formula is displayed explicitly in the program text. For example: a single conditional always involves an explicit program predicate. A predicate is implicit when it is not visible in the source code of the program. An example is a program exception, which can occur at any time.

Technology Glossary-D

2008-07-21T10:25:00.000-07:00

Data Mode. In this mode of execution of keysave files, text is interpreted as saved keystrokes to be played back along with timing information which is inclosed in brackets. The Data mode is supported by CAPBAK/X and CAPBAK/UNIX.
DD-path. See Logical branch.
De-instrumentation. When certain parts of your code have already been tested, you can use TCAT's and S-TCAT's de-instrumentations utilities to exclude those parts from instrumentation. For large programs, this can save time.
Debug. After testing has identified a defect, one "debugs" the software by making certain changes that repair the defect.
Decision node. A node in the program directed graph which corresponds to a decision statement within the program.
Decision statement. A decision statement in a module is one in which an evaluation of some predicate is made, which (potentially) affects the subsequent execution behavior of the module.
Decision-to-decision path. See Logical branch.
Decisional depth. The number of decisions that must take on a particular value prior to arriving at a specified logical branch. "The decisional depth for this logical branch is..."
Defect. Any difference between program specifications and actual program behavior of any kind, whether critical or not. What is reported as causing any kind of software problem.
Deficiency. See Defect.
Delay multiplier. The multiplier used to expand or contract playback rates.
Directed graph. A directed graph consists of a set of nodes which are interconnected with oriented arcs. An arbitrary directed graph may have many entry nodes and many exit nodes. A program directed graph has only one entry and one exit node.
Dynamic analysis. A process of systematically demonstrating properties of programs by a series of constructed executions. The STW/Coverage suite of tools performs dynamic analysis.
Dynamic call-tree display. An organic diagram showing modules and their call-pair structure, where the call-pairs are "animated" based on behavior of the instrumented program being tested.
Dynamic directed graph display. An organic diagram showing the connection between logical branches in a program, where the logical branches are "animated" based on behavior of the instrumented program being tested.

Testing Technology Glossary-C

2008-07-21T10:24:00.001-07:00

C. The programming language C. ANSI standard and K&R C are normally grouped as one language. Certain extensions supported by popular C compilers are also included as normal C.
C++. The C++ object oriented programming language. The current standard is ANSI C++ and/or AT&T C++. Both are supported by TCAT/C++.
C0 coverage. The percentage of the total number of statements in a module that are exercised, divided by the total number of statements present in the module.
C1 coverage. The percentage of logical branches exercised in a test as compared with the total number of logical branches known in a program.
Call graph. The function call tree capability of S-TCAT. This utility show caller-callee relationship of a program. It helps the user to determine which function calls need to be tested further.
Call pair. A connection between two functions in which one function "calls" (references) the other function.
Coding rule. A rule that specifies a particular way in which a program is to be expressed.
Coding style. A general measure of the programming nature of a system; abstractly, the way the programming language is used in a real system.
Combinational flow. Combinational flow is represented by a sequence of logical branches with the property that no logical branch is repeated within the flow.
Command mode. This mode of execution of keysave files allows the user to program the keysave file in order to do conditional execution based on system calls. The other mode of execution is Data Mode. Command mode is supported by CAPBAK/X, and CAPBAK/UNIX.
Complexity. A relative measurement of the ``degree of internal complexity'' of a software system, expressed possibly in terms of some algorithmic complexity measure.
Complexity report. This METRIC report lists all a source code program's encountered procedures and lists Software Science metrics (which are concerned with the "size" of software) and Cyclomatic Complexity measures (which are concerned with the flow of control within the program's code). This report is also referred to as a Full report.
Component. A part of a software system smaller than the entire system but larger than an element.
Conditional playback. Certain STW components incorporate a language that provides for logical operations to control behavior during test execution. E.g. a SMARTS test can involve use of the if or while constructs, as can a CAPBAK script. See also Playback programming.
Configuration file. A file used to declare start-up time parameter values. Usually suffixed as *.rc.
Connected directed graph. A directed graph is connected if there is at least one path from every entry node to every exit node.
Control statement. A statement that involves some predicate operation. For example: an if statement or a while statement.
correctness proof. A mathematical process which demonstrates the consistency between a set of assertions about a program and the properties of the program, when executed in a known environment.
Coverage testing. Coverage testing is concerned with the degree to which test cases exercise or cover the logic (source code) of the software module or unit. It is also a measure of coverage of code lines, code branches and code branch combinations.
Cross-reference. An indication, for a selected symbol, of where instances of that symbol lie in a software system.
Ct coverage. The percentage of independently executable sub-trees of the hierarchical decomposition tree of a program that has been exercised, in terms of all of the possible sub-trees that can be executed for that program.
Cumulative coverage. The test coverage attained by a set of several test runs.
Cumulative report. This TCAT or S-TCAT report charts branch and/or call-pair coverage for the current test cumulatively, and for each module in the total system.
Cycle. A sequence of logical branches that forms a closed loop, so that at least one node is repeated.
Cyclomatic number. A number which assesses program complexity according to a program's flow of control. A program's flow of control is based on the number and arrangement of decision statements within the code. The cyclomatic number of a flow graph can be calculated as follows

Testing Technology Glossary-B

2008-07-21T10:22:00.000-07:00

Back-to-back testing. For software subject to parallel implementation, back-to-back testing is the execution of a test on the similar implementations and comparing the results.

Basis paths. The set of non-iterative paths.
Black-Box testing. A test method where the tester views the program as a black box, that is the test is completely unconcerned about the internal behavior and structure of the program. Rather the tester is only interested in finding circumstances in which the program does not behave according to its specifications. Test data are derived solely from the specifications without taking advantage of knowledge of the internal structure of the program. Black-box testing is performed with the STW/Regression suite of tools.
Bottom-up testing. Testing starts with lower level units. Driver units must be created for units not yet completed, each time a new higher level unit is added to those already tested. Again a set of units may be added to the software system at the time, and for enhancements the software system may be complete before the bottom up tests starts. The test plan must reflect the approach, though. The STW/Coverage suite of tools supports this type of testing.
Built-in testing. Any hardware or software device which is part of an equipment, subsystem of system and which is used for the purpose of testing that equipment, subsystem or system.
Byte mask. A differencing mask used by EXDIFF that specifies to disregard differences based on byte counts.

Testing Technology Glossary - A

2008-07-21T10:10:00.000-07:00

• Acceptance test. Formal tests conducted to determine whether or not a system satisfies its acceptance criteria and to enable the customer to determine whether or not to accept a system. This particular kind of testing is performed with the STW/Regression suite of tools.
• Action statement. A non-decision statement in a program that results in executable code.
• Activation clause. A clause in the SMARTS' ATS file which is comprised of the activation keyword and a sequence of system commands to be performed during test execution.
• Ada. The DoD standard programming language.
• Ancestor node. A node in a STW/Coverage directed graph that lies on some path (i.e., sequence of logical branches) that leads to the specified node.
• apg. All Paths Generator. A TCAT-PATH facility that generates equivalence classes that include all program paths from a directed graph.
• Arc. In a directed graph, the oriented connection between two nodes. This is also referred to as an edge.
• Archive file. A file generated from STW/Coverage's cover, scover or ctcover utility containing test trace information in reduced form.
• ASCII synchronization. The process by which a playback (e.g. from CAPBAK/X) holds back execution until a character string is located.
• ATS. A SMARTS user-designed description file which references a test suite. Test cases are referenced in a hierarchically organized structure and can be supplemented with activation commands comparison arguments, pass/fail evaluation criteria, and system commands. When SMARTS is run on either a X Window or UNIX system, the ATS is written in SMARTS' Description Language (which is similar to C language syntax). The ATS file is written in SMARTS C-Interpreter Language when SMARTS is run on a MS Windows system.
• AUT. Application-under-test.
• Automated Test Script. See ATS.
• Automatic flow control. When CAPBAK/UNIX is being run in terminal emulation record mode, a record of the manual flow control is stored in the keysave file and response file.When CAPBAK/UNIX is transmitting keys in playback mode the flow control is maintained by using the information saved in these files.
• Automated Test Script. See ATS.
• Automatic flow control. When CAPBAK/UNIX is being run in terminal emulation record mode, a record of the manual flow control is stored in the keysave file and response file.When CAPBAK/UNIX is transmitting keys in playback mode the flow control is maintained by using the information saved in these files.

Software Testing FAQs

2008-07-16T10:55:00.001-07:00

1. What is 'Software Testing'?

The operation of a system or application under controlled conditions refers to ‘software testing’. Software testing is carried out, in order to evaluate the results of the operation. In the process of software testing, the controlled conditions would include both normal and abnormal conditions.

While testing applications, testers would intentionally try to make things go wrong in order to determine if application works well and is error-free. Software testing teams usually consist of testers, developers and project managers. At times software testing would also include a part of QA testing.

2. How can World Wide Web sites be tested?

Testing websites can become a long-time effort because websites are client/server applications with browser clients and web servers. While testing web sites, firewalls, internet connections, TCP/IP communications, applications that run in web pages and applications that run on the server side. Testing websites is also a major effort because there are a wide variety of browsers and servers to be considered, multiple protocols and standards, different connection speeds and varied technologies.

The expected load on the server, target audience, performance expected on the client side, security, HTML specifications, standards of page appearance, validation of links, tracking of programs and other features are kept in mind while testing websites.

3. What's a 'test plan'?

A test plan is a document that contains details about the objectives, approach, scope and focus of a software testing project. Test plans are usually created in order to enable people outside the group of software testers to understand more about the project. These test plans should contain enough information to make it useful but not too much information to make it difficult for people outside the project to understand.

Test plans usually include details about the target audience, the objectives of testing, overviews of software products, legal requirements, traceability requirements, testing priorities, database set up requirements and project risk analysis amongst others.

4. What's a 'test case'?

A test case is a document that contains details about an action and the expected response, to verify if a particular feature in an application is working correctly. Test cases are developed early in the development cycle so that software testers can find problems in the design or requirements of a software application.

Test plans usually contains details such as, test case name, test case identifier, test conditions, objectives, steps, input data requirements and the desired results.

5. When should testing be stopped?

With latest software applications becoming more complex, it is difficult to decide when testing should be stopped. However, there are some common factors that can help you decide when testing should be stopped. When test cases are completed, test budgets depleted, beta/alpha period terminated, bug rates fallen and deadlines ended, it is time to stop testing the application.

6. What is 'good code'?

Good code is code that has no bugs, works well, is easily readable and can be maintained with ease. Developers and testers usually follow varied theories, metrics and standards to ensure good code. With the help of pair programming, reviews and code analysis tools, software testers can check applications to check for programs and later enforce standards.

7. What is ‘good design’ in software programs?

‘Design’ in a software program could mean either ‘internal design’ or ‘functional design’. If the overall structure of the software program can be easily modified, understood and maintained, then it can be said that the software program has a good internal design. The features of a good internal design also include status logging capabilities, correct functioning and error-handling capabilities.

Good functional design refers to how the users of the software find the functionality of the design. If the software program meets all the requirements of the end-user, then it means that the software has a good functional design. Most users may have little computer knowledge and may not read the user manual or online-help. In such cases, if the software has a good functional design, then users can use the software with ease.

What are the different types of software testing that can be considered?

The following is a list of the different types of software testing that can be considered

White box testing
Unit testing
Functional testing
Black box testing
System testing
Acceptance testing
End-to-end testing
Integration testing
Mutation testing
Compatibility testing
Regression testing
Usability testing
User acceptance testing
Performance testing
Stress testing
Comparison testing
Alpha testing
Fail over testing
Beta testing
Smoke testing
Install/uninstall testing
Security testing
Load testing
Exploratory testing

Software Testing FAQs

2008-07-16T10:55:00.000-07:00

1. What is 'Software Testing'?

2. How can World Wide Web sites be tested?

3. What's a 'test plan'?

4. What's a 'test case'?

Test plans usually contains details such as, test case name, test case identifier, test conditions, objectives, steps, input data requirements and the desired results.

5. When should testing be stopped?

6. What is 'good code'?

7. What is ‘good design’ in software programs?

What are the different types of software testing that can be considered?

The following is a list of the different types of software testing that can be considered

White box testing
Unit testing
Functional testing
Black box testing
System testing
Acceptance testing
End-to-end testing
Integration testing
Mutation testing
Compatibility testing
Regression testing
Usability testing
User acceptance testing
Performance testing
Stress testing
Comparison testing
Alpha testing
Fail over testing
Beta testing
Smoke testing
Install/uninstall testing
Security testing
Load testing
Exploratory testing

ISTQB study Material

2008-07-16T09:52:00.000-07:00

Question
	NOTE: Only one answer per question
1	We split testing into distinct stages primarily because: a) Each test stage has a different purpose. b) It is easier to manage testing in stages. c) We can run different tests in different environments. d) The more stages we have, the better the testing.
2	Which of the following is likely to benefit most from the use of test tools providing test capture and replay facilities? a) Regression testing b) Integration testing c) System testing d) User acceptance testing
3	Which of the following statements is NOT correct? a) A minimal test set that achieves 100% LCSAJ coverage will also achieve 100% branch coverage. b) A minimal test set that achieves 100% path coverage will also achieve 100% statement coverage. c) A minimal test set that achieves 100% path coverage will generally detect more faults than one that achieves 100% statement coverage. d) A minimal test set that achieves 100% statement coverage will generally detect more faults than one that achieves 100% branch coverage.
4	Which of the following requirements is testable? a) The system shall be user friendly. b) The safety-critical parts of the system shall contain 0 faults. c) The response time shall be less than one second for the specified design load. d) The system shall be built to be portable.
5	Analyse the following highly simplified procedure: Ask: “What type of ticket do you require, single or return?” IF the customer wants ‘return’ Ask: “What rate, Standard or Cheap-day?” IF the customer replies ‘Cheap-day’ Say: “That will be £11:20” ELSE Say: “That will be £19:50” ENDIF ELSE Say: “That will be £9:75” ENDIF Now decide the minimum number of tests that are needed to ensure that all the questions have been asked, all combinations have occurred and all replies given. a) 3 b) 4 c) 5 d) 6
6	Error guessing: a) supplements formal test design techniques. b) can only be used in component, integration and system testing. c) is only performed in user acceptance testing. d) is not repeatable and should not be used.
7	Which of the following is NOT true of test coverage criteria? a) Test coverage criteria can be measured in terms of items exercised by a test suite. b) A measure of test coverage criteria is the percentage of user requirements covered. c) A measure of test coverage criteria is the percentage of faults found. d) Test coverage criteria are often used when specifying test completion criteria.
8	In prioritising what to test, the most important objective is to: a) find as many faults as possible. b) test high risk areas. c) obtain good test coverage. d) test whatever is easiest to test.
9	Given the following sets of test management terms (v-z), and activity descriptions (1-5), which one of the following best pairs the two sets? v – test control w – test monitoring x - test estimation y - incident management z - configuration control 1 - calculation of required test resources 2 - maintenance of record of test results 3 - re-allocation of resources when tests overrun 4 - report on deviation from test plan 5 - tracking of anomalous test results a) v-3,w-2,x-1,y-5,z-4 b) v-2,w-5,x-1,y-4,z-3 c) v-3,w-4,x-1,y-5,z-2 d) v-2,w-1,x-4,y-3,z-5
10	Which one of the following statements about system testing is NOT true? a) System tests are often performed by independent teams. b) Functional testing is used more than structural testing. c) Faults found during system tests can be very expensive to fix. d) End-users should be involved in system tests.
11	Which of the following is false? a) Incidents should always be fixed. b) An incident occurs when expected and actual results differ. c) Incidents can be analysed to assist in test process improvement. d) An incident can be raised against documentation.
12	Enough testing has been performed when: a) time runs out. b) the required level of confidence has been achieved. c) no more faults are found. d) the users won’t find any serious faults.
13	Which of the following is NOT true of incidents? a) Incident resolution is the responsibility of the author of the software under test. b) Incidents may be raised against user requirements. c) Incidents require investigation and/or correction. d) Incidents are raised when expected and actual results differ.
14	Which of the following is not described in a unit test standard? a) syntax testing b) equivalence partitioning c) stress testing d) modified condition/decision coverage
15	Which of the following is false? a) In a system two different failures may have different severities. b) A system is necessarily more reliable after debugging for the removal of a fault. c) A fault need not affect the reliability of a system. d) Undetected errors may lead to faults and eventually to incorrect behaviour.
16	Which one of the following statements, about capture-replay tools, is NOT correct? a) They are used to support multi-user testing. b) They are used to capture and animate user requirements. c) They are the most frequently purchased types of CAST tool. d) They capture aspects of user behaviour.
17	How would you estimate the amount of re-testing likely to be required? a) Metrics from previous similar projects b) Discussions with the development team c) Time allocated for regression testing d) a & b
18	Which of the following is true of the V-model? a) It states that modules are tested against user requirements. b) It only models the testing phase. c) It specifies the test techniques to be used. d) It includes the verification of designs.
19	The oracle assumption: a) is that there is some existing system against which test output may be checked. b) is that the tester can routinely identify the correct outcome of a test. c) is that the tester knows everything about the software under test. d) is that the tests are reviewed by experienced testers.
20	Which of the following characterises the cost of faults? a) They are cheapest to find in the early development phases and the most expensive to fix in the latest test phases. b) They are easiest to find during system testing but the most expensive to fix then. c) Faults are cheapest to find in the early development phases but the most expensive to fix then. d) Although faults are most expensive to find during early development phases, they are cheapest to fix then.
21	Which of the following should NOT normally be an objective for a test? a) To find faults in the software. b) To assess whether the software is ready for release. c) To demonstrate that the software doesn’t work. d) To prove that the software is correct.
22	Which of the following is a form of functional testing? a) Boundary value analysis b) Usability testing c) Performance testing d) Security testing
23	Which of the following would NOT normally form part of a test plan? a) Features to be tested b) Incident reports c) Risks d) Schedule
24	Which of these activities provides the biggest potential cost saving from the use of CAST? a) Test management b) Test design c) Test execution d) Test planning
25	Which of the following is NOT a white box technique? a) Statement testing b) Path testing c) Data flow testing d) State transition testing
26	Data flow analysis studies: a) possible communications bottlenecks in a program. b) the rate of change of data values as a program executes. c) the use of data on paths through the code. d) the intrinsic complexity of the code.
27	In a system designed to work out the tax to be paid: An employee has £4000 of salary tax free. The next £1500 is taxed at 10% The next £28000 is taxed at 22% Any further amount is taxed at 40% To the nearest whole pound, which of these is a valid Boundary Value Analysis test case? a) £1500 b) £32001 c) £33501 d) £28000
28	An important benefit of code inspections is that they: a) enable the code to be tested before the execution environment is ready. b) can be performed by the person who wrote the code. c) can be performed by inexperienced staff. d) are cheap to perform.
29	Which of the following is the best source of Expected Outcomes for User Acceptance Test scripts? a) Actual results b) Program specification c) User requirements d) System specification
30	What is the main difference between a walkthrough and an inspection? a) An inspection is lead by the author, whilst a walkthrough is lead by a trained moderator. b) An inspection has a trained leader, whilst a walkthrough has no leader. c) Authors are not present during inspections, whilst they are during walkthroughs. d) A walkthrough is lead by the author, whilst an inspection is lead by a trained moderator.
31	Which one of the following describes the major benefit of verification early in the life cycle? a) It allows the identification of changes in user requirements. b) It facilitates timely set up of the test environment. c) It reduces defect multiplication. d) It allows testers to become involved early in the project.
32	Integration testing in the small: a) tests the individual components that have been developed. b) tests interactions between modules or subsystems. c) only uses components that form part of the live system. d) tests interfaces to other systems.
33	Static analysis is best described as: a) the analysis of batch programs. b) the reviewing of test plans. c) the analysis of program code. d) the use of black box testing.
34	Alpha testing is: a) post-release testing by end user representatives at the developer’s site. b) the first testing that is performed. c) pre-release testing by end user representatives at the developer’s site. d) pre-release testing by end user representatives at their sites.
35	A failure is: a) found in the software; the result of an error. b) departure from specified behaviour. c) an incorrect step, process or data definition in a computer program. d) a human action that produces an incorrect result.
36	In a system designed to work out the tax to be paid: An employee has £4000 of salary tax free. The next £1500 is taxed at 10% The next £28000 is taxed at 22% Any further amount is taxed at 40% Which of these groups of numbers would fall into the same equivalence class? a) £4800; £14000; £28000 b) £5200; £5500; £28000 c) £28001; £32000; £35000 d) £5800; £28000; £32000
37	The most important thing about early test design is that it: a) makes test preparation easier. b) means inspections are not required. c) can prevent fault multiplication. d) will find all faults.
38	Which of the following statements about reviews is true? a) Reviews cannot be performed on user requirements specifications. b) Reviews are the least effective way of testing code. c) Reviews are unlikely to find faults in test plans. d) Reviews should be performed on specifications, code, and test plans.
39	Test cases are designed during: a) test recording. b) test planning. c) test configuration. d) test specification.
40	A configuration management system would NOT normally provide: a) linkage of customer requirements to version numbers. b) facilities to compare test results with expected results. c) the precise differences in versions of software component source code. d) restricted access to the source code library.

Testometer

2008-07-09T09:59:00.001-07:00

So you must be wondering what is Testometer? Well Testometer is a new term coined by TestingGeek :), TestingGeek uses it to measure your testing skills.

Basic idea behind Testometer is to give you an indication on what areas you can improve upon. A Testometer can be anything, from a simple problem to a puzzle which can help you measure your testing skills against a scale. This scale is nothing but a set of pre identified categories against which your input is checked. For example given a problem what are all the possible categories in which you can write your test cases. Take tests given below to find out more :).

As of now this Testometer is in it's infancy, it will evolve with your feedback & suggestions.

* On Off Test - If you think boundary values has to do with only integer values, think again. This test will make you think differently with respect to boundary values. You will also understand the concept of powerful test case and what king of errors can be expected from developers.

* Line Distance Test - Have you forgotten how to calculate distance between two co-ordinates? Well do not worry, formula is given on the test, but what kind of data would you use to test function which will give you length of the line if you give co-ordinates?

* TicTacToe Test - This test will check your ability to create test cases for the TicTacToe game. It will be very different exercise for you because you will need to think about various conditions in which game can be tested.

* Sine Wave Test - This test will check your knowledge of the boundary value analysis. It also gives you information on how and why output boundaries are also important while testing any application for boundary conditions.

* Triangle Test - Purpose of this test is to assess and give you information on the various classes of data which can be used in any given problem. TestingGeek has seen this question being used in many interviews to check how good test cases you can write for any given problem.

Our special thanks to people who are identifying weakness and defects in Testometer and giving their constant feedback to improve Testometer. As a token of our appreciation, we are mentioning their names in the relevant test where they found defects or helped us in improving test.

Testing Certification-2

2008-07-09T09:53:00.000-07:00

his is certainly a debatable issue, whether certifications are good or bad. People have different opinion about the various certification program and the way its content are organized. In my opinion, certifications are good, because it gives you an opportunity to learn things, which you might not be using.
Doing a test certification might add some value to your profile. Certification could be important because of various reasons. You might want to do certification because

There are not many certified software test engineer and so certification can increase your value in the organization.

* It can add value to your career path.
* You stand out from your peers with your professional certification.
* You get to learn things on which you are not working.
* Certification could also mean that you are serious about software testing as a profession.

There are various certification programs available for the software testing professional. On this page we will not compare programs for their merit demerit. Purpose of this page is to provide you information and links about the certification opportunities available to you.

I am in no way claiming that this list is complete, I am sure there are more options available. If it is not present here, its mean that I am not aware of them and so there will be some more people like me. Why not share this information with all of us?

Certification Opportunity Certification options available to the Test professional can be divided in two categories.

* Subject knowledge
* Tool Knowledge
* Certification Based on the Subject Knowledge

In this category, candidates knowledge of software testing is examined and certification is awarded based on the subject knowledge. In this category, certifications are available for quality control and quality assurance, both the fields. These certifications are also divided according to the professional level and knowledge required. I have divided certifications available based on the controlling institute or organization

* Certifications from Quality Assurance Institute, QAI

Certified Software Tester (CSTE) and Certified Software Quality Analyst (CSQA), both the certifications are very popular in the testing community. CSTE is related to the Quality Control and CSQA is related to the Quality Assurance.

According to the QAI, " Acquiring the designation of Certified Software Tester (CSTE) indicates a professional level of competence in the principles and practices of quality control in the IT profession." Where as for CSQA it indicates "Acquiring the designation of Certified Software Quality Analyst (CSQA) indicates a professional level of competence in the principles and practices of quality assurance in the IT profession"

QAI, also have advance level of certification for people, who have already done CSTE or CSQA. These certifications are called CMST (Certified Manager Of Software Testing) and CMSQ (Certified Manager Of Software Quality).

More information about these certifications can be obtained from official website.

* Certifications from International Software Testing Qualification Board, ISTQB

ISTQB have three level of certifications, all under one brand. Certifications from ISTQB are called ISTQB Certified Tester for Foundation, Advance or Expert level. These certifications are based on a syllabus created by ISTQB.

More information about these certifications can be obtained from official website.

* International Institute for Software Testing, IIST

IIST offers two certification programs, CSTP (Certified Software Test Professional) and CTM (Certified Test Manager). More information about these certifications can be obtained from official website.

Certification Based on the Tool Knowledge
In this category, test tool usage proficiency of the candidate is examined. This is normally conducted by various tool vendors. Vendor certifications are available from vendors like Rational, Mercurry, Seague etc. Some of the vendor certifications are mentioned below.

IBM Rational
Rational, well IBM Rational is probably one of the organization to have tool for all the activities involved in SDLC (Software Development Life Cycle). On this page, I will give information about certifications offered by IBM Rational, which are related to software testing. Surprisingly, IBM has two categories of certifications

* Certification For all candidates

There are two certification available, for IBM Rational Manual Tester (RMT) and IBM Rational Performance Tester (RPT)

IBM Certified Solution Designer - RMT - According to IBM website "This intermediate-level solution designer is an individual with extensive product knowledge who understands how to setup, configure and create a manual testing framework with Rational Manual Tester."

* IBM Certified Solution Designer - RPT - Again from IBM " An intermediate level solution designer is an individual with extensive product knowledge who understands how to use the IBM Rational Performance Tester tool to validate the performance, scalability and reliability of Web-based, SAP, Siebel, or Citrix hosted systems. This person is proficient in creating dynamic tests, developing workload schedules for various performance and load testing scenarios, execute tests with both small and large loads, and evaluating resulting data to measure, analyze and pinpoint factors that affect system performance and other related issues before deployment." Certification For IBM Business Partners, Educational Partners and IBM Employees

I do not think I need to give this information here. If you are not in IBM, you will not care about these certification and if you are in IBM, you already know about it. In case you do not know and would like to know, go here.

Borland Segue

Segue, Well Borland Segue has two categories of certifications.

* Test and Performance Management certifications

Under this category, there are three certifications available. These certification address different aspect of application.
o Functional Test Management Expert - Focus of this certification is on the proficiency level of SilkTest, SilkCentral Test Manager and SilkCentral Issue Manager. More details about the certification can be found here
o Performance Test Management Expert - Focus of this certification is on SilkPerformer, SilkCentral Test Manager and SilkCentral Issue Manager. More details can be found here.
o Application Performance Management Expert - Focus of this certification is on SilkPerformer, SilkCentral Performance Manager and SilkCentral Issue Manager. For more details visit Segue.
* Engineer Certifications

Under this category of certification, emphasis is given on only one tool, instead of complete solution. There are two certifications available under this category.
o Borland Certified SilkTest Engineer - According to the Segue website "his process will ensure that candidates demonstrate a functional knowledge of the product as well as possess the ability to create and run tests to validate accuracy within an application under test." More details here.
o Borland Certified SilkPerformer Engineer - According to the Segue website "This process will ensure that candidates will have demonstrated functionality knowledge of the product, the ability to effectively define, implement and execute load tests, generate reports, and interpret the results to provide accurate conclusions and recommendations." For more detail visit here.

Mercury

Well, HP Mercury in some time. So this should give you idea of the software testing market. All the big names of software testing tool organizations are now part of even bigger organizations. Mercury has divided its certification program in two categories.

* Certified Product Consultant - According to the website "The CPC exam certifies that candidates have demonstrated extensive knowledge and ability with Mercury Interactive products. These exams are designed for mid to senior level professionals with several years of experience in their respective fields. The CPC is a hands-on, lab-based examination using actual Mercury Interactive software. Certifications are organized by product and based on a specific release family of the software." More details about the program can be found here.
* Certified Instructor - According to the website "While the CPC focuses on product knowledge and ability, the CI certifies that a candidate is a skilled instructor with a seasoned knowledge of Mercury Interactive course materials and instructional philosophies. An individual who has achieved a CI is able to download Mercury Interactive training materials from the web site and conduct a training course for products they hold a CPC in."