Monday, July 28, 2008

HTML Validators

RealValidator - Shareware HTML validator based on SGML parser by Liam Quinn. Unicode-enabled, supports documents in virtually any language; supports XHTML 1.0, HTML 4.01, HTML 4.0, HTML 3.2, HTML 3.0, and HTML 2.0 ; extensible - add proprietary HTML DTDs or change the existing ones; fetches external DTDs by HTTP and caches them for faster validation; HTML 3.2 and HTML 4.0 references included as HTML Help. For Windows.

CSE 3310 HTML Validator - HTML syntax checker for Windows from AI Internet Solutions. Supports wide variety of standards; accessibility (508) checking; uppercase/lowercase converter. Free 'lite' version. For Windows.

Link Checking Tools

SiteAnalysis - Hosted service from Webmetrics, used to test and validate critical website components, such as internal and external links, domain names, DNS servers and SSL certificates. Runs as often as every hour, or as infrequent as once a week. Ideal for dynamic sites requiring frequent link checking.

HiSoftware Link Validation Utility - Link validation tool; available as part of the AccVerify Product Line.

ChangeAgent - Link checking and repair tool from Expandable Language. Identifies orphan files and broken links when browsing files; employs a simple, familiar interface for managing files; previews files when fixing broken links and before orphan removal; updates links to moved and renamed files; fixes broken links with an easy, 3-click process; provides multiple-level undo/redo for all operations; replaces links but does not reformat or restructure HTML code. For Windows.

Link Checker Pro - Link check tool from KyoSoft; can also produce a graphical site map of entire web site. Handles HTTP, HTTPS, and FTP protocols; several report formats available. For Windows platforms.

Web Link Validator - Link checker from REL Software checks links for accuracy and availability, finds broken links or paths and links with syntactic errors. Export to text, HTML, CSV, RTF, Excel. Freeware 'REL Link Checker Lite' version available for small sites. For Windows.

Site Audit - Low-cost on-the-web link-checking service from Blossom Software.

Xenu's Link Sleuth - Freeware link checker by Tilman Hausherr; supports SSL websites; partial testing of ftp and gopher sites; detects and reports redirected URL; Site Map; for Windows.

Linkalarm - Low cost on-the-web link checker from Link Alarm Inc.; free trial period available. Automatically-scheduled reporting by e-mail.

Alert Linkrunner - Link check tool from Viable Software Alternatives; evaluation version available. For Windows.

InfoLink - Link checker program from BiggByte Software; can be automatically scheduled; includes FTP link checking; multiple page list and site list capabilities; customizable reports; changed-link checking; results can be exported to database. For Windows. Discontinued, but old versions still available as freeware.

LinkScan - Electronic Software Publishing Co.'s link checker/site mapping tool; capabilities include automated retesting of problem links, randomized order checking; can check for bad links due to specified problems such as server-not-found, unauthorized-access, doc-not-found, relocations, timeouts. Includes capabilities for central management of large multiple intranet/internet sites. Results stored in database, allowing for customizable queries and reports. Validates hyperlinks for all major protocols; HTML syntax error checking. For all UNIX flavors, Windows, Mac.

CyberSpyder Link Test - Shareware link checker by Aman Software; capabilities include specified URL exclusions, ID/Password entries, test resumption at interruption point, page size analysis, 'what's new' reporting. For Windows.

Java Test Tools

EMMA - Open-source toolkit, written in pure Java, for measuring and reporting Java code coverage. Targets support for large-scale enterprise software development while keeping individual developer's work fast and iterative. Can instrument classes for coverage either offline or on the fly (using an instrumenting application classloader); supported coverage types: class, method, line, basic block; can detect when a single source code line is covered only partially; coverage stats are aggregated at method, class, package, and "all classes" levels. Reports support drill-down, to user-controlled detail depth; HTML reports support source code linking. Does not require access to the source code; can instrument individial .class files or entire .jars (in place, if desired). Runtime overhead of added instrumentation is small (5-20%); memory overhead is a few hundred bytes per Java class.

PMD - Open source static analyzer scans java source for problems static analyzer. Capabilities include scanning for: Empty try/catch/finally/switch statements; Dead code - unused local variables, parameters and private methods; Suboptimal code - wasteful String/StringBuffer usage; Overcomplicated expressions - unnecessary if statements, for loops that could be while loops; Duplicate code - copied/pasted code - could indicate copied/pasted bugs.

Hammurapi - Code review tool for Java (and other languages with latest version) released under the GNU Lesser General Public License. Utilizes a rules engine to infer violations in source code. Doesn't fail on source files with errors, or if some inspectors throw exceptions. Parts of tool can be independently extended or replaced. Can review sources in multiple programming languages, perform cross-language inspections, and generate a consolidated report.

TestNG - A testing framework inspired from JUnit and NUnit; supports JDK 5 Annotations, data-driven testing (with @DataProvider), parameters, distribution of tests on slave machines, plug-ins (Eclipse, IDEA, Maven, etc); embeds BeanShell for further flexibility; default JDK functions for runtime and logging (no dependencies).

Concordian - An open source testing framework for Java developed by David Peterson. Utilizes requirements in plain English using paragraphs, tables and proper punctuation in HTML. Developers instrument the concrete examples in each specification with commands (e.g. "set", "execute", "assertEquals") that allow test scenarios to be checked against the system to be tested. The instrumentation is invisible to a browser, but is processed by a Java fixture class that accompanies the specification. The fixture is also a JUnit test case. Results are exported with the usual green and red indicating successes and failures. Site includes info re similarities and diffs from Fitnesse.

DBUnit - Open source JUnit extension (also usable with Ant) targeted for database-driven projects that, among other things, puts a database into a known state between test runs. Enables avoidance of problems that can occur when one test case corrupts the database and causes subsequent tests to fail or exacerbate the damage. Has the ability to export and import database data to and from XML datasets. Can work with very large datasets when used in streaming mode, and can help verify that database data matches expected sets of values.

StrutsTestCase - Open source Unit extension of the standard JUnit TestCase class that provides facilities for testing code based on the Struts framework, including validation methods. Provides both a Mock Object approach and a Cactus approach to actually run the Struts ActionServlet, allowing testing of Struts code with or without a running servlet engine. Uses the ActionServlet controller to test code, enabling testing of the implementation of Action objects, as well as mappings, form beans, and forwards declarations.

DDSteps - A JUnit extension for building data driven test cases. Enables user to parameterize test cases, and run them more than once using different data. Uses external test data in Excel which is injected into test cases using standard JavaBeans properties. Test cases run once for each row of data, so adding new tests is just a matter of adding a row of data in Excel.

JKool - A light weight performance measurement and monitoring tool from Nastel Inc. for live J2EE, Web and Web service-based applications. It provides timing information for web sessions, including JSP/servlets, JDBC, JMS and Java method calls, to measure performance, detect bottlenecks and failures. Probes include a Web probe (JSP, Servlets), a Java probe (Byte Code Instrumentation), a JMS probe, and a JDBC probe.

StrutsTestCase for JUnit - Open source extension of the standard JUnit TestCase class that provides facilities for testing code based on the Struts framework. Provides both a Mock Object approach and a Cactus approach to actually run the Struts ActionServlet, allowing testing Struts code with or without a running servlet engine. Because it uses the ActionServlet controller to test code, can test not only the implementation of Action objects, but also mappings, form beans, and forwards declarations. Since it already provides validation methods, it's quick and easy to write unit test cases.

JavaNCSS - A free Source Measurement Suite for Java by Clemens Lee. A simple command line utility which collects various source code metrics for Java. The metrics are collected globally, for each class and/or for each function.

Open Source Profilers for Java - Listing of about 25 open source code profilers for Java from 2006 from the Manageability.org web site.

SofCheck Inspector - Tool from SofCheck Inc. for analysis of Java for logic flaws and vulnerabilities. Exlpores all possible paths in byte code and detects flaws and vulnerabilities in areas such as: array index out of bounds, buffer overflows, race conditions, null pointer dereference, dead code, etc. Provides 100% path coverage and can report on values required for 100% unit test coverage. Patented precondition, postcondition and presumption reporting can help detect Malware code insertion.

WindowTester - Test automation tool from Instantiations Inc. for Swing or SWT UI's. Quickly and easily record GUI tests by interacting with the application as you normally would; WindowTester watches your actions and generates a test case automatically. Generated tests are pure Java and easily customized using the power available in the Java language. Provides a rich GUI Test Library, hiding the complexities and threading issues of GUI test execution; test cases are based on the JUnit standard

Squish for Java - Automated Java GUI testing tool for Java Swing, AWT, SWT and RCP/Eclipse applications. Record or create/modify scripts using Tcl, Python, JavaScript. Automatic identification of GUI objects of the AUT; inspect AUT's objects, properties and methods on run-time using the Squish Spy. Can be run via a GUI front-end or via command line tools. Can execute tests in a debugger allowing setting breakpoints and stepping through test scripts.

Klocwork K7 - Static analysis technology for Java, C, C++, analyzes defects & security vulnerabilities, architecture & header file anomalies, metrics. Developers can run Klocwork in Eclipse or various other IDE's. Users can select scope of reporting as needed by selecting software component, defect type, and defect state/status.

Coverity Prevent - Tool from Coverity Inc. for analysis of Java source code for security issues. Explores all possible paths in source code and detects security vulnerabilities and defects in multiple areas: memory leaks, memory corruption, and illegal pointer accesses, buffer overruns, format string errors and SQL injections vulnerabilities, multi-threaded programming concurrency errors, etc.

GUIDancer - Eclipse-based tool from Bredex GmbH for automated testing of Java/Swing GUI's, Tests are specified, not programmed - no code or script is produced. Test specification is initially separate from the AUT, allowing test creation before the software is fully functional or available. Specification occurs interactively; components and actions are selected from menus, or by working with the AUT in an advanced "observation mode". Test results and errors viewable in a results view, can be saved as html or xml file.

CMTJava - Complexity measurement tool from Verifysoft GmbH. Includes McCabe cyclomatic complexity, lines-of-code metrics, Halstead metrics, maintainability index.

JavaCov - A J2SE/J2EE Coverage testing tool from Alvicom; specializes in testing to MC/DC (Modified Condition/Decision Coverage) depth. Capabilities include: Eclipse plugin; report generation into HTML and XML; Apache Ant integration and support for test automation.

Jameleon - Open source automated testing harness for acceptance-level and integration testing, written in Java. Separates applications into features and allows those features to be tied together independently, in XML, creating self-documenting automated test cases. These test-cases can then be data-driven and executed against different environments. Easily extensible via plug-ins; includes support for web applications and database testing.

Agitator - Automated java unit testing tool from Agitar Software. Creates instances of classes being exercised, calling each method with selected, dynamically created sets of input data, and analyzing results. Stores all information in XML files; works with Eclipse and a variety of IDEs

PMD - Open source tool scans Java code for potential bugs, dead code, duplicate code, etc. - works with a variety of configurable and modifiable rulesets. Integrates with a wide variety of IDE's.

JLint - Open source static analysis tool will check Java code and find bugs, inconsistencies and synchronization problems by doing data flow analysis and building the lock graph.

Lint4j - A static Java source and byte code analyzer that detects locking and threading issues, performance and scalability problems, and checks complex contracts such as Java serialization by performing type, data flow, and lock graph analysis. Eclipse, Ant and Maven plugins available.

FindBugs - Open source static analysis tool to inspect Java bytecode for occurrences of bug patterns, such as difficult language features, misunderstood API methods, misunderstood invariants when code is modified during maintenance, garden variety mistakes such as typos, use of the wrong boolean, etc. Can report false warnings, generally less than 50%.

CheckStyle - Open source tool for checking code layout issues, class design problems, duplicate code, bug patterns, and much more.

Java Development Tools - Java coverage, metrics, profiler, and clone detection tools from Semantic Designs.

AppPerfect Test Studio - Suite of testing, tuning, and monitoring products for java development from AppPerfect Corp. Includes: Unit Tester, Code Analyzer, Java/J2EE Profiler and other modules.

GJTester - Java unit, regression, and contract (black box) test tool from TreborSoft. Enables test case and test script development without programming. Test private and protected functions, and server application's modules, without implementing test clients, regression testing for JAVA VM upgrades. Useful for testing CORBA, RMI, and other server technologies as well. GUI interface emphasizing ease of use.

QFTest - A cross-platform system and load testing tool from Quality First Software with support for for Java GUI test automation (Swing, Eclipse/SWT, Webstart, Applets, ULC). Includes small-scale test management capabilities, capture/replay mechanism, intuitive user interface and extensive documentation, reliable component recognition and can handle complex and custom GUI objects, integrated test debugger and customizable reporting.

Cactus - A simple open-source test framework for unit testing server-side java code (Servlets, EJBs, Tag Libs, Filters, etc.). Intent is to allow fine-grained continuous testing of all files making up an application: source code but also meta-data files (such as deployment descriptors, etc) through an in-container approach. It uses JUnit and extends it. Typically use within your IDE, or from the command line, using Ant. From Apache Software Foundation.

JUnitPerf - Allows performance testing to be dynamically added to existing JUnit tests. Enables quick composition of a performance test suite, which can then be run automatically and independent of other JUnit tests. Intended for use where there are performance/scalability requirements that need re-checking while refactoring code. By Mike Clark/Clarkware Consulting, licensed under the BSD License.

Koalog Code Coverage - Code coverage analyzer for Java applications from Koalog SARL. Includes: in-process or remote coverage computation, capability of working directly on Java method binaries (no recompilation), predefined (XML, HTML, LaTex, CSV, TEXT) or custom report generation, and session merging to allow compilation of overall results for distinct executions. Integrates with Ant and JUnit.

Abbot Java GUI Test Framework - Testing framework by Timothy Wall provides automated event generation and validation of Java GUI components, improving upon the very basic functions provided by the java.awt.Robot class. (Abbot = "A Better 'Bot'). The framework may be invoked directly from Java code or accessed without programming through the use of scripts via 'Costello', a script editor/recorder. Suitable for use both by developers for unit tests and QA for functional testing. Free - available under the GNU Lesser General Public License

JUnit - Framework to write repeatable java unit tests - a regression testing framework written by Erich Gamma and Kent Beck. For use by developers implementing unit tests in Java. Free Open Source Software released under the IBM Public License and hosted on SourceForge. Site includes a large collection of extensions and documentation.

jfcUnit - Framework for developing automated testing of Java Swing-based applications at the UI layer (as opposed to testing at lower layers, for which JUnit may be sufficient). Provides recording and playback capabilities. Also available as plugins for JBuilder and Eclipse. Free Open Source Software from SourceForge site.

JBench - Freeware Java benchmarking framework to compare algorithms, virtual machines, etc. for speed. Available as binary distribution (including documentation), source distribution, or jar file.

Clover - Code coverage tool for Java from Cenqua. Fully integrated plugin for NetBeans, JBuilder, and other IDE's. Seamless integration with projects using Apache ANT. View coverage data in XML, HTML, PDF, or via a Swing GUI.

JCover - Java code test coverage analysis tool from Codework Limited. Works with source or compiled files. Gathers coverage measures of branches, statements, methods, classes, file, package and produces reports in multiple formats. Coverage difference comparison between runs. Coverage API provided.

Structure101 - Java source code visualization tool from Headway Software. Lets user understand, measure, and control architecture, design, composition, and dependencies of code base. Analyzes byte code and shows all dependencies, at all levels and between all levels; method, class, package, application. Measures code complexity using a measurement framework called XS. For Windows, Linux and Mac OS X.

Java Tool Suite from Man Machine Systems - Includes JStyle, a Java source analyzer to generate code comments and metrics such as inheritance depth, Cyclomatic Number, Halstead Measures, etc; JPretty reformats Java code according to specified options; JCover test coverage analyzer; JVerify Java class/API testing tool uses an invasive testing model allowing access to internals of Java objects from within a test script and utilizes a proprietary OO scripting language; JMSAssert, a tool and technique for writing reliable software; JEvolve, an intelligent Java code evolution analyzer that automatically analyzes multiple versions of a Java program and shows how various classes have evolved across versions; can 'reason' about selective need for regression testing Java classes; JBrowser class browser; JSynTest, a syntax testing tool that automatically builds a Java-based test data generator.

JProbe Suite - Collection of Java debugging tools from Quest Software; includes JProbe Profiler and JProbe Memory Debugger for finding performance bottlenecks and memory leaks, LProbe Coverage code coverage tool, and JProbe Threadalyzer for finding deadlocks, stalls, and race conditions. JProfiler freeware version available.

Krakatau Professional for Java - Software metrics tool from Power Software includes more than 70 OO, procedural, complexity, and size metrics related to reusability, maintainability, testability, and clarity. Includes Cyclomatic Complexity, Enhanced Cyclomatic Complexity, Halstead Software Science metrics, LOC metrics and MOOD metrics. Has online advisor for quality improvement.

OptimizeIt - Profiler, thread debugger, and code coverage tool suite from Borland (formerly from VMGear).

Jtest - ParaSoft's Jtest is an integrated, automatic unit testing and standards compliance tool for Java. It automatically generates and executes JUnit tests and checks whether code follows 400 coding standards and can automatically correct for many.

DevPartner Java Edition - Compuware's (formerly NuMega) debugging/productivity tool to detect and diagnose Java bugs and memory and performance problems; thread and event analysis, coverage analysis. Integrates with several Java IDE's.

VTune - Intel's performance tuning tool for applications running on Intel processors; includes Java support. Includes suggestions for optimization techniques.

TCAT for Java - Part of Software Research's TestWorks suite of test tools; code coverage analyzer and code analysis for Java; written in Java.

Open Source code analyzers listing - A listing of open source Java code analysis tools written in Java.

Open Source code coverage tools listing - A listing of open source Java code analysis tools written in Java.

Open Source Java test tools listing - A listing of open source tools and frameworks for Java testing, written in Java.

Open Source web test tools listing - A listing of open source web test tools and frameworks written in Java.

(Note: some other tools in these listings also handle testing, management, or load testing of java applets, servlets, and applications, or are planning to add such capabilities. Check listed web sites for current information.)

Load and Performance Test Tools

Hammerhead - Open source tool by Geoff Wong for generating concurrent http loads. Define test cases in an XML file - specify requests - url, method, body/payloads etc - and verifications. Verification is by matching content to regular expressions and with HTTP status codes. HTTP and HTTPS (SSL) support. Monitor and execute test suites from GUI (wxPython), and adjust load, number of agents, request intervals, rampup time, test duration. Real-time stats and error reporting are displayed.

Pylot - Open source tool by Corey Goldberg for generating concurrent http loads. Define test cases in an XML file - specify requests - url, method, body/payload, etc - and verifications. Verification is by matching content to regular expressions and with HTTP status codes. HTTP and HTTPS (SSL) support. Monitor and execute test suites from GUI (wxPython), and adjust load, number of agents, request intervals, rampup time, test duration. Real-time stats and error reporting are displayed.

JCrawler - An open-source stress-testing tool for web apps; includes crawling/exploratory features. User can give JCrawler a set of starting URLs and it will begin crawling from that point onwards, going through any URLs it can find on its way and generating load on the web application. Load parameters (hits/sec) are configurable via central XML file; fires up as many threads as needed to keep load constant; includes self-testing unit tests. Handles http redirects and cookies; platform independent.

vPerformer - Performance and load testing tool from Verisium Inc. to assess the performance and scalability of web apps. Use recorded scripts or customized scripts using Javascript. Targeted platforms: Windows

Curl-Loader - Open-source tool written in 'C', simulating application load and behavior of tens of thousand HTTP/HTTPS and FTP/FTPS clients, each with its own source IP-address. In contrast to other tools curl-loader is using real C-written client protocol stacks, namely, HTTP and FTP stacks of libcurl and TLS/SSL of openssl. Activities of each virtual client are logged and collected statistics include information about: resolving, connection establishment, sending of requests, receiving responses, headers and data received/sent, errors from network, TLS/SSL and application (HTTP, FTP) level events and errors.

RealityLoad XF On-Demand Load Testing - An on-demand load testing service (no licenses) from Gomez.com. Leverages Gomez' peer panel, which consists of over 15,000 end-user desktop testing locations distributed across the world, to provide distributed load tests that accurately reproduce the network and latency characteristics encountered by real users in a live environment.

OpNet LoadScaler - Load test tool from OpNet Technologies Inc. Create tests without programming; generate loads against web applications, and other services including Web Services, FTP, and Email. Record end-user browser activity in the OPNET TestCreatorTM authoring environment to automatically generate test scripts in industry-standard JavaScript. Modify, extend and debug tests with the included JavaScript editor. Alternatively, drag and drop icons onto the test script tree. No knowledge of a scripting language is required to customize test scripts.

StressTester - Enterprise load and performance testing tool for web applications from Reflective Solutions Ltd. Advanced user journey modeling, scalable load, system resources monitors and results analysis. No scripting required. Suitable for any Web, JMS, IP or SQL Application. OS independent.

The Grinder - A Java load-testing framework freely available under a BSD-style open-source license. Orchestrate activities of a test script in many processes across many machines, using a graphical console application. Test scripts make use of client code embodied in Java plug-ins. Most users do not write plug-ins themselves, instead using one of the supplied plug-ins. Comes with a mature plug-in for testing HTTP services, as well as a tool which allows HTTP scripts to be automatically recorded.

Proxy Sniffer - Web load and stress testing tool from from Ingenieurbüro David Fischer GmbH Capabilities include: HTTP/S Web Session Recorder that can be used with any web browser; recordings can then be used to automatically create optimized Java-based load test programs; automatic protection from "false positive" results by examining actual web page content; detailed Error Analysis using saved error snapshots; real-time statistics.

Testing Master - Load test tool from Novosoft, capabilities include IP spoofing, multiple simultaneous test cases and website testing features for sites with dynamic content and secure HTTPS pages.

Funkload - Web load testing, stress testing, and functional testing tool written in Python and distributed as free software under the GNU GPL. Emulates a web browser (single-threaded) using webunit; https support; produces detailed reports in ReST, HTML, or PDF.

Avalanche - Load-testing appliance from Spirent Communications, designed to stress-test security, network, and Web application infrastructures by generating large quantities of user and network traffic. Simulates as many as two million concurrently-connected users with unique IP addresses, emulates multiple Web browsers, supports Web Services testing Supports HTTP 1.0/1.1, SSL, FTP, RTSP/ RTP, MS Win Media, SMTP, POP3, DNS, Telnet, and Video on Demand over Multicast protocols.

Loadea - Stress testing tool runs on WinXP; free evaluation version for two virtual users. Capture module provides a development environment, utilizes C# scripting and XML based data. Control module defines, schedules, and deploys tests, defines number of virtual users, etc. Analysis module analyzes results and provides reporting capabilities.

LoadManager - Load, Stress, Stability and Performance testing tool from Alvicom. Runs on all platforms supported by Eclipse and Java such as Linux, Windows, HP Unix, and others.

QEngine Web Performance Testing - Automated testing tool from AdventNet for performance testing (load and stress testing) of web applications and web services. For Linux and Windows.

NeoLoad - Load testing tool for web applications from Neotys with clear and intuitive graphical interface, no scripting/fast learning curve, clear and comprehensive reports and test results. Can design complex scenarios to handle real world applications. Features include data replacement, data extraction, SOAP support, system monitoring (Windows, Linux, IIS, Apache, WebLogic, Websphere...), SSL recording, PDF/HTML/Word reporting, IP spoofing, and more. Multi-platform: Windows, Linux, Solaris.

webStress - Load and stress testing service from MoniForce BV. Includes recommendations on how to fix performance-related problems.

Test Complete Enterprise - Automated test tool from AutomatedQA Corp. includes web load testing capabilities.

WebPartner Test and Performance Center - Test tool from WebPartner for stress tests, load performance testing, transaction diagnostics and website monitoring of HTTP/HTTPS web transactions and XML/SOAP/WSDL web services.

QTest - Web load testing tool from Quotium Technologies SA. Capabilities include: cookies managed natively, making the script modelling phase shorter; HTML and XML parser, allowing display and retrieval of any element from a HTML page or an XML flux in test scripts; option of developing custom monitors using supplied APIs; more.

LoadDriver - Load test tool from Inforsolutions emphasizes ease of use; directly drives multiple instances of MSIE, rather than simulating browsers. Supports browser-side scripts/objects, HTTP 1.0/1.1, HTTPS, cookies, cache, Windows authentication. Tests can be scriptlessly parameterized with data from text files or custom ODBC data source, for individual userID, password, page to start, think times, data to enter, links to click, cache, initial cache state, etc.

Test Perspective Load Test - Do-it-yourself load testing service from Keynote Systems for Web applications. Utilizes Keynote's load-generating infrastructure on the Internet; conduct realistic outside-the-firewall load and stress tests to validate performance of entire Web application infrastructure.

SiteTester1 - Load test tool from Pilot Software Ltd. Allows definition of requests, jobs, procedures and tests, HTTP1.0/1.1 compatible requests, POST/GET methods, cookies, running in multi-threaded or single-threaded mode, generates various reports in HTML format, keeps and reads XML formatted files for test definitions and test logs. Requires JDK1.2 or higher.

httperf - Web server performance/benchmarking tool from HP Research Labs. Provides a flexible facility for generating various HTTP workloads and measuring server performance. Focus is not on implementing one particular benchmark but on providing a robust, high-performance, extensible tool. Available free as source code.

NetworkTester - Tool (formerly called 'NetPressure') from Agilent Technologies uses real user traffic, including DNS, HTTP, FTP, NNTP, streaming media, POP3, SMTP, NFS, CIFS, IM, etc. - through access authentication systems such as PPPOE, DHCP, 802.1X, IPsec, as necessary. Unlimited scalability; GUI-driven management station; no scripting; open API. Errors isolated and identified in real-time; traffic monitored at every step in a protocol exchange (such as time of DNS lookup, time to logon to server, etc.). All transactions logged, and detailed reporting available.

WAPT - Web load and stress testing tool from SoftLogica LLC. Handles dynamic content and HTTPS/SSL; easy to use; support for redirects and all types of proxies; clear reports and graphs.

Visual Studio Team System 2008 Test Edition - A suite of testing tools for Web applications and services that are integrated into the Microsoft Visual Studio environment. These enable testers to author, execute, and manage tests and related work items all from within Visual Studio.

OpenLoad - Affordable and completely web-based load testing tool from OpenDemand; knowledge of scripting languages not required - web-based recorder can capture and translate any user action from any website or web application. Generate up to 1000 simultaneous users with minimum hardware.

Apache JMeter - Java desktop application from the Apache Software Foundation designed to load test functional behavior and measure performance. Originally designed for testing Web Applications but has since expanded to other test functions; may be used to test performance both on static and dynamic resources (files, Servlets, Perl scripts, Java Objects, Data Bases and Queries, FTP Servers and more). Can be used to simulate a heavy load on a server, network or object to test its strength or to analyze overall performance under different load types; can make a graphical analysis of performance or test server/script/object behavior under heavy concurrent load.

TestMaker - Free open source utility maintained by PushToTest.com and Frank Cohen, for performance, scalability, and functional testing of Web application. A framework and utility to build and run intelligent test agents that implement user behaviors and drive the system as users would. Features an XML-based scripting language and library of test objects to create test agents. Includes capability to check and monitor email systems using SMTP, POP3, IMAP protocols. Java-based tool - runs on any platform.

SiteStress - Remote, consultative load testing service by Webmetrics. Simulates end-user activity against designated websites for performance and infrastructure reliability testing. Can generate an infinitely scalable user load from GlobalWatch Network, and provide performance reporting, analysis, and optimization recommendations.

e-Load - Load test tool from Empirix Software; for use in conjunction with test scripts from their e-Tester functional test tool. Allows on-the-fly changes and has real-time reporting capabilities. Includes script editor with advanced debugging and maintenance capabilities. Works with a wide variety of platforms. (Note: (Note: in March 2008 it was announced that Oracle aquired Empirix's e-TEST suite product line and will incorporate it into the Oracle Enterprise Manager and Oracle Real Application Testing products.)

Siege - Open source stress/regression test and benchmark utility; supports basic authentication, cookies, HTTP and HTTPS protocols. Enables testing a web server with a configurable number of concurrent simulated users. Stress a single URL with a specified number of simulated users or stress multiple URL's simultaneously. Reports total number of transactions, elapsed time, bytes transferred, response time, transaction rate, concurrency, and server response. Developed by Jeffrey Fulmer, modeled in part after Lincoln Stein's torture.pl, but allows stressing many URLs simultaneously. Distributed under terms of the GPL; written in C; for UNIX and related platforms.

JBlitz - Load, performance and functional test tool from Clan Productions. Runs multiple concurrent virtual users.to simulate heavy load. Validates each response using plain text or regular expression searches, or by calling out to your own custom code. Full Java API. For testing and 'bullet-proofing' server side software - ASPs, JSPs, servlets, EJBs, Perl / PHP / C / C++ / CGI scripts etc.

WebServer Stress Tool - Web stress test tool from Paessler AG handles proxies, passwords, user agents, cookies, AAL.

Web Polygraph - Freely available benchmarking tool for caching proxies, origin server accelerators, L4/7 switches, and other Web intermediaries. Other features: for high-performance HTTP clients and servers, realistic traffic generation and content simulation, ready-to-use standard workloads, powerful domain-specific configuration language, and portable open-source implementation. C++ source available; binaries avail for Windows.

OpenSTA - 'Open System Testing Architecture' is a free, open source web load/stress testing application, licensed under the Gnu GPL. Utilizes a distributed software architecture based on CORBA. OpenSTA binaries available for Windows.

PureLoad - Java-based multi-platform performance testing and analysis tool from Minq Software. Includes 'Comparer' and 'Recorder' capabilities, dynamic input data, scenario editor/debugger, load generation for single or distributed sources.

ApacheBench - Perl API for Apache benchmarking and regression testing. Intended as foundation for a complete benchmarking and regression testing suite for transaction-based mod_perl sites. For stress-testing server while verifying correct HTTP responses. Based on the Apache 1.3.12 ab code. Available via CPAN as .tar.gz file.

Torture - Bare-bones Perl script by Lincoln Stein for testing web server speed and responsiveness and test stability and reliability of a particular Web server. Can send large amounts of random data to a server to measure speed and response time of servers, CGI scripts, etc.

WebSpray - Low-cost load testing tool from CAI Networks; includes link testing capabilities; can simulate up to 1,000 clients from a single IP address; also supports multiple IP addresses with or without aliases. For Windows.

eValid LoadTest - Web test tool from Software Research, Inc that uses a 'Test Enabled Web Browser' test engine that provides browser based 100% client side quality checking, dynamic testing, content validation, page performance tuning, and webserver loading and capacity analysis.

WebPerformance Load Tester - Load test tool emphasizing ease-of-use, from WebPerformance Inc. Supports all browsers and web servers; records and allows viewing of exact bytes flowing between browser and server; no scripting required. Modem simulation allows each virtual user to be bandwidth limited. Can automatically handle variations in session-specific items such as cookies, usernames, passwords, IP addresses, and any other parameter to simulate multiple virtual users. For Windows, Linux, Solaris, most UNIX variants.

WebSuite - A collection of load testing, capture/playback, and related tools from Technovations for performance testing of web sites. Modules include WebCorder, Load Director, Report Generator, Batch, Manager, and others. WebSizr load testing tool supports authentication, SSL, cookies, redirects. Recorded scripts can be modified manually. For Windows.

FORECAST - Load testing tool from Facilita Software for web, client-server, network, and database systems. Capabilities include proprietary, Java, or C++ scripting; windows browser or network recording/playback. Network traces can also be taken from over 15 third party tracing tools. Virtual user data can be parameterized. Works with a wide variety of platforms.

http-Load - Free load test application from ACME Labs to generate web server loads, from ACME Software. Handles HTTP and HTTPS; for Unix.

QA Center Performance Edition - Compuware's tool for load/stress testing of web, database, and character-based systems. Integration with other Compuware tools provides an in-depth view by monitoring its operating system, database and network components, as well as the application itself. Works with a variety of databases, middleware, ERP.

Microsoft WCAT load test tool - Web load test tool from Microsoft for load testing of MS IIS servers; other MS stress tools also listed.

IBM Rational Performance Tester - Performance testing tool from IBM/Rational; has optional extensions to Seibel applications and SAP Solutions. Supports Windows, Linux and z/OS as distributed controller agents; provides high-level and detailed views of tests.

SilkPerformer - Enterprise-class load-testing tool from Borland (formerly Segue). Can simulate thousands of users working with multiple protocols and computing environments. Allows prediction of behavior of e-business environment before it is deployed, regardless of size and complexity.

Radview's WebLoad - Load testing tool from Radview Software, also available as part of their TestView web testing suite. Basic version available as open source, add-ons available for purchase. Capabilities include over 75 Performance Metrics; can view global or detailed account of transaction successes/failures on individual Virtual Client level, assisting in capturing intermittent errors; allows comparing of running test vs. past test metrics. Test scripting via visual tool or Javascript. Wizard for automating non-GUI-based services testing; DoS security testing.

Loadrunner - HP's (formerly Mercury's) load/stress testing tool for web and other applications; supports a wide variety of application environments, platforms, and databases. Large suite of network/app/server monitors to enable performance measurement of each tier/server/component and tracing of bottlenecks.

ISTQB Certified Tester Foundation Level Section 2 Practice Quiz

  1. The _____ method is the classic approach to project management, especially to the management of large projects.
    1. agile
    2. waterfall
    3. code and fix
    4. iterative/evolutionary

  1. …life-cycle models can be considered evolutionary or incremental in the sense that a given release grows up around a core of features that are delivered, often on dynamic dates.
    1. True
    2. False

  1. Testing throughout the project in a three-dimensional sense refers to the following dimensions:
    1. Time, Resources, and Risk
    2. Verification, Validation, and Defect Reporting
    3. Time, Organizational, and Cultural
    4. None of the above

  1. What are appraisal costs?
    1. All testing costs and the costs of everything else the company does to look for errors.
    2. Everything the company spends to prevent software and documentation errors.
    3. All costs of coping with errors discovered during development and testing.
    4. All costs of coping with errors discovered, typically by your customers, after the product is released.

  1. The _____ starts testing during the “Fragments Coded: first functionality” development stage?
    1. programmer
    2. end user
    3. tester, programmer’s assistant, someone
    4. A & C

  1. During the component or subsystem testing, testers focus on bugs in constituent pieces of the system.
    1. True
    2. False

  1. Select a reason that does not support the idea of incremental testing:
    1. It is easier to pin down the cause of an error.
    2. The programmer can see internal boundaries in the code that are completely invisible to the outside tester.
    3. Testing of only one module, any errors are either in that module or in a simple program wrote to test it.
    4. The programmer focuses on each module individually, which yields to better test coverage.

  1. Integration testing:
    1. involves testers look for various types of bugs in the entire system, fully integrated.
    2. involves testers looking for bugs in the relationships and interfaces between pairs and components of groups of components in the system under test.
    3. occurs often in a staged fashion.
    4. B & C
    5. A & B

  1. System testing tends toward the _____ area of the testing spectrum.
    1. behavioral
    2. functional
    3. performance
    4. reliable

  1. A test run during System Testing is:
    1. Error recovery.
    2. Terminal handling.
    3. Installation.
    4. Keyboard handling.

  1. Acceptance testing is:
    1. validating end-to-end functionality based upon requirements.
    2. checking whether the software is stable enough to be tested.
    3. testing usability of the systems interfaces.
    4. None of the above.

  1. Acceptance testing is the demonstration that a system meets requirements.
    1. True
    2. False

  1. Which is not indicative of Black Box testing?
    1. Performed by the testing group
    2. Identifies how previous fixes have caused side effects.
    3. Identifies how previous fixes failed.
    4. Fails to identify masked bugs because it skips regression testing.

  1. Functional tests are:
    1. sometimes has the same meaning as behavioral tests.
    2. simultaneously designing, developing, and executing tests.
    3. must be augmented with other test approaches to deal with potentially important quality risks such as performance, load, capacity, and volume.
    4. A & C.

  1. The difference between volume tests and stress tests is:
    1. Testing the audio components of an application vs. testing the breaking point of the system under load.
    2. The study of the program’s ability to deal with the largest task vs. the study of the program’s response to peak bursts of activity.
    3. The ability of a system to sustain load vs. the ability of a system to handle imported fields a large size.
    4. None of the Above

  1. Structural testing is not:
    1. glass box testing.
    2. proper selection of program or subprogram paths.
    3. feeding the component input and examining the output.
    4. exercised during the battery of tests.

  1. As a result of a change in the system under test, a new revision of the system, Sn+1, contains a defect not present in revision Sn, the quality of the system is has progressed:
    1. True
    2. False

  1. In order to reduce the time needed for regression testing you can apply the following method(s):
    1. Automate if you can.
    2. Combine test cases.
    3. Designate some tests for periodic testing.
    4. Narrow focus of your testing scope.

  1. Maintenance testing of the retirement of a system may include:
    1. Operations tests of the changed software.
    2. The testing of data migration.
    3. Archiving if long data-retention periods are required.
    4. B or C.

  1. Acceptance testing may occur at more than just a single test level. With the exception of:
    1. A COTS software product may be acceptance tested when it is installed or integrated.
    2. Acceptance testing of the usability of a component may be done during component testing.
    3. Acceptance testing after a change has been released to the user community.
    4. Acceptance testing of a new functional enhancement may come before system testing.

ISTQB Certified Tester Foundation Level Section 1 Practice Quiz

  1. Test granularity refers to:
    1. Any way of determining the expected result for a test case.
    2. A quality improvement idea common in software development.
    3. The fineness or coarseness of a test’s focus.
    4. The impact of a bug on the system under test.

  1. The prime benefit of testing is that it results in improved defects
    1. True
    2. False

  1. A bug report is a:
    1. A collection of independent, reusable test cases.
    2. A technical document that describes the various symptoms or failure modes associated with a single bug.
    3. A deliverable that details the strategic approach to a testing effort
    4. A & B

  1. A software error can be described as:
    1. A description of the relationship between two or more variables or set members in which the value of one does not influence the values of others.
    2. Any ill-advised, substandard, or temporary fix applied to an urgent problem in the (often misguided) belief that doing so will keep a project moving forward.
    3. The process in which developers determine the root cause of a bug and identify possible fixes.
    4. A mismatch between the program and its specification.

  1. Select a reason that does not agree with the fact that complete testing is impossible:
    1. The domain of possible inputs is too large to test.
    2. Limited financial resources.
    3. There are too many possible paths through the program to test.
    4. The user interface issues (and thus the design issues) are too complex to completely test.

  1. Testing looks for situations in which a product fails to meet the developer’s expectations in specific areas.
    1. True
    2. False

  1. Select a reason that does not support the idea of using separate test plans for test subprojects that are distinct in one or more ways:
    1. Different resources
    2. Different time periods
    3. Different methodologies
    4. Different objectives
    5. Different audiences

  1. The testing effort begins with:
    1. Test planning
    2. Test case design
    3. Test execution
    4. B & C
    5. A & B

  1. Testing during the design stage involves:
    1. Examining the design documents
    2. Reading drafts of the planning documents
    3. Acceptance or qualification testing
    4. None of the above

  1. A well-designed test system promotes:
    1. Principles
    2. Actions
    3. Resources
    4. Accountability

  1. When testing operating systems or applications, the first step of testing a new build should consist of :
    1. Notifying test lead
    2. Updating requirements
    3. Testing the upgrade/installation procedures
    4. A & B

  1. The general rule of test execution is that you must always create a test procedure that will force the program to use the data you’ve entered and to prove that it is using your data correctly.
    1. True
    2. False

  1. Which is not a goal of writing effective Problem/Bug reports?
    1. Illustrate how to fix the problem
    2. Explain how to reproduce the problem
    3. Analyze the error so you can describe it in a minimum number of steps
    4. Write a report that is complete, easy to understand, and non-antagonistic

  1. Which of the following displays an exit criterion for the test team?
    1. All software released to the test team is accompanied by release notes
    2. The test team has executed the entire planned tests against the application under test.
    3. Twice-weekly bug review meetings (under the Change Control Board) occur until System Test Phase Exit to manage the open bug backlog and bug closure times.
    4. The Development teams have unit-tested all features and bug fixes scheduled for release.

  1. The daily closure period refers to:
    1. The average for all closed bugs, including the current day and all previous days
    2. The amount of bugs opened over a 24 hour period
    3. The average number of days between the opening of a bug and its resolution for all bugs closed on the same day.
    4. None of the Above

  1. Integrity testing involves:
    1. The testing of pseudo code
    2. Performance testing
    3. Alpha testing
    4. The final phase of testing prior to deployment

  1. Testing literature reflects and promotes a strongly held belief that product reliability will not be better if testing is done by a fully independent test agency.
    1. True
    2. False

  1. Select the item(s) that are general testing principles:
    1. Testing shows a presence of defects
    2. Exhaustive testing is impossible
    3. Automation tools can be a great strategy
    4. Absence-of-errors fallacy

  1. Which is not a major task of test implementation and execution:
    1. Develop and prioritizing test cases, creating test data, writing test procedures and optionally, preparing test harness and writing automated test scripts.
    2. Logging the outcome of test execution and recording the identities and versions of the software under test, test tools and testware.
    3. Checking test logs against the exit criteria specified in test planning.
    4. Verifying that the test environment has been set up correctly.
  2. Select the item(s) that compose test objectives:
    1. Finding defects
    2. Gaining confidence about the level of quality and providing information
    3. Preventing tools
    4. Utilization of testware

Tuesday, July 22, 2008

Download QTP 9.2 with Tutorial

Dear Friends Click Here and Download

http://rapidshare.com/files/71379877/Quicktest_Professional_v9.2.part1.rar
http://rapidshare.com/files/71389900/Quicktest_Professional_v9.2.part2.rar
http://rapidshare.com/files/71423634/Quicktest_Professional_v9.2.part3.rar
http://rapidshare.com/files/71300660/Quicktest_Professional_v9.2.part4.rar

QTP Tutorial : Learning Recording

Lets start recording with the mercury sample application FLIGHT.

We will begin with a simple test like the "Hello World" program with which we start learning any programming language.


1) First of all click on Record toolbar button ( or Automation menu --> Record or press F3). when we click on Record, "Record and Run Settings" window opens up. Go to "Windows Applications" tab and choose first option "Record and run test on any open windows based application." and click on ok.


2) Open Flight application (Start -->All programs -->QuickTest Professional --> Sample Applications --> Flight)


3) Type Agent name as "Shree" (You can enter any, but must be 4 characters or more) and enter password as "mercury".


4) In the flight reservation window that opens up:

a) In the Flight Schedule area, in Date of Flight enter tomorrow's date in mm/dd/yyyy format.( I used tabs to move to next fields)
b)In Fly From enter Denver
c)In Fly To enter London
d) Click on Flights button which is on the right hand side of "Fly To".
Let the default option be selected in the Flights Table.
e) Click ok
f)In the name field enter your name. It will fill rest of the required information by itself.
g) Click on insert order and after the order is inserted
h) Click on File --> exit.


5) Now you have recorded your first script. You can click on the expert view tab to see the script which QTP has recorded for you automatically.


6) Click on run from Automation Menu (or press F5) to open up run dialog box.


7) Go to the "Results Location" tab, and below "write run results to", in "new run results folder" radio button- let it be the default option --C:\Program Files\Mercury Interactive\QuickTest Professional\Tests\Test1\Res1(you can also change it)


8) click on ok.

Monday, July 21, 2008

Testing Technology Glossary-L

Length. Maurice Halstead defined the length of a program to be

  • Line mask. An EXDIFF statement that permits masking a line or group of lines.
  • Linear histogram. A dynamically updated linear-style histogram showing accumulating C1 or S1 coverage for a selected module.
  • Log file. An established or default SMARTS file where all test information is automatically accumulated.
  • Logical branch. The set of statements in a module which are executed as the result of the evaluation of some predicate (conditional) within the module. The logical branch should be thought of as including the outcome of a conditional operation and the subsequent statement execution up to and including the computation of the value of the next predicate, but not including its evaluation in determining program flow.
  • Logical units. A logical unit is a concept used for synchronization when differencing two files with the EXDIFF system. A logical unit may be a line of text, a page of text, a CAPBAK screen dump, or the keys (or responses) between marker records in a keysave file.
  • Loop. A sequence of logical branches in a program that repeats at least one node. See Cycle.
(M,N)-cycle. An M-entry, N-exit cycle in a flow graph. A program is perfectly structured ("pure-structured") if it is composed of loops that involve only (1,1)-cycles. Most real-world programs contain many multi-exit cycles, however. Some studies show that over 99% of programs are non-pure-structured

Testing Technology Glossary-K

Keysave file. A test script file automatically generated during the CAPBAK's recording session. A keysave file contains a sequence of event statements (including keystrokes, mouse movements and screen captures), which represent user input directed to the AU. When a test is played back, the event statements in the keysave file are regenerated and the AUT executes the previously recorded statements exactly as before.

  • Kiviat chart. A graphical depiction of the metric results from the METRIC Summary report, where each metric is represented by an axis and results are plotted with reference to user-definable upper and lower bounds. The Kiviat chart quickly identifies the metrics to focus on for a particular program.

Testing Technology Glossary-J

  • Java. A programming language, not unlike C++, that is used to program applets that are interpretively executed by Java applet viewers associated with several InterNet browsers. Some say that Java := ((C++)--)++.
  • Junction node. A junction node within a program directed graph is a node, which has an in-degree of two or greater, and an out-degree of exactly one.

Testing Technology Glossary-I

  • Image synchronization. The process by which a playback (e.g. from CAPBAK/X and CAPBAK/MSW) holds back execution until the image or window is redrawn or found.
  • In-degree. In a directed graph, the number of in-ways for a node.
  • Incompatible logical branch. Two segments in one program are said to be incompatible if there is no logically feasible execution of the program which will permit both of them to be executed in the same test. See also Essential logical branch.
  • Independent logical branch pair. A pair of logical branches is (sequentially) independent when there are no assignment actions along the first branch. This changes any of the variables used in the predicate of the second statement.
  • Infeasible path. A logical branch sequence is logically impossible if there is no collection of setting of the input space relative to the first branch in the sequence, which permits the sequence to execute.
  • Inspection/review. A process of systematically studying and inspecting programs in order to identify certain types of errors, usually accomplished by human rather than mechanical means.
  • Instrumentation. The first step in analyzing test coverage, is to instrument the source code. Instrumentation modifies the source code so that special markers are positioned at every logical branch or call-pair or path. Later, during program execution of the instrumented source code, these markers will be tracked and counted to provide data for coverage reports.
  • Integration Testing. Exposes faults during the process of integration of software components or software units and it is specifically aimed at exposing faults in their interactions. The integration approach could be either bottom-up (using drivers), top-down (using stubs) or a mixture of the two. The bottom up is the recommended approach.
  • Interface. The informational boundary between two software systems, software system components, elements, or modules.
  • Invocation point. The invocation point of a module is normally the first statement in the module.
  • Invocation structure. The tree-like hierarchy that contains a link for invocation of one module by another within a software system.
  • Iteration level. The level of iteration relative to the invocation of a module. A zero-level iteration characterizes flows with no iteration. A one-level iteration characterizes program flow which involves repetition of a zero-level flow.