Skip to content

Unit Testing 3

1. Combinatorial Testing

Combinatorial testing addresses the challenge that program behavior is often affected by many interacting factors, such as input parameters, environment configurations (global variables), and state variables.

  • Motivation: It is often impractical to test every possible combination of values. For example, a font dialog box with 10 effects would require $2^{10}$ (1,024) tests for just those settings, which becomes overwhelming when combined with other parameters like font type and size.
  • Decision Table-Based Testing (DTT):
    • Applicability: Best for requirements involving complex conditional actions.
    • Structure: Consists of Conditions (inputs), Actions (outputs), and Rules (individual test situations/columns).
    • Guidelines: Rules must be complete (all combinations present) and consistent (no conflicting actions for the same condition set). Pasted image 20260424220609.png
  • Cause-Effect Graphs: These graphs represent the logical relationship between specific input combinations (causes) and their resulting outputs (effects). They include constraints and intermediate nodes to link causes to effects. Pasted image 20260424184337.png
  • Pair-wise Testing (2-Way Combinatorial):
    • Principle: Every possible pair of input parameter values must occur at least once in the test suite.
    • Rationale: Most software defects are caused by the interaction of two parameters rather than three or more simultaneously.
    • Benefit: Dramatically reduces the number of test cases while remaining highly effective at fault detection.
  • Fault Models:
    • Single-mode: Bug occurs regardless of other settings (e.g., a print option always fails).
    • Double-mode: Bug occurs only when two specific options are combined.
    • Multi-mode: Bug requires three or more settings to trigger.

2. White-Box Testing (Structural Testing)

White-box testing designs test cases based on the internal code structure of the program.

  • Complementary Nature: White-box testing can find "extra" functionality not in the spec or uncover unreachable code, whereas black-box testing ensures the program matches the specification.
  • Testing Strategies:
    • Coverage-based: Designing cases to execute specific program elements (e.g., every statement).
    • Fault-based: Designing cases to expose specific categories of faults (e.g., mutation testing).

3. Types of Program Element Coverage

Week 11 details several "adequacy criteria" used to measure how thoroughly a program is tested:

  • Statement Coverage:
    • Goal: Every statement in the program is executed at least once.
    • Rationale: An error in a statement cannot be found unless that statement is executed.
  • Branch (Decision) Coverage:
    • Goal: Each branch (True and False outcomes of every decision) is executed at least once.
    • Strength: Branch coverage is stronger than statement coverage because traversing every edge in a control flow graph ensures every node (statement) is visited.
  • Condition Coverage:
    • Basic Condition Testing (BCT): Every basic Boolean condition in a decision statement must evaluate to both True and False at least once.
    • Issue: BCT does not necessarily imply branch coverage. For example, in an OR statement, if the first condition is True, the second may never be evaluated due to short-circuit evaluation.
  • Multiple Condition Coverage (MCC):
    • Goal: All possible combinations of truth values for atomic conditions are tested.
    • Drawback: It is exponential ($2^n$) and only practical if the number of conditions ($n$) is very small.

4. Subsumption Hierarchy

Testing techniques can be ranked by their relative "strength," where a stronger technique covers all elements of a weaker one plus more:

  1. Multiple Condition (Strongest)
  2. Condition/Decision
  3. Decision (Branch)
  4. Statement

  5. Basic Condition (Weakest)

  6. Combinatorial Testing (Decision-Table, Cause-Effect Graph, Pairwise Programming)

  7. White Box Testing (Coverage based and Fault based)