Gabe Shanahan, Software Quality Engineer I

Hello, fellow QA people.

As we continue to strive for excellence in our software testing for all the current (and future) projects at Acato, it is crucial to revisit one of the cornerstones of our craft: defect reporting. An effective defect report is not just a formality but a critical communication tool that ensures issues are addressed promptly and accurately. Let’s dive into the best practices that make our defect reports clear, actionable, and efficient. To preface this: all this information was obtained from and can be found in the “Defect Reporting” section of the G2 Docs for everyone’s reference.

The Anatomy of a Defect Report

  1. Title and Summary
  • Title: Be concise yet descriptive. A good title quickly conveys the essence of the defect.
  • Summary: Provide a brief overview of the defect, highlighting the key issue. This helps in quickly grasping the problem, especially for POs who may be looking at multiple defects at one time.
  1. Steps to Reproduce
  • Detailed Steps: Outline the exact steps taken to encounter the defect. Each step should be clear and unambiguous to ensure it can be replicated consistently.
  • Preconditions: Mention any specific conditions or data required before starting the steps. This might include user permissions, environment setups, or specific configurations such as impersonation of a specific user/role.
  1. Expected vs. Actual Results
  • Expected Result: Clearly describe what should happen if the application works correctly.
  • Actual Result: Detail what actually happened, including any error messages or anomalies observed.
  1. Required Statuses and Fields in Rally
  • Project (Agile Team): Specify the Agile team associated with the defect.
  • State: Indicate the current state of the defect (open, closed, passed).
  • Schedule State: Set to ‘Defined’ at the time of creation.
  • Release: Should always be the current working release.
  • Iteration: Should always be the current working iteration.
  • Environment: The environment where the defect was found.
  • Furthest Environment: The furthest environment to which the defect has been promoted (e.g., DEV, Staging, QA-Test/Prod).
  • Resolution: The plan to resolve the defect, determined by a conversation with the product owner and the team. Options include:
    • Undetermined
    • Resolving Now
    • Let Escape, Resolve This PI
    • PO to Convert to Feature
    • Don’t Resolve
    • Converted
  • Severity: Categorize the defect based on probability and consequence:
    • High probability, High consequence
    • High probability, Low consequence
    • Low probability, High consequence
    • Low probability, Low consequence
  • Issue Type: Usually ‘Bug’ but can be ‘Question’ if applicable.
  • Owner: Assign to the Product Owner if the defect pertains to one subsystem; if it involves multiple subsystems, assign to the QA Director to coordinate with the applicable PO’s.
  • Relevant Screenshots or Videos: Attach visual evidence to help demonstrate the issue and to aid in your description.

Best Practices for Effective Defect Reporting

  1. Be Precise and Objective: Stick to the facts and avoid subjective language. Clear, objective descriptions help in understanding the defect without ambiguity.
  2. Reproduce Before Reporting: Always attempt to reproduce the defect at least twice to confirm its consistency. This step ensures you are not reporting transient issues.
  3. Use Simple Language: Avoid technical jargon that might confuse stakeholders who are not familiar with specific development terms and/or error codes. Aim for clarity and simplicity. The goal is to be able to communicate the defect to as wide an audience as possible, while simultaneously being as specific about the problem as possible.
  4. Stay Neutral: Focus on describing the defect without assigning blame. The goal is to resolve issues collaboratively.
  5. Keep Updates Flowing: If there are changes or additional findings related to the defect, update the report accordingly. Timely updates can prevent redundant efforts and miscommunication.

Real-World Examples (not project specific)

  • Example 1: Poor Defect Report
    • Title: “Button not working”
    • Summary: “The submit button does not work.”
    • Steps: Not provided.
    • Expected vs. Actual: “The button should work, but it isn’t working.”
    • Environment: Not provided.
    • Attachments: None.
  • Example 2: Excellent Defect Report
  • Title: “Submit Button Fails to Respond on Checkout Page”
  • Summary: “The submit button on the checkout page does not respond when clicked, preventing order submission.”
  • Steps:
  1. Navigate to the checkout page.
  2. Fill in the required billing and shipping information.
  3. Click the ‘Submit’ button.
  • Expected Result: “Order confirmation page should be displayed.”
  • Actual Result: “Nothing happens when the submit button is clicked. No error message displayed.”
  • Required Statuses and Fields in Rally:
    • Project (Agile Team): “Checkout Team”
    • State: “Open”
    • Schedule State: “Defined”
    • Release: “37.x”
    • Iteration: “37.x Iteration 3”
    • Environment: “Team Environment”
    • Furthest Environment: “Staging”
    • Resolution: “Resolving Now”
    • Severity: “High probability – High consequence”
    • Issue Type: “Bug”
    • Owner: “Checkout Module PO”
    • Relevant Screenshots or Videos: “Screenshot of the checkout page with the console log showing no errors.”

By adhering to these practices, we can ensure our defect reports are both comprehensive and effective, facilitating smoother and faster resolution. Let’s continue to elevate our defect reporting standards and maintain our reputation for delivering high-quality software.

Happy Testing!

Gabriel Shanahan, G2 QA Technical Contributor