Software Testing 5 - Code Review and Defects

Code review

code review / inspection對找出defects很有幫助，不過實務上似乎很少進行。通常能透過code review找到的defects如下:

1. 缺少或是錯誤的comments
2. 重複性的code
3. 過於複雜的設計
4. 無法測試的路徑
5. 錯誤使用技術
6. coding errors (這啥?!)

 細分的話，其實code review還可以分成以下三類:
1. walkthru:
找個senior的幫你看一些部分，稱為peer review。

2. code review:
這是最常進行的，可能牽涉多個reviewers，主要檢查implementation。


3. inspection
這是最正式的code review流程，可以有以下腳色:

Moderator – works with author to schedule inspection, send out materials, runs meeting, follows up with author on follow-ups from inspection (often team lead or project manager)

·         Author – provides moderator with inspection material, explains material as needed, responds to questions, responsible for rework based on inspection comments

·         Reader – walks though the code, saying what it does in English and stops for comments on that piece

·         Reviewer – technical experts for the document being inspected; provides comments and questions

·         Recorder – one of the reviewers who agrees to document issues raised at the meeting; usually compiles a list with categories (defect, suggestion, comment, etc.) and distributes to all after meeting; opens internal defect reports to help track fixes

Defect Management

俗稱bug XD，就是不按照design發生的行為。

defect不一定能找出來，找出來也不一定要或能修改，修改後也不一定正確。
所以必須要有系統性的方法來面對defects，需要一個defect tracking system。

defect 可以有類似以下的tracking status:

Defect Report

defect reports不只是展現QA人員的專業，也是developers的重要資產。
defect report需要包含以下:
1. isolation: 切割出可以復現的操作方法
2. generalization: 即便切割出來，但仍需要考量此defect對整體性的影響
3. severity: bug分級定義，根據發生機率以及衝擊性

綜合以上，一個可能的defect report template長這樣:

Item	Description
Title	A unique, concise and descriptive title for a defect is vital. It will allow the defect to be easily identified and discussed. Good : “Closed” field in “Account Closure” screen accepts invalid date Bad : “Closed field busted”
Severity	An assessment of the impact of the defect on the end user
Status	The current status of the defect
Initial configuration	The state of the program before the actions in the “steps to reproduce” is to be followed. All too often this is omitted and the reader must guess or intuit the correct pre-requisites for reproducing the defect.
Software Configuration	The version and release of software-under-test as well as any relevant hardware or software platform details (e.g. Win 7 vs Win Vista)
Steps to Reproduce	An ordered series of steps to reproduce the defect Good : 1. Enter “Account Closure” screen 2. Enter an invalid date such as “54/01/07” in the “Closed” field 3. Click “Okay” Bad: If you enter an invalid date in the closed field it accepts it!
Expected behavior	What was expected of the software, upon completion of the steps to reproduce. Good: The functional specification states that the “Closed” field should only accept valid dates in the format “dd/mm/yy” Bad: The field should accept proper dates.
Actual behavior	What the software actually does when the steps to reproduce are followed. Good: Invalid dates (e.g. “54/01/07”) and dates in alternate formats (e.g. “07/01/54”) are accepted and no error message is generated. Bad: Instead it accepts any kind of date.
Impact	An assessment of the impact of the defect on the software-under-test. It is important to include something beyond the simple “severity” to allow readers to understand the context of the defect report. Good: An invalid date will cause the month-end “Account Closure” report to crash the system and corrupt all affected customer records. Bad: This is serious dude!
(Proposed solution)	An optional item of information testers can supply is a proposed solution. Testers often have unique and detailed information of the products they test and suggesting a solution can save designers and developers a lot of time.
Priority	An optional field to allow development managers to assign relative priorities to defects of the same severity
Root Cause	An optional field allowing developers to assign a root cause to the defect such as “inadequate requirements” or “coding error”

Root Cause Analysis

root cause analysis通常有助於降低未來defect發生的可能性，可能的種類如下:

Classification	Description
Requirements	The defect was caused by an incomplete or ambiguous requirement with the result assumption differing from the intended outcome.
Design Error	The design differs from the stated requirements or is ambiguous or incomplete resulting in assumptions.
Code Error	The code differs from the documented design or requirements or there was a syntactic or structural error introduced during coding.
Test Error	The test as designed was incorrect (deviating from stated requirements or design) or was executed incorrectly or the resultant output was incorrectly interpreted by the tester, resulting in a defect “logged in error”.
Configuration	The defect was caused by incorrectly configured environment or data.
Existing bug	The defect is existing behavior in the current software (this does not determine whether or not it is fixed).