Which behavior is missing from the trial evidence?
Check that the run records cover the current build and every required behavior.
The question
A team reviews a candidate mobile robot build after a mock trial. The test matrix covers three job behaviors across open and busy layouts. Each layout and behavior combination requires 2 distinct passing run IDs on the candidate build.
| Open layout | Busy layout | |
|---|---|---|
| Start a job | 2 | 2 |
| Retry a job | 2 | 2 |
| Cancel a job | 2 | 2 |
The dashboard does not show run IDs or build versions.
“Would you tell the review owner that the mock trial evidence is complete?”
Not saved by this site. Your browser may restore text when you navigate back.
A strong answer, step by step
Open each step in turn. This is one strong answer, not the only one.
Step 1: Read the coverage rule
What would you do here? Open the move you’d make.
Approve the evidence because the 12 rows match the total.
Tempting, but it misses something. Duplicate runs or runs from another build can appear in the same total.
Report the apparent coverage and request the run records.
The strong move. The matrix needs distinct records in each cell. A row total alone does not establish coverage.
See how a strong candidate says it
“There are six layout and behavior combinations. At two distinct runs each, the brief needs 12 passing records. The report shows 12 rows, so I’d check the run IDs and builds.”
Why it works: The matrix needs distinct records in each cell. A row total alone does not establish coverage.
Step 2: Request the run record
What would you do here? Open the move you’d make.
Check run identity, build and observed result per cell.
The strong move. The record must cover the behavior that the matrix names. A successful start gives no evidence about cancellation.
Combine start and cancel passes to simplify the count.
Tempting, but it misses something. They describe different robot behaviors. The brief gives each its own cell.
See how a strong candidate says it
“I’d ask for the run ID, build version and observed behavior in each cell. For cancellation, I’d check whether the recorded mock job stopped moving its object after cancellation.”
Why it works: The record must cover the behavior that the matrix names. A successful start gives no evidence about cancellation.
Step 3: Reconcile the rows
The 2 busy retry rows share 1 run ID. The 2 busy cancel rows are from the previous build. All other rows are distinct passes on the candidate build.
What would you do here? Open the move you’d make.
Remove the previous-build rows and count both busy retry rows.
Tempting, but it misses something. Both retry rows are the same run. That cell still needs a second distinct pass.
Report the two incomplete cells and the exact shortfall in each.
The strong move. The duplicate adds no distinct run. The previous-build records do not count for the candidate build.
See how a strong candidate says it
“There are 9 valid runs and 4 complete cells. Busy retry has one distinct pass and needs 1 more. Busy cancel has zero passes for this build and needs 2.”
Why it works: The duplicate adds no distinct run. The previous-build records do not count for the candidate build.
Step 4: Name the missing evidence
What would you do here? Open the move you’d make.
Add three open-layout starts to reach 12 valid records.
Tempting, but it misses something. Open-layout starts do not cover busy retry or busy cancellation.
Ask for evidence in the two incomplete cells.
The strong move. The review needs three specified passes. More tests in a complete cell cannot address those gaps.
See how a strong candidate says it
“I’d tell the review owner we need one more distinct busy retry and two busy cancellations on this build. I’d keep the recommendation open until those cells are covered.”
Why it works: The review needs three specified passes. More tests in a complete cell cannot address those gaps.
The answer, in short
- Decision
- Keep the trial recommendation open.
- Because
- Only four of six cells have two distinct passes on the candidate build.
- What would change it
- One busy retry and two busy cancellation passes on this build would complete the matrix.
Common traps
- Treating a row count as coverage.
- Using passes from a different build.
Check your answer
- I applied the rule per cell.
- I asked for build and run IDs.
- I counted distinct runs.
- I named the missing behaviors.
Last reviewed Nov 11, 2026