National Cheng Kung UniversityLAiMM Laboratory

W16 / 2026-12-21 / 35 MIN / ENGLISH

Build an evaluation set that exposes a bug

Classroom formative practice. This does not add a weekly graded assignment. The exercise time is included in class practice, not added after the three-hour meeting.

Open exercise in ColabDownload exercise notebookWorked walkthroughAll exercises

Before you start

  1. Use a CPU runtime. Save a personal copy or open the downloaded notebook in Jupyter.
  2. Run the included complete walkthrough first. Read its setup, definitions, and checkpoints.
  3. Predict before running each exercise cell; then change only the requested factor.
  4. Keep your actual outputs, configuration, explanation, and limitation. Free tools are sufficient.

TASK 1 / PREDICT - RUN - EXPLAIN

Calculate before comparing candidates

Compute the mean of [1,2,3] by hand. Compare the reference and intentionally buggy candidate.

  1. Write your prediction in the notebook response field before running.
  2. Run the starter cell below after the worked walkthrough.
  3. Make the requested controlled change and record the configuration.
  4. Explain the mechanism using the actual output, not an invented result.
case_values=[1,2,3]
observations['task1']={'expected':2.,'baseline':float(baseline(case_values)),'candidate':float(buggy_candidate(case_values))}
print(observations['task1'])

Self-check: The expected mean is 2. The candidate must be judged against the reference, not against its own output.

TASK 2 / PREDICT - RUN - EXPLAIN

Expose a misleading passing case

Compare zero-sum and negative-mean cases. Explain why passing one symmetric case can hide a denominator bug.

  1. Write your prediction in the notebook response field before running.
  2. Run the starter cell below after the worked walkthrough.
  3. Make the requested controlled change and record the configuration.
  4. Explain the mechanism using the actual output, not an invented result.
observations['task2']=[]
for values_ex in [[-2,2],[-2,-4],[5]]:
    observations['task2'].append({'values':values_ex,'expected':float(np.mean(values_ex)),'candidate':float(buggy_candidate(values_ex))})
print(observations['task2'])

Self-check: The zero-sum case can pass despite the bug. At least one nonzero-mean case is needed to expose the failure.

TASK 3 / PREDICT - RUN - EXPLAIN

Declare a numerical tolerance

Compare two close values under two tolerances, and explain the difference between absolute and relative error.

  1. Write your prediction in the notebook response field before running.
  2. Run the starter cell below after the worked walkthrough.
  3. Make the requested controlled change and record the configuration.
  4. Explain the mechanism using the actual output, not an invented result.
expected_ex=1.; observed_ex=1.+1e-7
observations['task3']={str(tol):bool(np.isclose(observed_ex,expected_ex,atol=tol,rtol=0)) for tol in [1e-6,1e-9]}
print(observations['task3'])

Self-check: The comparison passes at absolute tolerance 1e-6 but fails at 1e-9. A tolerance must be justified before interpreting pass rates.

Completion checklist

The notebook marks incomplete response records as drafts. Completing the fields is not proof that an explanation is correct.

The guided walkthroughs were previously executed. The new exercise extension cells have not yet been execution-tested for this revision. Agent exercises are deterministic simulations; no paid model or consequential external action is used.

Return to this weekExercise collection