W10 / 2026-11-09 / 40 MIN / ENGLISH

Reproduce a frozen experiment

Classroom formative practice. This does not add a weekly graded assignment. The exercise time is included in class practice, not added after the three-hour meeting.

Open exercise in Colab Download exercise notebook Worked walkthrough All exercises

Before you start

Use a CPU runtime. Save a personal copy or open the downloaded notebook in Jupyter.
Run the included complete walkthrough first. Read its setup, definitions, and checkpoints.
Predict before running each exercise cell; then change only the requested factor.
Keep your actual outputs, configuration, explanation, and limitation. Free tools are sufficient.

TASK 1 / PREDICT - RUN - EXPLAIN

Preserve split membership

Freeze a train/test split of 20 row IDs. Store the IDs and check that neither overlap nor missing rows exist.

Write your prediction in the notebook response field before running.
Run the starter cell below after the worked walkthrough.
Make the requested controlled change and record the configuration.
Explain the mechanism using the actual output, not an invented result.

frozen_order=np.random.default_rng(2026).permutation(20)
frozen_train,frozen_test=frozen_order[:15],frozen_order[15:]
observations['task1']={'train_ids':frozen_train.tolist(),'test_ids':frozen_test.tolist(),'overlap':np.intersect1d(frozen_train,frozen_test).tolist(),'all_rows_accounted_for':len(np.union1d(frozen_train,frozen_test))==20}
print(observations['task1'])

Self-check: The overlap is empty and all 20 IDs are accounted for. Reordering raw rows without preserving IDs changes the experiment.

TASK 2 / PREDICT - RUN - EXPLAIN

Refit an identical deterministic pipeline

Fit the same ridge pipeline twice on the frozen training rows. Compare test predictions using an explicit tolerance.

Write your prediction in the notebook response field before running.
Run the starter cell below after the worked walkthrough.
Make the requested controlled change and record the configuration.
Explain the mechanism using the actual output, not an invented result.

from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import Ridge
ex_rng=np.random.default_rng(7); ex_X=ex_rng.normal(size=(20,3)); ex_y=2*ex_X[:,0]-ex_X[:,1]
replica_1=make_pipeline(StandardScaler(),Ridge(alpha=1)).fit(ex_X[frozen_train],ex_y[frozen_train])
replica_2=make_pipeline(StandardScaler(),Ridge(alpha=1)).fit(ex_X[frozen_train],ex_y[frozen_train])
replica_delta=float(np.max(np.abs(replica_1.predict(ex_X[frozen_test])-replica_2.predict(ex_X[frozen_test]))))
observations['task2']={'max_prediction_difference':replica_delta,'tolerance':1e-10,'within_tolerance':replica_delta<1e-10}
print(observations['task2'])

Self-check: The deterministic rerun should agree within tolerance. This is narrower than independent scientific replication.

TASK 3 / PREDICT - RUN - EXPLAIN

Write a reproducibility contract

List one item missing from a seed-only report. Save versions, split IDs, preprocessing, parameters, and a limitation in your final evidence record.

Write your prediction in the notebook response field before running.
Run the starter cell below after the worked walkthrough.
Make the requested controlled change and record the configuration.
Explain the mechanism using the actual output, not an invented result.

observations['task3']={'numpy':np.__version__,'python':platform.python_version(),'split_ids':observations['task1'],'configuration':{'scaler':'training-only StandardScaler','ridge_alpha':1},'claim_scope':'same toy data and pipeline'}
print(observations['task3'])

Self-check: A seed alone is insufficient. Data identity, ordering, versions, and the actual configuration must also be recorded.

Completion checklist

Three predictions and three explanations.
One controlled change with its actual outcome.
One limitation and a brief AI-use/checking disclosure, or "none".
A downloaded W10-evidence.json, then a two-minute peer explain-back.

The notebook marks incomplete response records as drafts. Completing the fields is not proof that an explanation is correct.

The guided walkthroughs were previously executed. The new exercise extension cells have not yet been execution-tested for this revision. Agent exercises are deterministic simulations; no paid model or consequential external action is used.

Return to this week Exercise collection