W07-H2 / 2026-10-19 / 20 MIN / ENGLISH

Attention arithmetic, masks, and interpretation

Classroom formative practice. This does not add a weekly graded assignment. The exercise time is included in class practice, not added after the three-hour meeting.

Open exercise in Colab Download exercise notebook Worked walkthrough All exercises

Before you start

Use a CPU runtime. Save a personal copy or open the downloaded notebook in Jupyter.
Run the included complete walkthrough first. Read its setup, definitions, and checkpoints.
Predict before running each exercise cell; then change only the requested factor.
Keep your actual outputs, configuration, explanation, and limitation. Free tools are sufficient.

TASK 1 / PREDICT - RUN - EXPLAIN

Calculate retrieval coordinate by coordinate

Predict the three weights and the two output coordinates. Use the contributions table to explain which value supplies each coordinate.

Write your prediction in the notebook response field before running.
Run the starter cell below after the worked walkthrough.
Make the requested controlled change and record the configuration.
Explain the mechanism using the actual output, not an invented result.

q_ex=np.array([[1.,0.]])
k_ex=np.array([[1.,0.],[0.,1.],[1.,1.]])
v_ex=np.array([[10.,0.],[0.,10.],[10.,10.]])
s_ex=q_ex@k_ex.T/np.sqrt(2)
w_ex=np.exp(s_ex-s_ex.max()); w_ex/=w_ex.sum(axis=-1,keepdims=True)
contributions=w_ex[0,:,None]*v_ex
observations['task1']={'weights':w_ex.tolist(),'contributions':contributions.tolist(),'context':(w_ex@v_ex).tolist()}
print(observations['task1'])

Self-check: Weights are about [0.4011,0.1978,0.4011]; the context is about [8.0222,5.9889]. The sum must be performed separately for each coordinate.

TASK 2 / PREDICT - RUN - EXPLAIN

Mask before normalizing

Combine a causal mask with a padded last key. Check permitted-row sums and blocked weights. Do not mask a previously normalized matrix without renormalizing.

Write your prediction in the notebook response field before running.
Run the starter cell below after the worked walkthrough.
Make the requested controlled change and record the configuration.
Explain the mechanism using the actual output, not an invented result.

raw_scores=np.array([[1.,0.,1.],[0.,1.,1.],[1.,1.,2.]])/np.sqrt(2)
allowed_ex=np.tril(np.ones((3,3),dtype=bool)) & np.array([True,True,False])[None,:]
masked_ex=np.where(allowed_ex,raw_scores,-np.inf)
a_ex=np.exp(masked_ex-masked_ex.max(axis=-1,keepdims=True)); a_ex/=a_ex.sum(axis=-1,keepdims=True)
observations['task2']={'weights':a_ex.tolist(),'row_sums':a_ex.sum(axis=-1).tolist(),'blocked_weights_zero':bool(np.all(a_ex[~allowed_ex]==0))}
print(observations['task2'])

Self-check: Every row sums to one and the padded final key has zero weight. Each row must retain at least one permitted key.

TASK 3 / PREDICT - RUN - EXPLAIN

Separate routing from causal explanation

Change only the second value vector. Predict whether weights change and whether the retrieved context changes. Explain the scope of this intervention.

Write your prediction in the notebook response field before running.
Run the starter cell below after the worked walkthrough.
Make the requested controlled change and record the configuration.
Explain the mechanism using the actual output, not an invented result.

changed_v=v_ex.copy(); changed_v[1]=[100.,100.]
observations['task3']={'original_context':(w_ex@v_ex).tolist(),'changed_context':(w_ex@changed_v).tolist(),'same_weights':True}
print(observations['task3'])

Self-check: Q and K are fixed, so these weights stay fixed while context changes. This tests a computation; it does not establish causation in a physical system.

Completion checklist

Three predictions and three explanations.
One controlled change with its actual outcome.
One limitation and a brief AI-use/checking disclosure, or "none".
A downloaded W07-H2-evidence.json, then a two-minute peer explain-back.

The notebook marks incomplete response records as drafts. Completing the fields is not proof that an explanation is correct.

The guided walkthroughs were previously executed. The new exercise extension cells have not yet been execution-tested for this revision. Agent exercises are deterministic simulations; no paid model or consequential external action is used.

Return to this week Exercise collection