Timing: 00:00–02:00 (2 minutes).
Teaching intent: How do models turn ordered observations into useful memory?
Introduce the course and this hour explicitly: Sequential Data and LSTM, Monday 19 October 2026, 14:00–15:00. Explain the cover diagram from observations to a history window, recurrent memory, and a one-step forecast. The trace is illustrative, not experimental evidence. This hour establishes memory; H2 introduces attention and H3 assembles a Transformer.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 02:00–04:00 (2 minutes).
Teaching intent: Define a valid sequence task, then explain and test how LSTM stores information.
Walk through the four timed phases: 00–18 minutes for ordered data, windows and leakage-free evaluation; 18–38 for RNN limitations, LSTM states and gates, and the numerical update; 38–56 for the CPU notebook, baseline comparison, and evidence; 56–60 for limitations and summary. Prerequisites: vectors, elementwise multiplication, and a feed-forward layer. Success means explaining the cell update and interpreting a baseline comparison, not merely running code. Attention and Transformers are the following two hours, not additional topics to compress into this one.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 04:00–06:00 (2 minutes).
Teaching intent: A sequence contains both values and an ordering rule.
Both series contain identical values and have the same mean, but the trend and one-step prediction problem differ. Ask students what a bag-of-values representation loses. Do not imply every prediction task requires order.
Ask: What remains identical after shuffling?
Expected answer: The multiset and mean; adjacency, trend, and time-dependent structure can change.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 06:00–08:00 (2 minutes).
Teaching intent: The prediction must use only information available at its origin.
Use Python slicing explicitly: x[t-L:t] excludes index t. Our origin is just before observing x[t]. Define horizon one; multi-step forecasting needs a separate protocol and cannot reuse unknown future values.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 08:00–10:00 (2 minutes).
Teaching intent: Sequence models receive a batch of histories, with features at every time step.
Connect batch, time, and feature axes to notebook arrays. Batch size is not sequence length. With multiple sensors F increases while chronological order remains on T. Students identify one row and one time step.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 10:00–12:00 (2 minutes).
Teaching intent: Many-to-one predicts one value; many-to-many predicts a value at multiple positions.
The H1 notebook is many-to-one; H3 is causal many-to-many next-token training. Output shape follows the task. A recurrent model can support either arrangement.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 12:00–14:00 (2 minutes).
Teaching intent: Train on earlier targets, select on later validation, then evaluate on test.
Draw boundaries on raw time. Validation selects hyperparameters or checkpoints. Test is reserved for final comparison. Window history may cross a boundary when past observations are genuinely available; target periods remain disjoint.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 14:00–16:00 (2 minutes).
Teaching intent: Global means and variances can reveal future distribution information.
Explain that preprocessing is fitted behavior. Even without labels, a statistic estimated on future data can alter the experiment. The notebook computes scaling parameters from signal[:TRAIN_END].
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 16:00–18:00 (2 minutes).
Teaching intent: Predict the next observation using the last known observation.
For smoothly changing sensors, persistence can be strong. Its final repeated point is a prediction, not an observation. Compare the same target indices and report errors in original units. Introduce ridge regression as a stronger nonrecurrent baseline.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 18:00–20:00 (2 minutes).
Teaching intent: An RNN applies the same state-update parameters at every time step.
Unroll the graph to show state flowing left to right while parameters are reused. h_t is a learned vector summary. More time steps do not create independent new layer weights.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 20:00–22:00 (2 minutes).
Teaching intent: Backpropagation through time multiplies Jacobians along earlier states.
A scalar product illustrates one mechanism, not the complete Jacobian behavior. Repeated small factors can shrink gradients; large factors can amplify them. Explain why a useful early signal may be hard to learn through many recurrent transitions.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 22:00–24:00 (2 minutes).
Teaching intent: The cell state carries memory; the hidden state exposes a filtered view.
Introduce two vector states before all gate equations. The cell path is additive, which can support gradient flow. It does not guarantee perfect retention or remove every optimization difficulty.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 24:00–26:00 (2 minutes).
Teaching intent: Sigmoid outputs values between zero and one for each hidden dimension.
A gate is differentiable, not a discrete if statement. Different dimensions can retain different quantities. In actual models the gate depends on current input and previous hidden state.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 26:00–28:00 (2 minutes).
Teaching intent: Multiply the previous cell state by a learned forget coefficient.
Use the upper cell path and highlight only its forget multiplication. f near one preserves the direct path locally. It still depends on learned weights and does not ensure indefinite task-relevant memory.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 28:00–30:00 (2 minutes).
Teaching intent: A candidate proposes content; the input gate scales how much is written.
Separate signed candidate values from nonnegative write coefficients. Candidates can erase by adding negative content. The input gate and forget gate are separate learned functions.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 30:00–32:00 (2 minutes).
Teaching intent: Retain selected old memory and add selected new content.
The three bars are chosen illustrative numbers: retained old content, new written content, and updated cell. Elementwise multiplication applies independently to every hidden dimension. This is the central LSTM equation.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 32:00–34:00 (2 minutes).
Teaching intent: The hidden state is a bounded, gated view of the updated cell.
Cell values and hidden values are not interchangeable. The output gate can suppress what is exposed while cell content remains. No guarantee exists that one hidden dimension has a human-readable meaning.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 34:00–36:00 (2 minutes).
Teaching intent: Chosen gates make every intermediate step visible.
Open the live gate demo. Have students predict the updated cell with f=.9, i=.2, candidate=-.5, previous cell=.8, o=.7. These values are chosen for instruction, not training outputs.
Ask: What is the hidden output?
Expected answer: h = 0.7 × tanh(0.62) ≈ 0.386.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 36:00–38:00 (2 minutes).
Teaching intent: A high forget coefficient does not prevent new content from changing memory.
Students use the sliders. First set f=1 and vary i: new content still changes c. Then set i=0 and vary f. Finally vary o and observe that c remains fixed while h changes. Ask for one numerical explanation per pair.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 38:00–40:00 (2 minutes).
Teaching intent: Run setup and inspect the synthetic sensor before fitting anything.
Allocate two minutes to launch Colab. CPU only. If dependencies need installing, read the next cell while installation finishes. The notebook is complete and does not depend on H2 or a local data file.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 40:00–42:00 (2 minutes).
Teaching intent: A correct shape is insufficient if the window contains its target.
Run notebook Steps 3 and 4 and inspect first target index. Students verbalize that all input indices are lower than t. The test protocol uses observed past values for rolling one-step prediction.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 42:00–44:00 (2 minutes).
Teaching intent: Ridge uses fixed window positions; LSTM updates a shared recurrent state.
Run the baselines. Neither model is automatically preferred. A simple periodic signal may be well handled by a linear model. Declare equal targets, metrics, and validation policy.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 44:00–46:00 (2 minutes).
Teaching intent: batch_first changes input/output layout, not hidden-state layout.
Read only the model class in Step 7, not a full screen of training code. The prediction head takes states[:,-1,:]. Hidden-state dimensions do not become batch-first.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 46:00–48:00 (2 minutes).
Teaching intent: Gradient updates use training; checkpoint selection uses validation.
Run Step 8. Gradient clipping limits gradient magnitude and does not solve vanishing gradients by itself. Explain model.train versus eval and no_grad. No test metric is consulted during selection.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 48:00–50:00 (2 minutes).
Teaching intent: Inspect actual training and validation trajectories in your notebook.
The slide gives an evidence workflow rather than a fabricated loss curve. Ask students to point to their actual minimum validation epoch. A decreasing training curve alone does not demonstrate generalization. The displayed plot is the recorded CPU notebook run on synthetic data, not an illustrative trajectory. Compare its minimum validation epoch with your own run.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Measured figure: h1-training-loss.png; recorded CPU notebook execution, 19 October 2026. Source arrays and checks: validation/result-figure-provenance.json.
Timing: 50:00–52:00 (2 minutes).
Teaching intent: MAE averages error magnitude; RMSE emphasizes large residuals.
The displayed residuals are illustrative. Students run Step 9 to obtain actual model residuals and a baseline table. Ask whether errors cluster at peaks and whether the LSTM truly improved on ridge. The displayed forecast and residual figure comes from the recorded CPU run on synthetic data; the chosen residual list in the original construction is not used in the displayed figure.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Measured figure: h1-forecast-residuals.png; recorded CPU notebook execution, 19 October 2026. Source arrays and checks: validation/result-figure-provenance.json.
Timing: 52:00–54:00 (2 minutes).
Teaching intent: PyTorch concatenates its learned gate parameters in i, f, g, o order.
Run Step 10 and compare the manually reconstructed cell against nn.LSTM with allclose. The heatmap in the notebook comes from learned weights. This is implementation parity for one cell, not proof of scientific validity.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Measured figure: h1-learned-gates.png; recorded CPU notebook execution, 19 October 2026. Source arrays and checks: validation/result-figure-provenance.json.
Timing: 54:00–56:00 (2 minutes).
Teaching intent: Change window length while preserving the forecast question and split boundaries.
Have pairs predict before changing the parameter and rerun from data construction. Target counts near the start change with window length; test targets remain the same because all chosen lengths are shorter than TRAIN_END. Keep separate evidence records.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 56:00–58:00 (2 minutes).
Teaching intent: Retention depends on learned gates, sequence structure, and optimization.
Use the decay curves to distinguish 0.9 from 0.99 across many steps. The plotted direct path omits other dependencies of gates on state. Do not claim LSTM eliminates vanishing gradients in every setting.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]
Timing: 58:00–60:00 (2 minutes).
Teaching intent: Order defines the task. LSTM gates control what memory is kept, written, and exposed.
Close with two core ideas: the available ordered history defines the task, and learned gates control the additive cell-memory update and exposed hidden state. Ask a student to identify the retained and written terms in the displayed equation. Briefly point to chronological splits, training-only scaling, baseline errors, and saved configuration as the evidence needed to trust the experiment. LSTM helps gradient flow but does not guarantee indefinite memory. Bridge to H2: a final-state encoder compresses history; attention can retrieve relevant earlier representations instead. Do not introduce new theory on the summary slide.
Visual: Original editable SVG or live calculation, designed for this slide. All non-run numeric diagrams are illustrative teaching calculations.
[Sources]