Replay in humans
… and how to find it.
Introduction
How does the brain use past experience to guide future actions? A particularly interesting phenomenon in this context is replay, typically described as the reactivation of activity patterns in the brain that reflect previous experiences.
Most insights about replay have come from electrophysiological studies of rodent spatial navigation. In these studies, replay is often investigated as follows: A rodent navigates a spatial environment, such as a simple linear track. In the rodent’s brain, cells in the hippocampus, known as place cells, are activated when the animal occupies a specific location in that environment. As the rodent moves along the track, these place cells are activated in a specific sequential order that reflects the animal’s trajectory (see Figure 1).
The fascinating part happens when the rodent is asleep or stationary: the same cells that were activated during navigation are reactivated in the same sequential order, but this time on a much faster timescale—within a few hundred milliseconds (see Figure 2). This phenomenon is called replay.
Replay, however, does not only occur during sleep; it also happens in the awake state. For example, when the rodent is at the beginning of the track, forward replay can depict the upcoming trajectory the animal is about to follow (see Figure 3).
Moreover, replay can occur not only in the same order as experience but also in reverse. Backward replay is typically observed when the animal reaches a rewarding location at the end of the track (see Figure 4).
While most replay research focuses on the hippocampus, many studies now show that replay-like sequences are distributed throughout the brain, including areas like the prefrontal cortex and visual cortex.
Replay in humans
When studying replay in humans, we must consider three key properties: replay is (1) sequential, (2) fast, and (3) anatomically distributed. This poses a challenge for non-invasive neuroimaging, as we need methods with both high temporal and spatial resolution.
Methods
fMRI background knowledge
- TR = Repetition Time, usually expressed in seconds or milliseconds. The TR indicates how frequently an entire volume of fMRI data (i.e., the entire brain is recorded)
- HRF = Hemodynamic Response Function, models the response of the blood supply in relation to a presumed neural response.
Replay in fMRI
Short answer: No, not necessarily, because the underlying hemodynamics (slow rise and fall of the HRF) stay the same, independent of the speed of the sampling rate. For a similar line of arguments, see Wittkuhn & Schuck (2021).
Why to we even think that we can measure a fast sequential neural event like replay with fMRI? fMRI is known to be a slow method, because it measures neural activity indirectly through slow sampling of a delayed blood-oxygen level dependent response function. Let’s imagine a prototypical replay sequence as it might occur in the rodent brain (Figure 5 a). This replay sequence consists of five place cells that are activated in sequence and the individual events are typically separated by 20 to 80 ms. What happens if the same replay sequence occurs in the human brain and we measure it with fMRI? Let’s imagine for simplicity that each of these events triggers its own independent HRF. This results in a set of largely overlapping HRFs but the critical idea is that these HRFs will be slightly delayed by the temporal offset between replay events in the sequence (Figure 5 b). This means that the sequentiality of the replay sequence will translate into ordered fMRI activation that we can asses within a single measurement (TR). During earlier time points, earlier events will have a higher activation than later events (Figure 5 b). This pattern reverses during the falling slopes of the HRFs (Figure 5 c). In Wittkuhn & Schuck (2021) we tested this idea experimentally using multivariate fMRI methods.
Between-TR decoding
- Used in Schuck & Niv (2019) and Wittkuhn & Schuck (2021) (Figure 6)
- Basic idea: Concurrently activated items (as during a replay event) will lead to clustered / overlapping activation of items that are decoded in successive TRs
- Procedure:
- Apply trained classifier to every TR of the time window of interest (e.g., resting-state period)
- Get the label of the decoded class (i.e., category with the highest classifier probability)
- Compare the frequency of label transitions in the decoded data to transitions between states in the task
Within-TR decoding
The slope of a linear regression relating classifier probabilities within each TR to their serial position can indicate the direction of the sequence (Figure 8). A forward sequence occurs when earlier events have higher probabilities and later events lower, resulting in a positive slope (forward sequentiality). Conversely, a backward sequence shows earlier events with lower probabilities and later events higher, producing a negative slope (backward sequentiality). This also emphasizes the need for probabilistic classifiers that represent the strength of an activated representation.
The main sequentiality metric used in Wittkuhn & Schuck (2021) is the slope of a linear regression relating classifier probabilities to serial positions at every TR.
Classifier training
In Wittkuhn & Schuck (2021)
In Wittkuhn & Schuck (2021), five logistic regression classifiers were trained, one for each of the five images used. Classifiers were trained on a separate task condition, where images were presented individually with longer delays (Figure 9 a). Decoding visual images from visual cortex worked well and reached about 70% decoding accuracy (Figure 9 b left). Classifiers were probabilistic. That means, instead of predicting a single category, they provide a probability for each of the five classes. When trained classifiers were applied to several TRs following onsets of single images, classifier probabilities increase, peak and decrease again (Figure 9 b middle). This pattern indicates that classifier probabilities trace the underlying hemodynamic response on single trials.
