Conferences
You can see the Awh/Vogel lab’s conference presentations here:
Conference:
OPAM/Psychonomics 2024
VSS 2024
OPAM/Psychonomics 2024
OPAM - Talks
Reconciling Meaningful Object Benefit in Visual Working Memory: The Role of Proactive Interference
Hyung-Bum Park & Edward Awh
Thursday, November 21, 3:40 - 4:40pm (Talk Session 4)
Visual working memory (VWM) capacity is often reported higher for meaningful objects due to their rich representations. However, previous work has confounded meaningfulness with the strength of proactive interference (PI) between stimulus types. By manipulating stimulus type and repetition in a recognition task, we found the VWM benefit for meaningful objects appeared only in the trial-unique condition due to lower false-alarm rates. Mouse trajectory analysis highlighted robust PI effects during initial movements in trial-repeated conditions. Dual-process modeling indicated the benefit was driven by global familiarity, not recollection. These results highlight the role of long-term memory, through PI, in shaping VWM capacity estimates.
Keynote Address: Working memory storage depends on content-independent pointers
Edward Awh
Thursday, November 21, 4:50-5:45pm
Working memory theorists have often highlighted the links between voluntary attentional control and the intentional storage of information in working memory (WM). For example, my early work provided initial evidence that covert spatial attention is oriented towards positions held in spatial WM, yielding faster behavioral responses and enhanced visually-evoked neural activity at memorized locations. Twenty-five years later, temporally resolved EEG measures of covert attention provide even clearer evidence for the parallel deployment of covert spatial attention during visual working memory tasks, even when location is irrelevant to the task. Nevertheless, here I will show that while spatial attention is a persistent partner of visual working memory, the deployment of spatial attention can be functionally dissociated from encoding into visual working memory. Specifically, WM storage recruits a content-independent indexing operation that tracks the number of items currently stored in WM, independent of the features contained in those items. Our hypothesis is that this content-independent load signal reflects the deployment of “pointers” that enables the binding of items to the surrounding context. Critically, I’ll present direct evidence that observers can spatially attend and categorize items without encoding those items into working memory. Thus, spatial attention and pointer deployment reflect dissociable aspects of voluntary attentional control, a distinction that may enrich our understanding of a range of major attentional phenomena.
OPAM - Posters
The brain knows more is stored in visual long-term memory than we can report
Chong Zhao, Keisuke Fukuda & Geoffrey F. Woodman
Thursday November 21, 11:30am-12:30pm (Poster Session 1)
When we attempt to recall a representation from visual long-term memory, there is a possibility that we might not succeed in retrieving it even though it exists in our brain. This study reveals that while explicit memory retrieval mechanisms sometimes fail to access these stored memories, executive control mechanisms can swiftly query the memory to confirm the presence of a representation. Our findings suggest that much of the memory typically accessed implicitly is, in fact, directly accessible by the brain’s higher-level control processes.
Psychonomics - Talks
Neural evidence for modality-general indexing in working memory
Henry Jones, Darius Suplica, William Thyer, Edward Awh
Friday, November 22, 10:40-11:00am (Visual Working Memory II)
Neural studies of working memory (WM) have strongly focused on the stimulus-specific neural patterns that track the stored content. By contrast, we highlight a distinct class of neural signals that appears to track the number of individuated items stored in a content-independent fashion. We propose that these load signals track the deployment of spatiotemporal pointers that bind stored items to the surrounding context. We provide support for this theory in 2 EEG experiments. First, we show that the multivariate signature of these pointers generalizes across color and motion stimuli, despite the cortically disparate representations of these features (Experiment 1). Moreover, representational similarity analysis (RSA) reveals a pointer signal that generalizes across visual and auditory sensory modalities, while also providing evidence for distinct signals reflecting spatial attention and feature load (Experiment 2). These results support the abstract indexing of individuated memory representations as a core component of WM storage and may provide ins
Recognition-memory asymmetries predicted from item memorability
Igor Utochkin, Nicholas Chiang, & Wilma Bainbridge
Friday, November 22, 4:50-5:10pm (Recognition I)
In two-alternative forced-choice (2-AFC) memory tests, target-foil similarity is a strong determinant of confusions. We hypothesize that confusions can be driven by intrinsic memorability properties of test items, the likelihood for each item to be correctly or falsely recognized individually. We tested old-new recognition for 240 images in two groups of 100 observers swapping target-foil roles between the groups, which yielded hit (H) and false alarm (FA) rates for each image. We z-transformed H and FA to locate both roles of all items in a signal-detection “memorability space” and predicted 2-AFC discriminability (d’) between arbitrary targets and foils. Critically, for some 2-AFC pairs we predicted strong d’ asymmetries when items swap their target-foil roles. These predictions were confirmed in a 2-AFC experiment with the same set of images and two new groups of observers studying either one or another set of targets but tested on the same target-foil pairs. The d’ asymmetries between identical pairs with swapped roles highly correlated with the asymmetries predicted by our model (rho=.73). These asymmetries cannot be explained by simple inter-item similarity and require incorporating memorability to the existing models.
Psychonomics - Posters
Characterizing individual differences in sustained attention, attentional control, and long-term memory
Chong Zhao, Anna Corriveau, Jin Ke, Edward K. Vogel, Monica Rosenberg
Friday, November 22, 7:45-9:15pm (Poster Session IV)
Individuals differ in their ability to sustain attention. However, whether differences in sustained attention reflect differences in attentional control (AC) and long-term memory (LTM) or whether these abilities comprise largely separable processes remains unknown. We conducted an online study measuring participants’ sustained attention, AC, and LTM. We measured sustained attention with an audio-visual continuous performance task (avCPT) in which participants responded to visual or auditory stimuli while inhibiting responses to infrequent targets, AC with Square tasks, and LTM with recognition and source memory tests. An exploratory factor analysis revealed that sustained attention formed a distinct factor (eigenvalue = 4.71) that loaded positively onto LTM, and, to a lesser extent, AC factors. To test how neural signatures of sustained attention related to AC and LTM, we analyzed fMRI functional connectivity patterns collected as 20 participants performed the avCPT. A pre-trained connectome-based model of sustained attention predicted participants’ performance on out-of-scanner LTM, but not AC, tasks. Together this suggests that differences in sustained attention, although correlated with AC, are more closely related to LTM.
Improving sustained attention with real-time monitoring of attentional lapses
Matthieu Chidharom, Monica D. Rosenberg, Edward K. Vogel
Saturday, November 23, 7:45-9:15pm (Poster Session VII)
Lapses in attention are common in everyday life, increasing the risk of errors such as missing a highway exit. In this study, we examined participants’ ability to externally monitor their attentional lapses in real-time to prevent behavioral errors before they occur. Thirty participants performed a go/no-go task in which they were instructed to respond to frequent-category images (e.g., indoor scenes) and withhold response to infrequent-category images (e.g., outdoor scenes). Fast reaction times, previously identified as strong predictors of no-go trial errors, were measured during task performance and used to track periods of inattention. During the intertrial-interval, a white fixation cross turned red whenever an error-prone period of fast responding was detected. A generalized mixed model showed that participants could reduce errors by monitoring their lapse states, compared to blocks where the red-cross appeared randomly. Notably, individuals with better performance overall were more likely to benefit from external monitoring.
VSS
Talks
Storage in working memory recruits a modality-independent pointer system
Henry Jones (henryjones@uchicago.edu), Darius Suplica, William Thyer, Edward Awh
Sunday, May 19, 11:30am (Talk Room 1)
Prominent theories of working memory (WM) have proposed that distinct working memory systems may support the storage of different types of information. For example, distinct dorsal and ventral stream brain regions are activated during the storage of spatial and object information in visual WM. Although feature-specific activity is likely critical to WM storage, we hypothesize that a content-independent indexing process may also play a role. Specifically, spatiotemporal pointers may be required for the sustained indexing and tracking of items in space and time, even while features change, within an unfolding event. Past evidence for such a content-independent pointer operation includes the finding that signals tracking the number of individuated representations in WM (load) generalize across colors, orientations and conjunctions of those features. However, overlapping orientation and color codes in early visual cortices may mimic a generalizable signal. Here, we provide a stronger demonstration of content-independence by using pairs of features that are as cortically disparate as possible. Study 1 (n=16) used color and motion coherence stimuli, and showed that load decoding models generalized across these disparate features. In addition, we used representational similarity analysis (RSA) to document “pure” load signals that tracked the number of items stored regardless of attended feature, while simultaneously documenting and controlling for feature-specific neural activity. Extending these observations, in Study 2 (n=24; n=16) we applied similar analytic approaches to demonstrate a common load signature between auditory and visual sensory modalities, while controlling for modality-specific neural activity and the spatial extent of covert attention. Our findings suggest that content-independent pointers may play a fundamental role in the storage of information in working memory, and may contribute to its overall limited capacity.
Piotr Styrkowiec, William Ngiam, Will Epstein, Ron Gneezy, Edward Awh, Edward Vogel
Sunday, May 19, 2:30pm (Talk Room 1)
Recent work has suggested that storage in visual working memory (VWM) occurs through the assignment of spatiotemporal pointers to the to-be-remembered items (Thyer et al., 2022). Thus, VWM capacity limits may not be set by the stimulus content exactly, but rather by attentional processes that define the spatiotemporal pointers for item-based storage. We examined whether this is the case in the contralateral delay activity (CDA), an event-related potential long known to track VWM load. The CDA has been shown to track the number of targets in multiple-object tracking (Drew and Vogel, 2008), but also the number of to-be-remembered colors (Vogel and Machizawa, 2004). To directly contrast the effects of attentional tracking load and stimulus content load on working memory, we developed a novel dual-task paradigm. Participants track either one or two moving discs (attentional tracking load), with either two or four colors displayed across each of the discs (working memory load). Participants completed a ‘tracking only’ condition, where they would need to monitor the moving target discs like in a multiple-object tracking task, and a ‘tracking plus memory’ condition, where they would track the discs and remember all displayed colors like in a multiple-identity tracking task. The key question was whether or not CDA amplitude would be determined by the number of individuated items tracked, or by the number of distinct colors associated with the currently tracked items. Strikingly, CDA amplitude was determined almost entirely by the number of items tracked, with no reliable effect of variations in the number of colors per tracked item. These findings suggest the CDA largely reflects the maintenance of spatiotemporal pointers for moving objects, not the number of feature values associated with those objects.
Associative learning changes multivariate neural signatures of visual working memory
William Ngiam, William Thyer, Henry Jones, Darius Suplica, Will Epstein, Edward Awh
Tuesday, May 21, 3:30pm (Talk Room 1)
A hallmark of visual working memory is its sharp capacity limit, though this limit can be circumvented using learned knowledge. For example, when arrays of to-be-remembered items contain statistical regularities, people can learn the associations between items and recall more information overall (Brady et al., 2009; Ngiam et al., 2019). One proposed mechanism for how this recall benefit is achieved is through ‘memory compression’ – redundancies introduce a reduction of information per item, enabling more items to be stored online. Another proposed mechanism is that pointers are efficiently allocated to each ‘chunk’ with the benefit coming from long-term memory retrieval rather than changes to working memory itself. In an attempt to distinguish between these possibilities, we turned to an EEG measure that tracks the number of individuated items stored in working memory (mvLoad; Thyer et al., 2022). The memory compression account predicts an overall increase in the number of items stored online, whereas the long-term memory retrieval account predicts a reduction in working memory load. Subjects completed a training session where they learned specific color-color pairs. In a subsequent EEG session, subjects completed a recall task with 2 random colors, 4 random colors, or 2 learned color pairs. mvLoad analysis showed a reduction in working memory load for the 2 learned pairs condition (from 4 towards 2), consistent with the notion that an item-based pointer is assigned to each chunk. Moreover, multidimensional scaling shows an additional independent signal that distinguishes the 2 learned pairs condition from the other conditions. We propose that this additional signal reflects the involvement of long-term memory, consistent with the notion that the learned association is being relied upon to maintain the information.
The effects of visual encoding speed on ERP markers of subsequent retrieval
Igor Utochkin, Chong Zhao, Edward Vogel
Wednesday, May 22, 8:15am (Talk Room 2)
Our memory for meaningful visual stimuli is remarkable: Even when we see thousands of images, each presented for a few seconds, we can later recognize them among new images with high accuracy and in detail (Standing et al., 1973; Brady et al., 2008). However, recognition suffers if the images are encoded at a speed of 2 images per second or faster (Intraub, 1980; Potter, 1976; Potter et al., 2002). Presumably, this happens because the encoding of each new rapidly presented image disrupts relatively slow short-term memory consolidation which is essential for the instantiation of subsequent long-lasting episodic memory. Here, we studied how encoding speed impacts EEG markers of subsequent recognition, namely, ERP Old/New effects, that is, differences between ERP responses to earlier presented (old) and never presented (new) stimuli. In each block, participants memorized sequences of 20 real-world object images at a slow or fast rate (one image each 1,750 ms or 250 ms, respectively). Their memory was then tested with an “old/new” recognition task combined with EEG recording. Our analysis focused on two ERP Old/New components typically distinguished in the literature (Curran, 2000; Paller et al., 2007; Rugg & Curran, 2007): earlier frontal, FN400, and later parietal, LPC. Although observers showed significantly worse recognition at the fast compared to the slow encoding condition, we found almost equally pronounced FN400 in both conditions. In contrast, the LPC was much larger in amplitude in the slow than in the fast encoding blocks. One interpretation of this dissociation can be that fast encoding speed selectively impairs recollection-based memory (which reflects in reduced LPE) but not familiarity-based memory (little effect on FN400). However, other interpretations (e.g., that slower encoding speed causes a stronger confidence signal reflected in LPE) can also be discussed.
Posters
Reaction Time Variability and Mind-Wandering Provide Complementary Insights
Matthieu Chidharom, Edward Vogel, Monica Rosenberg
Saturday, May 18m 8:30am (Pavilion)
Sustained attention refers to the ability to maintain focus on a task over an extended period of time. However, sustaining attention is challenging, as there are inherent fluctuations between periods of good attention (stable and less error-prone) and poor attention (unstable and error-prone). Two main methodologies have been used to isolate these attentional fluctuations in visual attention tasks: objective and subjective approaches. The objective methodology analyzes intraindividual reaction time (RT) variability. For example, Esterman et al. (2013) revealed higher errors during periods of high RT variability (out-of-the-zone state) compared to periods of low variability (in-the-zone state). The subjective methodology uses thought probes to identify mind-wandering episodes (task-unrelated thoughts), that can be either intentional (i.e., deliberate) or unintentional (Seli, 2016). Although previous studies revealed higher error rates during out-of-the-zone and mind-wandering states, it is unclear whether those two methodologies isolate the same or different type of attentional fluctuations. This study compared the two approaches in a single Go/NoGo sustained attention task (N=38), objectively measuring attention via RT variability, and subjectively via intermittent thought probes administered every 30 trials. The entire task lasted approximately 40 minutes. If both methods isolate similar fluctuations, we hypothesize that the time spent out-of-the-zone should be higher during mind-wandering than during on-task periods. Our results revealed significantly higher time out-of-the-zone during mind-wandering (52.1%) compared to on-task periods (47.8%), suggesting the methods isolate, to some extent, similar aspects of fluctuations. However, time out-of-the-zone was significantly higher during intentional (57.7%) versus unintentional mind-wandering (49.7%), indicating higher overlap between the objective and subjective methods for deliberate mind-wandering rather that spontaneous. These findings suggest the two methods capture complementary information about attentional fluctuations and highlights the utility of combining objective and subjective methods to gain a comprehensive understanding of sustained attention and its lapses.
A Change Localization Benefit for Mixed Arrays Over Uniform Arrays
Temilade Adekoya, Chong Zhao, Edward Vogel, Edward Awh
Sunday, May 19, 2:45pm (Pavilion)
Given relatively sharp capacity limits in visual working memory (WM), there has been sustained interest in whether these limits are based on the number of individuated items stored, or based on interference that varies as a function of inter-item similarity. Object-based models predict that the costs of concurrent storage will be determined only by the total number of items stored, while feature-based models of capacity often argue that competition for feature-specific resources is the key limiting factor. To examine this question we manipulated both the number of stimuli within each memory array, as well as the similarity between those items. Thus, we measured visual WM performance with uniform arrays with only one type of feature (e.g. color), or mixed arrays with two feature-types (e.g. color and orientation). Consistent with object-based models, we observed a significant cost of increasing load in both the mixed and uniform conditions. However, we also saw evidence for a modest advantage in the mixed array conditions, a difference that is not predicted by pure object-based models. Our follow-up work examined whether the advantage in the mixed condition reflects differences in whether items are stored, or differences in memory fidelity. We found that this advantage indeed reflected a change in memory fidelity, specifically a better precision in memory of items in the mixed arrays than in uniform arrays. Additionally, there was no evidence that participants were storing a larger number of items in the mixed condition. These results suggest that working memory performance is subject to item-based limits independent of item similarity, but the fidelity of those memories can be shaped by inter-item similarity.
Visual working memory load persists during the comparison phase
Chong Zhao, Temilade Adekoya, Sintra Horwitz, Edward Awh, Edward Vogel
Sunday, May 19, 2:45pm (Pavilion)
Working memory is often measured by presenting arrays of visual items to be remembered over a short delay that must be compared with a test display. Extensive work has elucidated the neural mechanisms that support the encoding and maintenance periods of these tasks. Though, little is still understood about the comparison process itself. For example, the contralateral delay activity (CDA) is a sustained EEG component that provides a sensitive measure of the current working memory load during the retention period. However, it is not clear whether this activity continues to track the full working memory load during the comparison phase of the task or if it is reduced to just the one item from the array that is being tested. In Experiment 1, we used a change localization task with 2-item and 4-item arrays of colors. At test, subjects had to report which item changed. We observed that the response-phase CDA for set size 4 was significantly larger than the CDA for set size 2. In Experiment 2 we used a single-probe change detection design, in which only a single item from the original array was shown at test. Despite only a single item shown on the screen, we again observed that the response-phase CDA for set size 4 was still significantly larger than the CDA for set size 2. Our results suggest that the working memory load during test reflects the load from all of the items from the array that were stored.
Crossing category boundaries: Perceptual hysteresis for scenes even with endpoint preview
Huiqin Chen, Mei Yang, Gaeun Son, Dirk Bernhardt-Walther
Monday, May 20, 8:30am (Pavilion)
n dynamically changing environments, how does our visual system make quick perceptual decisions based on information that is actively changing? The hysteresis effect suggests that when resolving perceptual ambiguity, people tend to stick with their current interpretation of sensory information and find it difficult to change their perception until there is a noticeable and significant change in input. We here demonstrate this effect for dynamically changing scenes. We further explored how prior exposure to target scene would affect the hysteresis effect. We used indoor scene images generated with a Generative Adversarial Network to create smooth yet realistic transitions between scene categories. Participants were asked to report when they perceived a shift in category during these transitions. Before each trial, participants were provided with information about the scene category at the end of the transition, either in the form of words, images, or both. Each transition was repeated in both directions (A to B and B to A), and the differences in responses between the two opposite directions were analyzed. Our findings indicated that exposure to words or images had no effect on perceptual hysteresis. Even when participants had knowledge of the target category, their perception of the image categories was still biased towards the initial category. This result suggests that neither semantic knowledge nor visual representation of the future can influence the hysteresis effect. The top-down knowledge of the future direction does not impact or eliminate the conservatism of the visual system. This has important implications, indicating that hysteresis is an innate characteristic of the visual system and is not easily influenced by higher-level control. Thus, perceptual conservatism is likely instrumental for the apparent stability of visual perception in most real-world settings.
Unfolding Serial Dependence Across Perception, Working Memory Consolidation and Retrieval
Hyung-Bum Park (hbpark@uchicago.edu), Edward Awh
Tuesday, May 21, 8:30am (Banyan Breezeway)
Abstract
The debate on serial dependence questions its origin in perceptual versus post-perceptual working memory (WM) components, with mixed findings on whether perceptual decisions exhibit attractive serial dependence toward previous target or repulsive bias linked to sensory adaptation. The present study aims to examine the dynamics of serial biases across perception and WM processes at consolidation and retrieval. Participant performed a two-part experiment, with the first part involved immediate continuous estimations of color on color-wheel within view (perceptual report). The second part involved WM recall with dual-response. A memory color followed by masks was reported after stimulus offset (consolidation report), and the same item was reported again after delay on another color-wheel (retrieval report). Mouse trajectories for all responses were recorded to index direction and magnitude of serial bias. Behavioral reports showed a gradual evolution of serial bias across opposite directions: repulsion in perceptual reports, moderate attraction in consolidation, and stronger attraction in retrieval. Critically, we found a unique mouse trajectory pattern in consolidation reports, with a repulsive curvature from the previous target appearing on the color-wheel, while the overall trajectory exhibited attraction bias. Further examination of moment-by-moment trajectories, coupled with a median-split of movement onset latency, captured a ‘repulsion-to-attraction’ transition in consolidation reports, with early repulsion evolving into attraction as movements progressed. Moreover, the median-split analysis revealed a magnification of bias from early-onset to late-onset trials within their directions (stronger repulsion vs. attraction in perceptual and WM reports, respectively). These suggest that changes in serial biases across processes are not solely time-dependent, but originate from distinct perceptual and WM mechanisms. The repulsion-to-attraction transition during the WM consolidation phase offers an intermediate window into the interplay between sensory adaptation and serial dependence, jointly shaping serial bias in behavioral reports. Together, our study provides evidence supporting the mnemonic origin of serial dependence.
Investigating Meaningfulness in Visual Working Memory
Leo Chang (woohyeukchang@uchicago.edu), William Ngiam, Edward Awh
Tuesday, May 21, 2:45pm (Banyan Breezeway) (Pre-data Poster!)
There has been a growing body of work in Visual Working Memory (VWM), where researchers are utilizing the real-world objects as test stimuli, instead of simple colored shapes and orientations, that are more meaningful. In addition, there are findings that show advantage in VWM capacity for real-world objects, suggesting that the meaningfulness is contributing to our visual cognition, possibly through the involvement of our episodic Long Term Memory (LTM) system. However, the concept of meaningfulness has not been defined nor tested thoroughly. To understand how such concept affects our VWM, we propose the usage of words that clearly have associated meanings and are prevalent in our daily lives, just like real-world objects. Before investigating the effect of meaningfulness in words on VWM, we will run a scalp EEG replication study with a slight modification, where we present words in CDA design with place holders. This would allow us to perform multivariate load analysis and further confirm that words do undergo similar VWM processing as simple colored stimuli. Next, we will run a more canonical working memory paradigms using words with a manipulation of meaningfulness (e.g., word frequency, emotion, semantics) and see if we observe different patterns in both behavior and neural responses in scalp EEG recordings as a function of meaningfulness in words. This dataset will not only create an opportunity for better understanding in how meaningfulness influence VWM but also allow us to directly investigate the interaction between VWM and LTM.
Copyright Notice
The documents distributed here have been provided as a means to ensure timely dissemination of scholarly and technical work on a noncommercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author’s copyright. These works may not be reposted without the explicit permission of the copyright holder. (Notice borrowed from Dave Plaut/Randy O’Reilly)