Object recognition during ambiguous rapid serial visual presentations using time-series EEG decoding

Poster No:

2074 

Submission Type:

Abstract Submission 

Authors:

Almudena Ramirez-Haro1, Denise Moerel1, Genevieve Quek1, Manuel Varlet1,2, Tijl Grootswagers1,3

Institutions:

1The MARCS Institute for Brain, Behaviour and Development, Western Sydney University, Sydney, Australia, 2School of Psychology, Western Sydney University, Sydney, Australia, 3School of Computer, Data and Mathematical Sciences, Western Sydney University, Sydney, Australia

First Author:

Almudena Ramirez-Haro  
The MARCS Institute for Brain, Behaviour and Development, Western Sydney University
Sydney, Australia

Co-Author(s):

Denise Moerel  
The MARCS Institute for Brain, Behaviour and Development, Western Sydney University
Sydney, Australia
Genevieve Quek  
The MARCS Institute for Brain, Behaviour and Development, Western Sydney University
Sydney, Australia
Manuel Varlet  
The MARCS Institute for Brain, Behaviour and Development, Western Sydney University|School of Psychology, Western Sydney University
Sydney, Australia|Sydney, Australia
Tijl Grootswagers  
The MARCS Institute for Brain, Behaviour and Development, Western Sydney University|School of Computer, Data and Mathematical Sciences, Western Sydney University
Sydney, Australia|Sydney, Australia

Introduction:

The success of visual object recognition relies on its robustness and speed. This is highlighted in ambiguous contexts where, even though recognition is still resolved in milliseconds (Thorpe et al., 1996; Wardle & Baker, 2020), the visual system needs more processing time to disentangle the visual complexity of the objects (Felleman & Van Essen, 1991). Manipulating the ambiguity with which objects are presented allows us to investigate the mechanisms involved in recognising ambiguous objects (O'Reilly et al., 2013). For example, behavioural categorisation performance decreased with increasing levels of ambiguity when disrupting processing time in the visual system (Wyatte et al., 2012). Our experiment expands upon this by investigating the effect of different levels of available object information in combination with two different presentation rates on object recognition in the brain, as measured through time-series electroencephalography (EEG) decoding.

Methods:

We used EEG (n = 30) to record the brain responses to 200 natural images of objects in ambiguous situations. We manipulated ambiguity in two ways: by changing the available information of the images, and by reducing processing time by speeding up their presentation (Robinson et al, 2019). Available information was manipulated using a black square to show 75, 50 and 25% of the image, arranged in 15 unique occlusion patterns per image. We reduced processing time by presenting 80 streams of 200 stimuli at either 5Hz or 20Hz, with a total of 80 trials per image. We decoded the object identity and animacy from the neural data, where we trained a classifier using linear discriminant analysis on the intact images and tested it on all stimuli for each of the presentation conditions.

Results:

We found differences in temporal dynamics with different levels of available information and speed. For the 5Hz presentation rate, there was a systematic reduction in object and animacy decoding under decreasing available information. At faster presentation rates, however, we found a difference in animacy decoding compared to object decoding, with a systematic reduction of the average of the accuracy scores for object decoding but with more constant scores throughout the 100, 75 and 50% of the information available in the animacy decoding. We also found that objects with only 50 and 25% of available information were still decodable even at 20Hz when processing time was disrupted.

Conclusions:

Our results complement, with neural decoding, previous literature on behavioural and computational models regarding the reduction of performance with increasing levels of ambiguity, and the disruption of processing times (Wyatte et al., 2012; O'Reilly et al., 2013). One possible explanation for this is the disruption of the feedback connections in different parts of the brain used to complement the first bottom-up feedforward stream when the recognition is ambiguous. The reduction of decoding accuracy in the 20Hz compared to the 5Hz condition, especially with more ambiguous partial images, could be due to the influence of the feedback mechanisms to solve the ambiguity. However, our results suggest that reduced feedback processing can still solve more ambiguous visual recognition, obtaining high accuracy with even less than 50% of the available information in the image at 20Hz. In conclusion, our study provides insight into the contribution of different mechanisms to object recognition varying the ambiguity of the objects and their presentation.

Modeling and Analysis Methods:

Multivariate Approaches

Novel Imaging Acquisition Methods:

EEG 2

Perception, Attention and Motor Behavior:

Perception: Visual 1

Keywords:

Electroencephaolography (EEG)
Multivariate
Perception
Vision

1|2Indicates the priority used for review

Abstract Information

By submitting your proposal, you grant permission for the Organization for Human Brain Mapping (OHBM) to distribute your work in any format, including video, audio print and electronic text through OHBM OnDemand, social media channels, the OHBM website, or other electronic publications and media.

I accept

The Open Science Special Interest Group (OSSIG) is introducing a reproducibility challenge for OHBM 2025. This new initiative aims to enhance the reproducibility of scientific results and foster collaborations between labs. Teams will consist of a “source” party and a “reproducing” party, and will be evaluated on the success of their replication, the openness of the source work, and additional deliverables. Click here for more information. Propose your OHBM abstract(s) as source work for future OHBM meetings by selecting one of the following options:

I do not want to participate in the reproducibility challenge.

Please indicate below if your study was a "resting state" or "task-activation” study.

Resting state

Healthy subjects only or patients (note that patient studies may also involve healthy subjects):

Healthy subjects

Was this research conducted in the United States?

No

Were any human subjects research approved by the relevant Institutional Review Board or ethics panel? NOTE: Any human subjects studies without IRB approval will be automatically rejected.

Yes

Were any animal research approved by the relevant IACUC or other animal research panel? NOTE: Any animal studies without IACUC approval will be automatically rejected.

Not applicable

Please indicate which methods were used in your research:

EEG/ERP

Provide references using APA citation style.

1. Felleman, D. J., & Van Essen, D. C. (1991). Distributed hierarchical processing in the primate cerebral cortex. Cerebral cortex (New York, NY: 1991), 1(1), 1-47.
2. O’Reilly, R. C., Wyatte, D., Herd, S., Mingus, B., & Jilk, D. J. (2013). Recurrent processing during object recognition. Frontiers in psychology, 4, 124.
3. Robinson, A. K., Grootswagers, T., & Carlson, T. A. (2019). The influence of image masking on object representations during rapid serial visual presentation. NeuroImage, 197, 224-231.
4. Thorpe, S., Fize, D., & Marlot, C. (1996). Speed of processing in the human visual system. nature, 381(6582), 520-522.
5. Wardle, S. G., & Baker, C. I. (2020). Recent advances in understanding object recognition in the human brain: deep neural networks, temporal dynamics, and context. F1000Research, 9.
6. Wyatte, D., Curran, T., & O'Reilly, R. (2012). The limits of feedforward vision: Recurrent processing promotes robust object recognition when objects are degraded. Journal of Cognitive Neuroscience, 24(11), 2248-2261.

UNESCO Institute of Statistics and World Bank Waiver Form

I attest that I currently live, work, or study in a country on the UNESCO Institute of Statistics and World Bank List of Low and Middle Income Countries list provided.

No