One year in science – 2021

Off we go! The distributed cognition and memory group starts operations

The lab at our inaugural retreat together with the Rademaker lab

We started off our lab in January 2021. This was a difficult year to start a new chapter in one’s life with a majority of the year spent in home office. In this year, we all learned a lot about how to run behavioral experiments online and we have started our first imaging experiments. We also learned a lot about how to be a lab and how many cat memes we need in our lives (a lot!). Finally, we learned to do the paperwork necessary for all this, but we still have some improvement to do on this end. We’ll summarise our work through the year on this page and we are excited for 2022.

Click here for an informal overview talk presented as part of the HU Lecture Series on Cognitive Science 2021 (with suboptimal sound)

A pitch and an orientation could be stored with high fidelity in visual and auditory cortex (green). Two orientations, however, might be spread over sensory and non-sensory regions to minimize interference (blue).

Multi-modal stroage in a dynamic, distributed working memory network

The question where our working memories are stored does not have to have a definite, unitary answer. We know that many parts of the human brain can represent memorized contents – but how are these representations utilized and how do they interact? Vivien Chopurian and Zhiqi Kang joined the lab in 2021 to investigate whether short-term memories for visual and auditory contents are dynamically allocated to minimize interference. It might be easy to remember two very different contents (e.g., a sound and an image). When remembering similar contents (e.g., two images), however, the neural representations of these contents might  interfere with each other, making it harder to arbitrate between them. In order to minimize such interference, these neural representations for different images could be allocated to different parts of the brain. Vivien will present some early results at VSS 2022.

  • Click here for Vivien’s abstract for VSS 2022

The different neural codes for working memory 

How errors during spatial working memory recall relate to verbal labels for spatial locations.

In a second line of research, we investigate the neural codes and strategies used during working memory. We recently submitted a preprint demonstrating that different brain regions use different neural codes to represent memorized contents (work with Chang Yan, Carsten Allefeld, and John-Dylan Haynes). We showed that regions of the ventral stream (areas V4 and V01) use a categorical mnemonic code during working memory, meaning that the colors are retained not as their original hues (its unique mixture of red, green and blue) but rather as belonging to rather broad categories (like ‘red’, ‘green’ and ‘blue’). We found no such benefit in sensory areas. This insight helps us to understand why different regions are used differentially during working memory. Joana Seabra and Andreea-Maria Gui joined the lab this year to dive deeper into this question. In particular, they want to find out what role is played by verbal neural representations and strategies while we memorize locations and orientations. When describing spatial properties (like ‘left’,’right, ‘horizontal’ and ‘diagonal’), can our word usage explain the errors we make during visual working memory?  We are also investigating how verbal distractors affect visual working memory.

·         Here is our color categorizazion preprint (tweeprint summary, here; talk, here)

·         Click here for Joana’s abstract for VSS 2022

Machine learning methods for reading overlapping brain signals

The central tools for our research are statistical and machine learning methods to identify neural representation of working memories and understand how they relate to behavior. A critical challenge for these tools is the overlap and interaction of multiple representations. How can we independently investigate these representations even if they are represented in the same region and how could representations of one item change if another is around? Thomas collaborated with Polina Iamshchinina, Surya Gayet, and Rosanne Rademaker on a commentary on pattern analysis for overlapping representations. We showed how analysis choice for discriminating overlapping representations, such as the selection of an appropriate training set (training on sensory or different mnemonic data) can greatly affect results. Currently, we are developing tools that allow us to quickly evaluate the effect of such choices across a large space of possible design and analyses approaches.

·         Iamshchinina, P., Christophel, T. B., Gayet, S., & Rademaker, R. L. (2021). Essential considerations for exploring visual working memory storage in the human brain. Visual Cognition, 29(7), 425–436. (here)