In this episode, Nathan sits down with Tanishq Mathew Abraham, 19-year-old UC Davis grad and one of the youngest people in the world to receive a Ph.D, with a degree in biomedical engineering. Tanishq is the founder of the Medical AI Research Center (MedARC), and with his teammates, recently published a paper: Reconstructions of the Mind's Eye, which encompasses their breakthrough research on reconstructing visual perceptions from fMRI scans into images. In this episode, Nathan and Tanishq talk about the technology behind the fMRI-to-image project, developing the model, and future applications for this research.
Part 2 with Tanishq will be released as the next episode.
The Cognitive Revolution is a part of the Turpentine podcast network. To learn more: Turpentine.co

TIMESTAMPS:
(00:00) Episode Preview
(05:43) The MindEye Project
(09:06) Resemblance between AI reconstruction of mind's eye and visual presented
(10:00) What is a voxel and which regions of the brain were studied?
(10:23) What would the raw data of a voxel be?
(11:44) Is there a time dimension to voxels?
(15:00) Sponsor: Omneky
(17:50) Goals for the MindEye project
(25:57) What is the starting point of the model?
(31:15) Aligning the model: reconstruction vs retrieval
(40:34) Would doing a full end-to-end training be fine for the reconstruction?
(42:15) The role of a limited data set
(43:09) Training separate models per subject
(45:07) Generalizability with a limited dataset
(47:20) Mapping from one high-dimensional space to another
(50:47) Stable Diffusion VAE encoding
(1:00:50) How long does it take to train the model?
(1:03:14) How similar or different are the subjects and their individual models?
(1:05:59) The future of this research: custom models for your brain?
(1:07:34) How much does this research contribute to brain research and wearables?
(1:11:15) Fuzzing data and future research applications

LINKS:
MedARC: medarc.ai
MindEye Paper: https://www.researchgate.net/publication/371136623_Reconstructing_the_Mind's_Eye_fMRI-to-Image_with_Contrastive_Learning_and_Diffusion_Priors

MP3 of this episode: https://chrt.fm/track/993DGA/traffic.megaphone.fm/RINTP1584997572.mp3?updated=1687271014

TWITTER:
@iScienceLuvr (Tanishq)
@MedARC_AI (MedARC)
@CogRev_Podcast
@labenz (Nathan)
@eriktorenberg (Erik)

SPONSOR:
Thank you Omneky (www.omneky.com) for sponsoring The Cognitive Revolution. Omneky is an omnichannel creative generation platform that lets you launch hundreds of thousands of ad iterations that actually work, customized across all platforms, with a click of a button. Omneky combines generative AI and real-time advertising data. Mention "Cog Rev" for 10% off.

MUSIC CREDIT:
MusicLM