34 min listen
Sequential Modeling Enables Scalable Learning for Large Vision Models
Sequential Modeling Enables Scalable Learning for Large Vision Models
ratings:
Length:
35 minutes
Released:
Dec 12, 2023
Format:
Podcast episode
Description
We introduce a novel sequential modeling approach which enables learning a Large Vision Model (LVM) without making use of any linguistic data. To do this, we define a common format,"visual sentences", in which we can represent raw images and videos as well as annotated data sources such as semantic segmentations and depth reconstructions without needing any meta-knowledge beyond the pixels. Once this wide variety of visual data (comprising 420 billion tokens) is represented as sequences, the model can be trained to minimize a cross-entropy loss for next token prediction. By training across various scales of model architecture and data diversity, we provide empirical evidence that our models scale effectively. Many different vision tasks can be solved by designing suitable visual prompts at test time.
2023: Yutong Bai, Xinyang Geng, K. Mangalam, Amir Bar, Alan Yuille, Trevor Darrell, Jitendra Malik, Alexei A. Efros
https://arxiv.org/pdf/2312.00785v1.pdf
2023: Yutong Bai, Xinyang Geng, K. Mangalam, Amir Bar, Alan Yuille, Trevor Darrell, Jitendra Malik, Alexei A. Efros
https://arxiv.org/pdf/2312.00785v1.pdf
Released:
Dec 12, 2023
Format:
Podcast episode
Titles in the series (100)
Editing Large Language Models: Problems, Methods, and Opportunities: Recent advancements in deep learning have precipitated the emergence of large language models (LLMs) which exhibit an impressive aptitude for understanding and producing text akin to human language. Despite the ability to train highly capable LLMs, t... by Papers Read on AI