31 min listen
A decoder-only foundation model for time-series forecasting
A decoder-only foundation model for time-series forecasting
ratings:
Length:
20 minutes
Released:
May 14, 2024
Format:
Podcast episode
Description
Motivated by recent advances in large language models for Natural Language Processing (NLP), we design a time-series foundation model for forecasting whose out-of-the-box zero-shot performance on a variety of public datasets comes close to the accuracy of state-of-the-art supervised forecasting models for each individual dataset. Our model is based on pretraining a patched-decoder style attention model on a large time-series corpus, and can work well across different forecasting history lengths, prediction lengths and temporal granularities.
2023: Abhimanyu Das, Weihao Kong, Rajat Sen, Yichen Zhou
https://arxiv.org/pdf/2310.10688
2023: Abhimanyu Das, Weihao Kong, Rajat Sen, Yichen Zhou
https://arxiv.org/pdf/2310.10688
Released:
May 14, 2024
Format:
Podcast episode
Titles in the series (100)
Scaling Relationship on Learning Mathematical Reasoning with Large Language Models: Mathematical reasoning is a challenging task for large language models (LLMs), while the scaling relationship of it with respect to LLM capacity is under-explored. In this paper, we investigate how the pre-training loss, supervised data amount, and a... by Papers Read on AI