19 min listen
Patterns and Middleware for LLM Applications with Kyle Roche - #659
FromThe TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
Patterns and Middleware for LLM Applications with Kyle Roche - #659
FromThe TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
ratings:
Length:
36 minutes
Released:
Dec 11, 2023
Format:
Podcast episode
Description
Today we’re joined by Kyle Roche, founder and CEO of Griptape to discuss patterns and middleware for LLM applications. We dive into the emerging patterns for developing LLM applications, such as off prompt data—which allows data retrieval without compromising the chain of thought within language models—and pipelines, which are sequential tasks that are given to LLMs that can involve different models for each task or step in the pipeline. We also explore Griptape, an open-source, Python-based middleware stack that aims to securely connect LLM applications to an organization’s internal and external data systems. We discuss the abstractions it offers, including drivers, memory management, rule sets, DAG-based workflows, and a prompt stack. Additionally, we touch on common customer concerns such as privacy, retraining, and sovereignty issues, and several use cases that leverage role-based retrieval methods to optimize human augmentation tasks.
The complete show notes for this episode can be found at twimlai.com/go/659.
The complete show notes for this episode can be found at twimlai.com/go/659.
Released:
Dec 11, 2023
Format:
Podcast episode
Titles in the series (100)
This Week In Machine Learning & AI - 5/20/16: AI at Google I/O, Amazon's Deep Learning DSSTNE: This Week In Machine Learning & AI - May 20, 2016… by The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)