Discover this podcast and so much more

Podcasts are free to enjoy without a subscription. We also offer ebooks, audiobooks, and so much more for just $11.99/month.

Your SSD lies but that's ok | Postgres fsync

Your SSD lies but that's ok | Postgres fsync

FromThe Backend Engineering Show with Hussein Nasser


Your SSD lies but that's ok | Postgres fsync

FromThe Backend Engineering Show with Hussein Nasser

ratings:
Length:
30 minutes
Released:
May 25, 2023
Format:
Podcast episode

Description

fsync is a linux system call that flushes all pages and metadata for a given file to the disk. It is indeed an expensive operation but required for durability especially for database systems. Regular writes that make it to the disk controller are often placed in the SSD local cache to accumulate more writes before getting flushed to the NAND cells.
However when the disk controller receives this flush command it is required to immediately persist all of the data to the NAND cells.

Some SSDs however don't do that because they don't trust the host and no-op the fsync. In this video I explain this in details and go through details on how postgres provide so many options to fine tune fsync

0:00 Intro
1:00 A Write doesn’t write
2:00 File System Page Cache
6:00 Fsync
7:30 SSD Cache
9:20 SSD ignores the flush
9:30 15 Year old Firefox fsync bug
12:30 What happens if SSD loses power
15:00 What options does Postgres exposes?
15:30 open_sync (O_SYNC)
16:15 open_datasync (O_DSYNC)
17:10 O_DIRECT
19:00 fsync
20:50 fdatasync
21:13 fsync = off
23:30 Don’t make your API simple
26:00 Database on metal?
Released:
May 25, 2023
Format:
Podcast episode

Titles in the series (100)

Welcome to the Backend Engineering Show podcast with your host Hussein Nasser. If you like software engineering you’ve come to the right place. I discuss all sorts of software engineering technologies and news with specific focus on the backend. All opinions are my own. Most of my content in the podcast is an audio version of videos I post on my youtube channel here http://www.youtube.com/c/HusseinNasser-software-engineering Buy me a coffee https://www.buymeacoffee.com/hnasr ?‍? Courses I Teach https://husseinnasser.com/courses