114 min listen
291: Storage Changes Software
FromBSD Now
ratings:
Length:
73 minutes
Released:
Mar 28, 2019
Format:
Podcast episode
Description
Storage changing software, what makes Unix special, what you need may be “pipeline +Unix commands”, running a bakery on Emacs and PostgreSQL, the ultimate guide to memorable tech talks, light-weight contexts, and more.
##Headlines
###Tracking a storage issue led to software change
Early last year we completed a massive migration that moved our customers’ hosting data off of a legacy datacenter (that we called FR-SD2) onto several new datacenters (that we call FR-SD3, FR-SD5, and FR-SD6) with much more modern, up-to-date infrastructure.
This migration required several changes in both the software and hardware we use, including switching the operating system on our storage units to FreeBSD.
Currently, we use the NFS protocol to provide storage and export the filesystems on Simple Hosting, our web hosting service, and the FreeBSD kernel includes an NFS server for just this purpose.
Problem
While migrating virtual disks of Simple Hosting instances from FR-SD2, we noticed high CPU load spikes on the new storage units.
###What Makes Unix Special
Ever since Unix burst onto the scene within the early '70s, observers within the pc world have been fast to put in writing it off as a unusual working system designed by and for knowledgeable programmers. Regardless of their proclamations, Unix refuses to die. Means again in 1985, Stewart Cheifet puzzled if Unix would turn out to be the usual working system of the longer term on the PBS present “The Laptop Chronicles,” though MS-DOS was effectively in its heyday. In 2018, it is clear that Unix actually is the usual working system, not on desktop PCs, however on smartphones and tablets.
What Makes Unix Special?
It is also the usual system for net servers. The actual fact is, hundreds of thousands of individuals all over the world have interacted with Linux and Unix programs daily, most of whom have by no means written a line of code of their lives.
So what makes Unix so beloved by programmers and different techie sorts? Let’s check out a few of issues this working system has going for it. (For some background on Unix, try The Historical past of Unix: From Bell Labs to the iPhone.)
##News Roundup
What you need may be “pipeline +Unix commands” only
I came across Taco Bell Programming recently, and think this article is worthy to read for every software engineer. The post mentions a scenario which you may consider to use Hadoop to solve but actually xargs may be a simpler and better choice. This reminds me a similar experience: last year a client wanted me to process a data file which has 5 million records. After some investigations, no novel technologies, a concise awk script (less than 10 lines) worked like a charm! What surprised me more is that awk is just a single-thread program, no nifty concurrency involved.
The IT field never lacks “new” technologies: cloud computing, big data, high concurrency, etc. However, the thinkings behind these “fancy” words may date back to the era when Unix arose. Unix command line tools are invaluable treasure. In many cases, picking the right components and using pipeline to glue them can satisfy your requirement perfectly. So spending some time in reviewing Unixcommand line manual instead of chasing state-of-the-art techniques exhaustedly, you may gain more.
BTW, if your data set can be disposed by an awk script, it should not be called “big data”.
Taco Bell Programming
###Running a bakery on Emacs and PostgreSQL
Just over a year ago now, I finally opened the bakery I’d been dreaming of for years. It’s been a big change in my life, from spending all my time sat in front of a computer, to spending most of it making actual stuff. And stuff that makes people happy, at that. It’s been a huge change, but I can’t think of a single job change that’s ever made me as happy as this one.
One of the big changes that came with going pro was that suddenly I was having to work out how much stuff I needed to mix to fill the orders I needed. On the face o
##Headlines
###Tracking a storage issue led to software change
Early last year we completed a massive migration that moved our customers’ hosting data off of a legacy datacenter (that we called FR-SD2) onto several new datacenters (that we call FR-SD3, FR-SD5, and FR-SD6) with much more modern, up-to-date infrastructure.
This migration required several changes in both the software and hardware we use, including switching the operating system on our storage units to FreeBSD.
Currently, we use the NFS protocol to provide storage and export the filesystems on Simple Hosting, our web hosting service, and the FreeBSD kernel includes an NFS server for just this purpose.
Problem
While migrating virtual disks of Simple Hosting instances from FR-SD2, we noticed high CPU load spikes on the new storage units.
###What Makes Unix Special
Ever since Unix burst onto the scene within the early '70s, observers within the pc world have been fast to put in writing it off as a unusual working system designed by and for knowledgeable programmers. Regardless of their proclamations, Unix refuses to die. Means again in 1985, Stewart Cheifet puzzled if Unix would turn out to be the usual working system of the longer term on the PBS present “The Laptop Chronicles,” though MS-DOS was effectively in its heyday. In 2018, it is clear that Unix actually is the usual working system, not on desktop PCs, however on smartphones and tablets.
What Makes Unix Special?
It is also the usual system for net servers. The actual fact is, hundreds of thousands of individuals all over the world have interacted with Linux and Unix programs daily, most of whom have by no means written a line of code of their lives.
So what makes Unix so beloved by programmers and different techie sorts? Let’s check out a few of issues this working system has going for it. (For some background on Unix, try The Historical past of Unix: From Bell Labs to the iPhone.)
##News Roundup
What you need may be “pipeline +Unix commands” only
I came across Taco Bell Programming recently, and think this article is worthy to read for every software engineer. The post mentions a scenario which you may consider to use Hadoop to solve but actually xargs may be a simpler and better choice. This reminds me a similar experience: last year a client wanted me to process a data file which has 5 million records. After some investigations, no novel technologies, a concise awk script (less than 10 lines) worked like a charm! What surprised me more is that awk is just a single-thread program, no nifty concurrency involved.
The IT field never lacks “new” technologies: cloud computing, big data, high concurrency, etc. However, the thinkings behind these “fancy” words may date back to the era when Unix arose. Unix command line tools are invaluable treasure. In many cases, picking the right components and using pipeline to glue them can satisfy your requirement perfectly. So spending some time in reviewing Unixcommand line manual instead of chasing state-of-the-art techniques exhaustedly, you may gain more.
BTW, if your data set can be disposed by an awk script, it should not be called “big data”.
Taco Bell Programming
###Running a bakery on Emacs and PostgreSQL
Just over a year ago now, I finally opened the bakery I’d been dreaming of for years. It’s been a big change in my life, from spending all my time sat in front of a computer, to spending most of it making actual stuff. And stuff that makes people happy, at that. It’s been a huge change, but I can’t think of a single job change that’s ever made me as happy as this one.
One of the big changes that came with going pro was that suddenly I was having to work out how much stuff I needed to mix to fill the orders I needed. On the face o
Released:
Mar 28, 2019
Format:
Podcast episode
Titles in the series (100)
1: BGP & BSD: We kick off the first episode with the latest BSD news, show you how to avoid intrusion detection systems and talk to Peter Hessler about BGP spam blacklists! by BSD Now