SadServers @sadservers.com · 30/09/2026New AWS FinOps hands-on challenge "Auckland": S3 delete & archive old backups sadservers.com/scenario/auc... 010
SadServers @sadservers.com · 29/09/2026(Funny they included the cents in the figure). So Linux sysadmin / DevOps / Infra Eng / Cloud Eng / Platform Eng / SRE / flavour of the month whatever want to call it Eng will still be in demand 🙂 020
SadServers @sadservers.com · 28/09/2026New SadServers AWS cli scenario "Istanbul": Delete unattached AWS EBS volumes , a pretty common cloud engineering task sadservers.com/scenario/ist... 011
SadServers @sadservers.com · 27/09/2026Waterloo Rocketry Reaches New Heights With World's Highest Amateur Liquid Bi-propellant Rocket. SadServers, another endeavour from Waterloo reaching new heights is happy to sponsor them! 🚀 www.waterloorocketry.com 010
SadServers @sadservers.com · 27/09/2026New SadServers scenario based on a real situation: needing to renew a TLS/SSL certificate and having many keys from different certificates without documentation of which key matched the certificate sadservers.com/scenario/sal...sadservers.comSadServers: "Salzburg": The TLS key that still matches - nginx, ssl Troubleshooting LabLinux Troubleshooting Labs Interview DevOps SRE 000
SadServers @sadservers.com · 21/09/2026Brave is a privacy-focused search engine with its own index. Glad to see more people coming to SadServers from it search.brave.com 000
SadServers @sadservers.com · 20/09/2026New optional SadServers UI for registered users (need to log in, Beta) pager.sadservers.com 🚀 000
SadServers @sadservers.com · 19/09/2026"It's like KodeKloud but way better" love.sadservers.com/t/1385b955-4...love.sadservers.comTestimonial from SheaAmazing build and platform! I will definitely share with my own students and use myself. I love how you built this! What an inspiration! It's like KodeKloud but way better in making sure you do the wo... 011
SadServers @sadservers.com · 19/09/2026Tools in our SadServers stack that we love: - @tailscale.com : simple VPN - @sentry.io : application alerts, metrics and logs (especially vs datadog pricing) - @plausible.io: awesome Open Source Google Analytics alternative - Sqlite3: fast database in a file simplifies a lot of things 121
SadServers @sadservers.com · 18/09/2026Four users so far have been able to solve the super-hard "Cadiz" SadServers scenario. Not for the faint of heart; here's the comment of one of those special users: love.sadservers.com/t/5d5b11fc-d...love.sadservers.comTestimonial from S Orekhivskyithis is so good. i spent 8 days on cadiz in between exams. was pain and torture and i cried a bit but i did it and even made a script to automate it. worth every second if you love being desperate eno... 011
SadServers @sadservers.com · 13/09/2026Sunday is for spicy takes. How to write a recruiter email engineers will actually answer (or whatever bait title you want, you get the point) docs.sadservers.com/blog/how-to-...docs.sadservers.comHow to write a recruiter email engineers will actually answerBrought to you by SadServers – Hands-On Linux & DevOps; Real Challenges. Real Infra. Real Skills. (No AI tokens were spent in the creation of this post). I used to get a lot of emails from recruiters... 000
SadServers @sadservers.com · 11/09/2026New #Linux SadServers scenario "Dakar": The half-installed package , what happens if an apt package is not fully installed? practice on a live server sadservers.com/scenario/dakarsadservers.comSadServers: "Dakar": The half-installed package - Packages, realistic-interviews Troubleshooting LabLinux Troubleshooting Labs Interview DevOps SRE 000
SadServers @sadservers.com · 11/09/2026Last SadServers bulletin: list.sadservers.com/archive/0f93... We released 7 new scenarios in the last 4 weeks plus new features 😀list.sadservers.comSadServers Newsletter #22: more scenarios 🤔 000
SadServers @sadservers.com · 10/09/2026Our hardest #Linux scenario yet, expecting very few people to solve it (AI agent couldn't after millions of tokens). Thinking of giving out a prize for defusing the bomb 💣 : sadservers.com/scenario/cadiz 021
SadServers @sadservers.com · 09/09/2026Canada just rebuilt 4 nuclear reactors at Darlington. 4 months EARLY. C$150M UNDER budget. They didn't work on the reactor but a copy of it, some "DevOps' practice here 😀 x.com/BenArendi/st...x.comBensu.eth (@BenArendi) on XThey didn't practice on the reactor. They practiced on a copy of it. Ontario Power Generation (the state-owned company) built the world's first full-scale reactor mockup and had crews rehearse the fu... 010
SadServers @sadservers.com · 09/09/2026SadServers scenario running in a browser sandbox using WebAssembly, thanks to vpod: vpod.sh/usecases/sad...vpod.shsadservers × vpodA SadServers troubleshooting scenario running live in a vpod sandbox — root shell, real Linux, no cloud. 001
SadServers @sadservers.com · 08/09/2026New #kubernetes SadServers scenario "Kyoto" . Investigate and troubleshoot why a recent change is making an API fail, and fix it. sadservers.com/scenario/kyoto 021
SadServers @sadservers.com · 07/09/2026Checklists are a great tool for DevOps/SRE professionals; a must-have for production procedures of any complexity. The book "The Checklist Manifesto" is a good read and the related podcast is just 1 hr tinyurl.com/rkhsenej 110
SadServers @sadservers.com · 05/09/2026Docker healthchecks can be confusing, we explore them in this new SadServers scenario "Spa": The Docker Compose healthcheck that does not heal sadservers.com/scenario/spa 111
SadServers @sadservers.com · 31/08/2026New SadServers scenario "Hangzhou", a bit advanced NGINX QA configuration challenge sadservers.com/scenario/han... 000
SadServers @sadservers.com · 30/08/2026New SadServers scenario "Nara": No ls on this system This is a fun challenge where we took out 'ls' and 'find', still there are at least 9 ways in this server to list and find a target file in a directory structure (and we learn something ourselves 😄) sadservers.com/scenario/narasadservers.comSadServers: "Nara": No ls on this system - linux-other, bash Troubleshooting LabLinux Troubleshooting Labs Interview DevOps SRE 000
SadServers @sadservers.com · 26/08/2026Not sure if to laugh or to cry; getting phishing attempts impersonating my own service 🫤 020
SadServers @sadservers.com · 25/08/2026We'd rather spend time adding tangible value to our users (new scenarios, new features) than doing maintenance. BUT it can be argued a security issue -even if unlikely - could affect users, so we upgrade to keep our bar of no dependencies with Critical or High vulnerabilities. 010
SadServers @sadservers.com · 25/08/2026It's been 35 years since Linus announced his Linux hobby project. It's a light one but I'm surprised not a lot of people have heard about his book talking about the early years. Same with Kernighan's book, which is a great read and anybody in the Unix/Linux world should read imho. 030
SadServers @sadservers.com · 24/08/2026New pricing page. Allows for upgrades directly from the page while blocking things like a distracted user buying a second subscription (it has happened). Also ask and you shall receive: new individual "lifetime" purchase option for the super fans. sadservers.com/pricingsadservers.comSadServers: DevOps & SRE Labs | Job Interview Prep & AssessmentLinux Troubleshooting Labs Interview DevOps SRE 020
SadServers @sadservers.com · 24/08/2026Have we invented a better command dependency tool (DAG) than "make"? it's from the 70s and I use it all the time (my cli is all 'git' and 'make' with the occasional 'docker') 020
SadServers @sadservers.com · 23/08/2026"overnight success" 😀 SadServers has been updated every week in the last two years, either code or scenarios (weekends especially productive, one gap due to external emergency) 140
SadServers @sadservers.com · 21/08/2026When you take out playwright testing our of your Docker image 🤏 031
SadServers @sadservers.com · 18/08/2026another case of retrying making things worse when there's a lot of traffic www.githubstatus.com/incidents/zk...githubstatus.comIncident with GitHub.comGitHub's Status Page - Incident with GitHub.com. 010
SadServers @sadservers.com · 18/08/2026Called this back in May. Copilot has different price tiers now with a high-cost one coming up. 011
SadServers @sadservers.com · 17/08/2026😠 www.githubstatus.com guess we'll have to develop locally, like our ancestors did 141
SadServers @sadservers.com · 12/08/2026New #Linux practical SadServers challenge "Bergen": Port already in use (kind of giving a clue here) , how fast can you fix this issue? sadservers.com/scenario/ber... 010
SadServers @sadservers.com · 11/08/2026Last SadServers newsletter went out list.sadservers.com/archive/3441... Sign up so you don't miss new scenarios and news list.sadservers.com/subscription...list.sadservers.comSadServers Newsletter: new challenges! 000
SadServers @sadservers.com · 11/08/2026New Linux SadServers scenario "Vienna": The Vanishing Process, a systemd hands-on challenge sadservers.com/scenario/vie...sadservers.comSadServers: "Vienna": The Vanishing Process - systemd Troubleshooting LabLinux Troubleshooting Labs Interview DevOps SRE 000
SadServers @sadservers.com · 10/08/2026Gitea is a self-hosted CI/CD platform. Check out SadServers latest practical challenge involving this tool: sadservers.com/labs/gitea/ 000
SadServers @sadservers.com · 08/08/2026Using Sentry for incident, logs and metrics. It's way cheaper than DataDog and the cost can be anticipated (you can cap it too). 010
SadServers @sadservers.com · 06/08/2026What We Learned Running AWS Spot Instances in Production docs.sadservers.com/blog/what-we...docs.sadservers.comWhat We Learned Running AWS Spot Instances in ProductionMost people think of AWS Spot Instances as cheap VMs that can disappear at any time. That’s true — but if Spot is a core part of your production infrastructure, there are a few less obvious details th... 040
SadServers @sadservers.com · 04/08/2026If the AI support agent of a multibillion company like @stripe hallucinates a dashboard option and highlights in a statement the opposite of reality, imagine what the AI agents doing support on other sites are saying. (Unlike other sites at least with Stripe I can RTFM) 010
SadServers @sadservers.com · 01/08/2026Classy and Canadian "not our fault but sorry anyways" tailscale.com/blog/hugging...tailscale.comTailscale in the Hugging Face intrusion: The good news and the bad newsAn AI agent used a stolen Tailscale auth key at Hugging Face. Workload identity federation, flow logs, and safer defaults could have reduced the risk. 072
SadServers @sadservers.com · 29/07/2026New SadServers scenario (gunicorn + ninx + Flask) "Banha" : Intermittent 504 Gateway Time-Out errors sadservers.com/scenario/banhasadservers.comSadServers: "Banha" : Intermittent 504 Gateway Time-Out errors - gunicorn, nginx Troubleshooting LabLinux Troubleshooting Labs Interview DevOps SRE 011
SadServers @sadservers.com · 28/07/2026New SadServers PostgreSQL challenge "Luxor" sadservers.com/scenario/luxor 111
SadServers @sadservers.com · 27/07/2026New SadServers scenario "Philadelphia": htop, btop and zombies 🧟♂️ sadservers.com/scenario/phi...sadservers.comSadServers: "Philadelphia": htop metrics disagree Troubleshooting LabLinux Troubleshooting Labs Interview DevOps SRE 032
SadServers @sadservers.com · 27/07/2026AWS spot instances: - Within the same region, price can vary a lot among AZs (eg less than half). - An instance type that's more expensive on-demand may be cheaper as a spot instance. (eg t3 and t3a) - Spot capacity can suddenly disappear, need to prepare for it. 020
SadServers @sadservers.com · 22/07/2026This brought so many memories, just released: The Java Story | The Official Documentary www.youtube.com/watch?v=ZqGS...youtube.comThe Java Story | The Official DocumentaryYouTube video by CultRepo 010
SadServers @sadservers.com · 17/07/2026Almost had a heart attack but it's all good, some AI agent at AWS woke up with a sense of humor 131
SadServers @sadservers.com · 07/07/2026100 questions (and answers!) related to SadServers, did we miss anything? 😀 sadservers.com/faqsadservers.comFAQ | SadServersAnswers about SadServers scenarios, subscriptions, interviews, accounts, billing, and platform features. 000
SadServers @sadservers.com · 27/06/2026Latest SadServers newsletter is out: 7 new scenarios (including Ansible, Terraform and ELK stack ones) and new Lab section 💪 list.sadservers.com/archive/6009...list.sadservers.comNew scenarios, New labs section 🚀 000