No Space Left on Device — But df Shows Free Space
Medium

Problem

orders-api on orders-host started throwing 500s. The app log is full of No space left on device, so someone paged "disk full" — but df -h shows the volumes have plenty of free space. Nothing adds up.

Figure out why writes are failing even though there's free disk, point at the exact filesystem, and name the fix.

Initial setup

  • Host: orders-host, Debian 12.
  • Three filesystems: /, /var, /home.
  • The app writes PHP session files under /var/lib/php/sessions.

Acceptance

You've solved it when:

  • You've read the error log and seen No space left on device (ENOSPC)
on a session write.
  • You've shown df -h says the filesystems have free SPACE (so it's not
blocks) and then used df -i to find /var at IUse% 100% / IFree 0 — the filesystem is out of inodes, not space.
  • You've named the fix: delete the runaway population of tiny session
files on /var (in batches — find ... -delete/xargs, never one giant rm *) AND repair the broken cleanup cron so it can't recur. NOT resize the disk (that adds blocks, not inodes), NOT hunt for large files.
Live session
Code
SavedNo commands yet