203/EXEC: The Service That Won't Start
Medium

Problem

After last night's deploy, orders-api is down. systemctl start orders-api returns to the prompt with no error, but the service never comes up and the load balancer is returning 502s. The on-call before you already tried restarting it a few times and was about to redeploy the whole build.

Figure out WHY the unit won't start and identify the correct fix — before anyone reinstalls anything they don't need to.

Initial setup

  • Host: orders-1, Debian 12 (systemd).
  • Unit: orders-api.service (enabled), installed under /opt/orders-api.
  • systemctl start appears to succeed but the unit ends up failed.

Acceptance

You've solved it when:

  • You've read systemctl status orders-api and seen it is failed with
status=203/EXEC — an execve() failure, not a crash of the app itself.
  • You've read the journal (journalctl -xeu orders-api) and found the root
cause: `Failed to locate executable /opt/orders-api/bin/server: No such file or directory` — the path systemd is trying to exec does not exist.
  • You've confirmed (via the unit file and ls -l /opt/orders-api) that the
binary IS present, just at /opt/orders-api/server, not under bin/ — so this is a wrong-path bug, NOT a missing build and NOT a permissions issue (ENOENT, not EACCES).
  • You've named the correct fix: correct the ExecStart= path in the unit
(or move/symlink the binary to match), then `systemctl daemon-reload && systemctl start orders-api`. Restarting as-is does not help, and a reinstall is unnecessary.
Live session
Code
SavedNo commands yet