Skip to content

Linux

On this page

The Linux fundamentals every operator needs: how the system is laid out, the commands you use daily, rules of thumb, real use cases, the issues that bite people, and battle-tested best practices.

Basics

Linux is a family of open-source, Unix-like operating systems built around the Linux kernel (first released by Linus Torvalds in 1991). A distribution (distro) bundles the kernel with system libraries (usually GNU), a package manager, and userland tools. Common families:

  • Debian / Ubuntu — apt, .deb packages. Ubuntu LTS is the default server choice for many.
  • RHEL / Rocky / AlmaLinux / Fedora — dnf/yum, .rpm packages. Common in enterprise.
  • SUSE / openSUSE — zypper.
  • Arch — pacman, rolling release.

The Filesystem Hierarchy (FHS)

Linux uses a single rooted tree starting at /. Key directories (per the Filesystem Hierarchy Standard):

Path Purpose
/ Root of everything
/bin, /sbin, /usr/bin Essential and user binaries
/etc System-wide configuration (text files)
/home User home directories
/var Variable data: logs (/var/log), spool, caches
/tmp Temporary files (often cleared on reboot)
/opt Optional/third-party software
/proc, /sys Virtual filesystems exposing kernel/process state
/dev Device files
/mnt, /media Mount points
/root Home for the root user

Users, groups, permissions

Every file has an owner, a group, and a permission triad for owner/group/other, each with read (r=4), write (w=2), execute (x=1).

-rwxr-xr--  1 moin devs  4096 Jun 5 10:00 deploy.sh
 │└┬┘└┬┘└┬┘   └─┬┘ └┬─┘
 │ │  │  │      │   group
 │ │  │  │      owner
 │ │  │  └ other: r--  (4)
 │ │  └─── group: r-x  (5)
 │ └────── owner: rwx  (7)  → chmod 754
 └──────── type: - file, d dir, l symlink

Processes & signals

Each process has a PID; processes form a tree from init/systemd (PID 1). Common signals: SIGTERM (15) polite stop, SIGKILL (9) force kill, SIGHUP (1) reload.

Service management (systemd)

Most modern distros use systemd. A unit (e.g. nginx.service) is managed with systemctl; logs go to the journal (journalctl).

Cheatsheet

pwd                       # print working directory
ls -lah                   # long, all, human-readable sizes
cd -                      # jump to previous directory
tree -L 2                 # directory tree, 2 levels
cp -a src/ dst/           # archive copy (preserve perms/links)
mv old new                # move/rename
rm -rf dir/               # remove recursively (DANGEROUS)
ln -s target linkname     # symbolic link
find / -name '*.conf' -type f 2>/dev/null
du -sh *                  # size of each item here
df -hT                    # disk free, with filesystem type

Viewing & editing text

cat file ; less file ; tail -f /var/log/syslog
head -n 20 file ; wc -l file
grep -rin "error" /var/log/        # recursive, case-insensitive, line numbers
sed -n '10,20p' file               # print lines 10-20
sed -i 's/foo/bar/g' file          # in-place substitute
awk -F: '{print $1}' /etc/passwd   # field 1, ':' delimiter
cut -d',' -f2 data.csv
sort -u ; uniq -c ; tr 'a-z' 'A-Z'

Permissions & ownership

chmod 750 script.sh        # rwxr-x---
chmod u+x,go-w file
chown moin:devs file       # owner:group
umask 022                  # default mask (new files 644, dirs 755)
sudo -l                    # what can I run as sudo?

Processes & resources

ps aux | grep nginx
top ; htop                 # live process view
kill -15 1234 ; kill -9 1234
pkill -f "python app.py"
nice -n 10 cmd ; renice 5 -p PID
free -h                    # memory
uptime ; vmstat 1 ; iostat -xz 1
lsof -i :8080              # who holds port 8080

systemd & logs

systemctl status nginx
systemctl start|stop|restart|reload nginx
systemctl enable --now nginx       # start now + on boot
systemctl list-units --failed
journalctl -u nginx -f             # follow a unit's logs
journalctl -p err -b               # errors since last boot
journalctl --since "1 hour ago"

Packages

# Debian/Ubuntu
sudo apt update && sudo apt upgrade
sudo apt install nginx ; apt search ; apt show pkg
# RHEL/Rocky/Fedora
sudo dnf install nginx ; dnf info ; dnf history

Networking quick hits

ip a ; ip route ; ip -s link
ss -tulpn                  # listening sockets (replaces netstat)
ping -c4 8.8.8.8 ; traceroute host
curl -I https://example.com ; dig example.com +short

Archives & transfer

tar -czvf out.tgz dir/ ; tar -xzvf out.tgz
rsync -avzP src/ user@host:/dst/   # resumable, progress
scp file user@host:/path

Thumb Rules

Rules of thumb

  • Read before you rm -rf. Double-check the path; prefer ls the glob first.
  • Least privilege. Don't log in as root; use sudo for specific commands.
  • Config lives in /etc, data in /var, your stuff in /home.
  • If it's not in a config file, it won't survive a reboot. Persist firewall rules, mounts (/etc/fstab), and sysctl settings.
  • man and --help first, web second. man 5 crontab, cmd --help.
  • Logs tell the truth. When something breaks, journalctl -u <svc> -e before guessing.
  • Idempotency: prefer commands/scripts you can run twice safely.
  • Disk fills silently. df -h and du -sh * are your early-warning system; /var/log and /tmp are usual culprits.
  • A process you can't kill with -15 may need -9, but -9 skips cleanup — try graceful first.

Use Cases

  • Server administration — web (nginx/Apache), DB, app servers; the backbone of cloud and on-prem infrastructure.
  • Automation & scripting — Bash for glue, cron/systemd timers for scheduling.
  • Containers — every Docker/Kubernetes node is Linux; namespaces and cgroups are kernel features.
  • Cloud instances — most EC2/GCE/Azure VMs run Linux images.
  • Embedded & IoT — routers, appliances, devices.
  • Development environments — WSL2, devcontainers, CI runners.

Common Issues

“Permission denied” running a script

The file isn't executable or you lack rights. chmod +x script.sh and check ownership with ls -l. For privileged actions use sudo. A script may also fail because of its shebang (#!/usr/bin/env bash) being wrong or CRLF line endings (dos2unix file).

Disk full but df and du disagree

A deleted file still held open by a process keeps consuming space until the process closes it. Find it with lsof | grep deleted and restart the holder. Also check inode exhaustion: df -i.

Service won't start

systemctl status <svc> and journalctl -u <svc> -e. Common causes: port already in use (ss -tulpn), bad config (validate before reload), missing permissions, or a crash loop. After editing a unit file run systemctl daemon-reload.

“Command not found” though it’s installed

The binary isn't on $PATH, or you need to re-source your shell (hash -r, new shell). Check which cmd / type cmd and echo $PATH.

SSH locked out / host key changed

“REMOTE HOST IDENTIFICATION HAS CHANGED” means the server's key differs (reinstall, or MITM). If expected, remove the stale entry: ssh-keygen -R hostname. Never blindly disable host-key checking on production.

High load average but low CPU

Load counts processes in uninterruptible (D) state — usually blocked on I/O. Investigate disk with iostat -xz 1 and iotop.

Best Practices

  • Use SSH keys, disable password auth, and don't permit root SSH login (PermitRootLogin no).
  • Keep systems patched. Enable unattended security updates where appropriate.
  • Automate configuration with Ansible/Chef instead of hand-editing many hosts.
  • Centralize logs (journald → rsyslog/Loki) and monitor (Prometheus node_exporter).
  • Back up /etc, /home, databases, and data volumes; test restores.
  • Run a firewall (ufw, firewalld, or nftables) with default-deny inbound.
  • Set resource limits and use systemd unit hardening (ProtectSystem, NoNewPrivileges).
  • Prefer text config in version control. Treat servers as cattle, not pets.
  • Time sync matters (chrony/NTP) — clock drift breaks TLS, auth, and logs.

Official Sources