journalctl Tutorial: Read, Filter and Limit Linux Logs

It’s 7 in the morning. A customer says the site was down overnight, and the VPS looks perfectly fine now. What happened? The answer is almost certainly already written down on the server, in the systemd journal. The problem is getting to it: type journalctl with no arguments and you land at the top of hundreds of thousands of lines going back weeks. In this journalctl tutorial we fix that. We will learn to ask the journal precise questions (only this service, only the last hour, only errors, only the boot before the crash) and to watch logs live while we test something. Then we will make sure the journal survives reboots and never grows big enough to fill the disk.

Everything here works the same on Debian 13 and Ubuntu 24.04, on a KVM VPS and also inside an LXC container, with one exception that we point out. Where the two distributions differ, we say so.

What journald and journalctl actually are

Before we type any commands, we need a few definitions. Knowing the vocabulary is half the battle.

A log is a chronological record of things a program wants to tell you: “started”, “listening on port 443”, “could not connect to the database”, “out of memory”. Every serious program on a server writes logs. When something goes wrong, they are the first place to look.

systemd is the init system on Debian and Ubuntu. It is the very first program the kernel starts (it has process ID 1), and it starts, stops and supervises everything else. systemd organizes everything into units. The units you will care about most are services: long-running programs such as ssh.service (the SSH server you are logged in through), cron.service (scheduled jobs), or nginx.service if you installed a web server.

systemd-journald is a small service that collects log messages from almost everywhere on the system:

• the kernel,
• everything a service prints to its standard output and error,
• programs that use the classic syslog() call,
• systemd itself.

It stores them in the journal, a set of binary, indexed files. The important word is indexed. Each entry is not just a line of text but a record with fields attached:

• the timestamp,
• the unit that produced it (_SYSTEMD_UNIT),
• the process ID (_PID),
• the program name (_COMM),
• a severity (PRIORITY),
• the boot it belongs to (_BOOT_ID).

That is why asking for “errors from nginx since yesterday” is instant. Without the index you would be running a slow grep across a dozen rotated text files.

journalctl is the command line tool that reads the journal. Because the files are binary, you never open them with cat or a text editor. You always go through journalctl, which decodes, filters and formats them for you. That is the one tradeoff of the journal: you need the tool. In exchange you get structured filtering, automatic rotation and built-in size limits, which we set up at the end.

Debian vs Ubuntu: what about /var/log/syslog?

Older tutorials tell you to read /var/log/syslog or /var/log/auth.log. Those plain text files are written by a separate program, rsyslog, which receives a copy of every message.

• Debian 13: on a fresh install rsyslog is usually not present. Debian stopped installing it by default with Debian 12, so those files simply do not exist and the journal is your system log.
• Ubuntu 24.04: server images typically still ship rsyslog, so you get both the text files (rotated by logrotate) and the journal.

journalctl works identically on both systems. The journald limits we configure later only affect the journal, not rsyslog’s files.

What you need

• A VPS running Debian 13 or Ubuntu 24.04. Any size works, because the journal is lightweight.
• A user with sudo rights and an SSH session to the server.
• About 20 minutes. Nothing here needs a reboot, although rebooting once at the end is the best way to prove that persistence works.
• Optionally, a service of your own to practice on (nginx, a Node.js app, a database). The examples use ssh and cron because every server has them.

Step 1: Check access and where logs are stored

First, let us see which systemd version we have and whether we can read all the logs:

journalctl --version
journalctl -n 5

The first line of --version shows systemd 257 on Debian 13 and systemd 255 on Ubuntu 24.04. Everything in this guide works on both. -n 5 prints the last five entries. You may see a message like this instead:

Hint: You are currently not seeing messages from other users and the system.
      Users in groups 'adm', 'systemd-journal' can see all messages.

That means your user can only read its own logs. You have two options:

• put sudo in front of every command, or
• add yourself to the systemd-journal group, whose only purpose is to grant read access to the journal.

sudo usermod -aG systemd-journal "$USER"

Group changes only apply to new sessions, so log out and back in. After that, journalctl -n 5 shows system messages too. On Ubuntu, the first user created at install time is often already in the adm group, which has the same effect.

Now, where does the journal live? journald can keep it in one of two places:

• RAM: /run/log/journal, wiped on every reboot.
• Disk: /var/log/journal, kept across reboots.

Check which one you have:

ls -ld /var/log/journal /run/log/journal 2>/dev/null
journalctl --disk-usage
journalctl --list-boots | tail -n 5

--disk-usage answers with a line like Archived and active journals take up 56.0M in the file system., and --list-boots prints one line per recorded boot. How to read the result:

• If /var/log/journal exists and several boots are listed, your journal is already persistent.
• If only /run/log/journal shows up and there is a single boot, logs disappear at every restart. We fix that in step 7.

Write down the disk usage number, because we come back to it in step 8.

Step 2: Read and navigate the journal

Run journalctl with no arguments:

journalctl

It opens the whole journal, oldest entry first, inside a pager. A pager is a program (here less) that lets you scroll through long output instead of having thousands of lines fly past. These are the keys worth knowing:

Starting at the oldest line is rarely what you want, so you will use these options all the time:

journalctl -e                  # open the pager already at the end
journalctl -n 50               # print only the last 50 entries
journalctl -r                  # newest first (reverse order)
journalctl -n 20 --no-pager    # print straight to the terminal, no pager

A typical line looks like this:

Nov 10 08:17:01 web01 CRON[2210]: (root) CMD (cd / && run-parts --report /etc/cron.hourly)

Reading it from left to right:

1. the timestamp, in your server’s time zone,
2. the hostname,
3. the identifier of the program that logged it, with its process ID in brackets,
4. the message.

Two output options help a lot:

• -o short-iso prints full ISO timestamps with the year and time zone. This is useful when you paste logs into a ticket.
• -x adds explanatory help text to some messages from systemd itself.

By default, long lines are cut off at the edge of the screen. You can scroll sideways with the arrow keys. Alternatively, you can make the pager wrap lines by removing the S flag from the less options that journalctl uses:

echo 'export SYSTEMD_LESS=FRXMK' >> ~/.bashrc
source ~/.bashrc

Finally, when journalctl’s output goes into a pipe, it turns off the pager automatically. That means classic tools still work, for example journalctl -b | grep -i error | tail.

Step 3: Filter by service

You will use this filter more than any other. -u (for unit) shows only messages from one service. First, find the exact service name:

systemctl list-units --type=service --state=running

Then ask for its logs:

journalctl -u ssh -n 30
journalctl -u cron --since today

The .service suffix is optional. Watch the name, though: the OpenSSH server unit is called ssh on both Debian and Ubuntu, not sshd. That is why journalctl -u sshd usually returns -- No entries --. On Ubuntu 24.04, SSH is started on demand through ssh.socket, but its log messages still belong to ssh.service. So -u ssh is the right filter on both systems.

You can repeat -u to see several services interleaved in time order, which is perfect for debugging a web stack. You can also use shell-style wildcards. Quote them so the shell does not expand them before journalctl sees them:

journalctl -u nginx -u 'php*-fpm' --since "1 hour ago"

Not sure which units have logged anything at all? -F lists every distinct value of a field found in the journal:

journalctl -F _SYSTEMD_UNIT

Besides units, you can filter on any journal field:

• -t matches the identifier shown in each line. For example, journalctl -t CRON shows every cron job that ran.
• Fields can also be given directly, as in journalctl _PID=1234 or journalctl _COMM=nginx.

When you combine filters, different fields are joined with AND, and the same field repeated is joined with OR.

Step 4: Filter by time and boot

Time filters use --since and --until (short forms -S and -U). They accept both exact and human-friendly values:

journalctl --since "2026-11-09 23:00" --until "2026-11-10 01:00"
journalctl --since "30 min ago"
journalctl --since yesterday --until today
journalctl -u ssh --since today

Times are read in the server’s time zone. If timestamps look a few hours off compared to your watch, run timedatectl to see the configured zone (many VPS images default to UTC). Then you can either change it with sudo timedatectl set-timezone Europe/Bucharest, or add --utc to journalctl and read everything in UTC.

The second way to slice the journal is by boot. Every time the server starts, journald tags new entries with a new boot ID, and journalctl lets you jump between boots:

journalctl --list-boots

IDX BOOT ID                          FIRST ENTRY                 LAST ENTRY
 -1 5c2f0e8d6a8b4f2c9e1a7b3d4c5e6f70 Sun 2026-11-08 09:12:03 EET Mon 2026-11-09 22:40:11 EET
  0 a1b2c3d4e5f60718293a4b5c6d7e8f90 Mon 2026-11-09 22:41:02 EET Tue 2026-11-10 08:05:17 EET

0 is the current boot, -1 is the one before it, and so on:

journalctl -b            # only the current boot
journalctl -b -1 -e      # the previous boot, opened at its last lines
journalctl -b -1 -p err  # only errors from the previous boot

journalctl -b -1 -e is the most useful command after an unexpected restart. The last lines of the previous boot often show exactly what happened: an out-of-memory kill, a kernel panic message, or a clean shutdown that someone requested.

This only works with a persistent journal, which is another reason to do step 7. Inside an LXC container, a “boot” means a start of the container, not of the physical host.

Step 5: Filter by priority and keyword

Every journal entry has a priority. This is a severity level inherited from the old syslog standard, from 0 (emergency) to 7 (debug). -p shows the level you choose plus everything more severe:

journalctl -p err -b              # errors and worse, current boot
journalctl -p warning --since today
journalctl -p err..warning        # only levels 3 and 4
journalctl -p 3                   # numbers work too (3 = err)

journalctl -p err -b is a great first command on a server you do not know. It filters a boot’s thousands of lines down to the handful that matter. Do not panic at every line it shows, though. Some harmless messages are logged as errors, so look for patterns and repeats.

For kernel messages (the same ones dmesg shows), use -k, which automatically limits output to the current boot. This is where you find out-of-memory kills and disk or network driver errors:

journalctl -k -p warning
journalctl -k -g "oom|killed process"

-g (or --grep) filters messages with a regular expression. Its case handling works like this:

• If the pattern is all lowercase, matching ignores case.
• If the pattern contains any uppercase letter, matching becomes case-sensitive.
• You can force either behavior with --case-sensitive=yes or --case-sensitive=no.

-g combines with every other filter. For example, to see today’s failed SSH logins:

journalctl -u ssh --since today -g "failed password|invalid user"

On Debian 13 (OpenSSH 10), these lines are written by a helper process called sshd-session rather than sshd. Filtering by the ssh unit catches both.

In an unprivileged LXC container, -k returns nothing. The kernel belongs to the host, so kernel messages go to the host’s journal, not yours.

Step 6: Follow logs live

-f (follow) prints the last lines and then keeps the terminal open, showing new entries the moment they are written. Think of it as tail -f for the journal:

journalctl -f
journalctl -u ssh -f
journalctl -u nginx -f -n 0     # only new lines, no history
journalctl -f -p warning        # live, but only warnings and worse

Press Ctrl+C to stop. Here is a good exercise:

1. Run journalctl -u ssh -f in your current session.
2. From another terminal, open a second SSH connection to the server.
3. Watch the login appear in the first window in real time.

We use the same trick to debug services every day. Follow the logs in one window, restart or test the service in another (sudo systemctl restart nginx), and read the reaction as it happens instead of guessing.

Step 7: Make the journal persistent

The Storage= setting controls where journald keeps logs:

• volatile: keep logs only in RAM (/run/log/journal).
• persistent: keep logs on disk (/var/log/journal), creating the directory if needed.
• auto: the usual default on Debian 13 and Ubuntu 24.04. Use disk if /var/log/journal exists, otherwise RAM.
• none: drop everything. Do not use this on a server.

With auto, whether you keep logs across reboots depends on a directory existing. That varies between images, and the directory can be removed by accident. We prefer to set it explicitly.

Instead of editing the main configuration file, we create a drop-in. A drop-in is a small file in /etc/systemd/journald.conf.d/ whose settings override the defaults. Package upgrades never touch it, and later it is obvious what you changed. Depending on the release, the commented default file lives at /etc/systemd/journald.conf or under /usr/lib/systemd/. Drop-ins work either way.

sudo mkdir -p /etc/systemd/journald.conf.d
sudo tee /etc/systemd/journald.conf.d/10-persistent.conf > /dev/null <<'EOF'
[Journal]
Storage=persistent
EOF
sudo systemctl restart systemd-journald
sudo journalctl --flush

What each command does:

1. mkdir -p creates the drop-in directory.
2. tee writes a file with a [Journal] section. The header is required and case-sensitive.
3. systemctl restart makes journald read the new setting.
4. journalctl --flush moves anything still sitting in RAM over to disk.

Restarting journald is safe: services keep running and logging. If a file already exists in journald.conf.d on your system (some images ship one), open it and merge the setting into it. Do not create a second file that contradicts it.

To check the result, run these two commands:

systemd-analyze cat-config systemd/journald.conf
ls /var/log/journal/

The first prints the effective configuration, including all drop-ins in the order they apply. The second should show a directory named after your machine ID, containing system.journal. After your next reboot, journalctl --list-boots will show at least two entries.

Step 8: Cap disk usage and clean up

Now for the part that keeps logs from ever filling your VPS. By default, journald applies two limits to the filesystem the journal lives on:

• SystemMaxUse=: the journal may use up to 10% of the filesystem, capped at 4 GB.
• SystemKeepFree=: journald tries to leave at least 15% of the filesystem free, also capped at 4 GB.

When a limit is reached, old files are deleted automatically. That is safe, but on a small VPS, 2 to 4 GB of logs is far more than most people need, and that space is better used for your data:

For a typical small or medium VPS, we use these values in a second drop-in:

sudo tee /etc/systemd/journald.conf.d/20-limits.conf > /dev/null <<'EOF'
[Journal]
SystemMaxUse=500M
SystemKeepFree=1G
SystemMaxFileSize=50M
MaxRetentionSec=1month
RuntimeMaxUse=50M
EOF
sudo systemctl restart systemd-journald

What each line means:

• SystemMaxUse=500M: the journal on disk never grows beyond 500 MB in total. For a single-purpose server this usually holds weeks of history. Busy servers can go to 1G or 2G.
• SystemKeepFree=1G: whatever happens, leave at least 1 GB free on the disk. If space gets tight because of other files, journald deletes old logs before your database runs out of room.
• SystemMaxFileSize=50M: individual journal files are rotated at 50 MB, so old logs are removed in small chunks instead of big jumps.
• MaxRetentionSec=1month: delete entries older than a month, even if there is room left. This helps when you do not want to keep personal data (like IP addresses) longer than needed. Whichever limit is reached first wins.
• RuntimeMaxUse=50M: the same kind of cap for the RAM journal, which is used early during boot or if persistence is ever turned off.

These limits are enforced as new data is written. To free space right now, use vacuuming. Vacuum only deletes archived files, so we combine it with --rotate, which first closes the active files so they become archived:

sudo journalctl --rotate --vacuum-size=500M
# or, by age:
sudo journalctl --rotate --vacuum-time=2weeks
journalctl --disk-usage

You will see lines like Deleted archived journal ... (48.0M)., followed by Vacuuming done, freed .... --disk-usage should now report a number at or below your cap.

Vacuuming is permanent. If you are in the middle of investigating an incident, export what you need first (see the FAQ). Last, check that the remaining files are healthy:

sudo journalctl --verify

Each file should report PASS. If you are on Ubuntu with rsyslog, remember that /var/log/syslog and the other text files are a separate copy managed by logrotate. journald’s caps do not shrink them.

journalctl cheat sheet

Troubleshooting and common mistakes

“No journal files were found”, or the “not seeing messages from other users” hint. Your user cannot read the system journal. Use sudo, or join the systemd-journal group as in step 1 and start a new SSH session.

“Specifying boot ID or boot offset has no effect, no persistent journal was found.” You asked for -b -1, but the journal lives only in RAM. Do step 7. Previous boots are only available from that point on; logs from before it are gone.

-u something prints “– No entries –”. There are three common causes:

• The unit name is wrong. sshd instead of ssh is the classic mistake.
• The service never logged anything in that time window.
• The service writes to its own files. For example, nginx writes access and error logs to /var/log/nginx/ by default, and the journal only gets its start-up messages.

Run journalctl -F _SYSTEMD_UNIT to see which unit names actually exist.

Timestamps are off by a few hours. The server time zone is not what you think. Check it with timedatectl, then change it or use --utc.

Disk usage did not drop after vacuuming. Vacuum skips active files, so add --rotate. If the disk is still full, the journal may not be the culprit. Run sudo du -xh /var --max-depth=2 | sort -h | tail -n 15 and check three places: text files in /var/log, /var/lib/docker (where container logs are stored by default), and backups.

“Suppressed N messages from some.service”. journald rate-limits noisy services to protect the disk. The default is 10000 messages per 30 seconds per service. Usually this means the service has a problem worth fixing. Raising RateLimitBurst= just hides it.

Messages about a corrupted or uncleanly shut down journal file. After a hard reset, journald renames the damaged file (its name now ends in .journal~) and starts a fresh one. This is harmless. journalctl still reads what it can from the old file, and normal rotation removes it later. sudo journalctl --verify tells you which files are affected.

Your config change has no effect. Check these three things:

1. The section header is exactly [Journal].
2. You restarted systemd-journald after the change.
3. No other file overrides yours. Run systemd-analyze cat-config systemd/journald.conf to check. Drop-ins are applied in alphabetical order, and the later file wins.

journalctl -k is empty. You are in an LXC container. Kernel messages are only visible on the host.

Security notes

• Logs are sensitive data. They contain IP addresses, usernames, and sometimes URLs with tokens or e-mail addresses. Membership in systemd-journal or adm means “can read everything that happened on this server”. Give it only to people you would trust with that, and redact logs before pasting them in public forums.
• Do not log secrets. If you write your own services, never print passwords or API keys. Whatever goes to stdout ends up in the journal and stays there until rotation removes it.
• Root can rewrite history. An attacker who gets root can delete or vacuum the journal. For servers that matter, also send logs off the machine. journald can forward them to a central log server with systemd-journal-upload, or you can ship them to a log stack.
• Sealing makes tampering visible. journald supports Forward Secure Sealing (journalctl --setup-keys plus Seal=yes). It makes tampering detectable with --verify, as long as you keep the verification key somewhere other than the server.
• Caps are a security control too. A misbehaving or attacked service can produce gigabytes of log lines. SystemMaxUse= and SystemKeepFree= make sure the result is lost old logs, not a full disk and a crashed database.
• Retention is a privacy decision. If you host services for EU users, MaxRetentionSec= helps you keep IP addresses only as long as you have a reason to.

So, what now?

You can now ask the journal precise questions instead of scrolling:

• which service (-u),
• when (--since, -b),
• how serious (-p),
• containing what (-g),
• and all of it live with -f.

Your journal survives reboots, so the story of the next crash will still be there. It is also capped, so it never competes with your data for disk space. Next time something misbehaves, start with journalctl -p err -b and journalctl -b -1 -e. Nine times out of ten, the answer is right there.

If you want a server to practice on, a VPS from our Romanian data center is ready in minutes, with Debian 13 or Ubuntu 24.04 preinstalled. And if you would rather have someone else watch the logs, configure retention and alerting, and react when something breaks, our server administration team does exactly that every day.

FAQ

Where are systemd logs stored on Debian and Ubuntu?

They are in /var/log/journal/<machine-id>/ when the journal is persistent, or in /run/log/journal/ when it is volatile. The files are binary, so read them with journalctl, not with a text editor. On Ubuntu with rsyslog, you also get text copies in /var/log/syslog and /var/log/auth.log.

Is it safe to delete files in /var/log/journal by hand?

It works, but use sudo journalctl --rotate --vacuum-size=... or --vacuum-time=... instead. Vacuuming removes the oldest archived files cleanly and never touches the file journald is currently writing to.

How do I save logs to a text file to send to someone?

Redirect the output: journalctl -u nginx --since today --no-pager > nginx-today.txt. Even if you put sudo before journalctl, your own shell creates the file, so it belongs to your user and you can copy it off with scp right away. Add -o short-iso for unambiguous timestamps, and remove anything sensitive before sharing.

Does journalctl show Docker container logs?

Not by default. Docker stores container output in its own JSON files, and you read it with docker logs. If you want container logs in the journal, set the logging driver for a service in your compose file (logging: with driver: journald). Then read them with journalctl CONTAINER_NAME=web, and docker logs keeps working too.

How much space should I allow for the journal?

For a small VPS, 200M to 500M is plenty. Busy servers with many services may want 1G to 2G. After a week, check journalctl --disk-usage, then look at the oldest entry with journalctl -o short-iso | head -n 1 to see how many days of history fit inside your cap.