Sentinel

Sentinel is an AI-powered service that autonomously monitors your server, makes decisions, and sends alerts to the notifications center and emails.

Using the command you can check current system health and get suggestions on improving configuration:

opencli sentinel

# opencli sentinel
--------------------------------------------------------------------------------
  Sentinel - OpenPanel server health monitor
--------------------------------------------------------------------------------
Checking services:
[✔] openpanel is active and responding.
[✔] admin is active.
[✔] caddy is active and responding.
[✔] podman.socket is active.
[✔] MariaDB container active and responding.
[✔] csf is active.
[✔] phpmyadmin container is active.
[✔] No OOM errors detected.
--------------------------------------------------------------------------------
Checking traffic:
[✔] No unusual traffic detected on web ports (80|443).
--------------------------------------------------------------------------------
Checking logins, resources, and DNS...
[✔] No new logins to OpenAdmin.
[✔] No active SSH sessions.
[✔] Disk 41% < threshold 85%
[✔] Load 0.42 < threshold 20.
[✔] RAM 38% < threshold 85%
[✔] CPU 7% < threshold 90%
[✔] No SWAP configured.
[✔] All nameservers resolve to local IPs.
[✔] No dead user containers found (checked in 2s).
--------------------------------------------------------------------------------
All Tests Passed!
--------------------------------------------------------------------------------
18 PASS  0 WARN  0 FAIL
--------------------------------------------------------------------------------

Resolved issues​

When a check passes again, Sentinel marks the matching unread notifications for that issue as read and resolved in OpenAdmin > Notifications, where they show a Resolved after … badge, since they no longer need admin attention. For example, if the OpenPanel container was reported as not running and is running on the next check, the OpenPanel container not running! notifications are marked as read. This applies to service and container checks (including recovery after a restart), disk, load, RAM, CPU, SWAP, web traffic, domain and nameserver checks. Event notifications like new logins, SSH logins, OOM kills and reboots are left unread.

It also sends a Resolved: <title> message by email/webhook saying when the issue was first reported, so admins who got the alert also learn that it's over.

# opencli sentinel
...
Checking services:
[✔] openpanel is active and responding.
[✔] Issue resolved, marked notification as read: OpenPanel container not running!
...

What gets emailed​

• Alerts need admin attention. They are logged as unread with a critical or warning severity and sent by email/webhook. Service alerts say what failed, what Sentinel tried, which command to check it with, and include the last log lines. If the same alert is detected again while it's still unread, its count and last seen time are updated instead of adding a new notification or sending another email.
• Info entries are for things Sentinel already fixed on its own, like restarting a stopped container. They are logged as already read for history and are not emailed.
• All alerts from one run are sent as one email and one webhook. With a single alert the subject is its title, with more it's N notifications from Sentinel on <hostname> and the body lists them all. In the email each notification is shown with its severity.

Load, CPU and RAM only alert after they stay over the threshold for 2 checks in a row (about 10 minutes with the default cron), so short spikes don't trigger alerts. CPU usage is measured over 1 second.

# opencli sentinel
...
[!] CPU 96% > threshold 90% (1/2 checks), alerting if it stays high.
...

Notifications log format​

Notifications are stored in /var/log/openpanel/admin/notifications.log, one JSON object per line:

{"id":"1790279898a1b2c3","time":"2026-09-24 19:58:17","last_seen":"2026-09-24 20:08:17","count":3,"status":"unread","severity":"critical","category":"service","source":"sentinel","title":"OpenPanel container not running!","message":"Container openpanel was not running. Sentinel tried to start it, but it is still not running. ...","resolved_at":"2026-09-24 20:13:17"}

• status: unread or read
• severity: critical, warning or info
• category: service, resources, security, dns, traffic, system, update or action
• source: sentinel, update, or the action name for --action notifications
• count and last_seen: how many times the alert was detected while unread, and when it was last detected
• resolved_at: when Sentinel detected that the issue was gone
• details: optional structured data used by the Notifications page, e.g. RAM/CPU/disk usage and top processes, OOM kills, or a link to the crashlog or update log

Lines in the old text format (from before 2.0.12) are still shown on the Notifications page as plain entries.

While an OpenPanel update is running, Sentinel skips all checks, so it doesn't alert about or recreate containers the update is restarting.

# opencli sentinel
--------------------------------------------------------------------------------
  Sentinel - OpenPanel server health monitor
--------------------------------------------------------------------------------
[!] OpenPanel update in progress, skipping checks until it finishes.
--------------------------------------------------------------------------------

Additional flags:

• --startup - run the actions performed after a server reboot: start stopped containers for root and all users and send the reboot notification.

  User containers are started in stages so 50+ accounts don't all hit the disk at once after a reboot: up to half the CPU cores worth of users at a time (min 2, max 8), 2 seconds apart, and the next user waits (up to 60s) while IO pressure is above 40% or load is above 2x the cores. Regular 5-minute runs skip the user container check until the staged start is done.

# opencli sentinel --startup
root: starting openpanel_mysql (was: exited)
root: starting openpanel (was: exited)
root: starting caddy (was: exited)
...
50 users to start, staged (max 4 at a time, 2s apart, waiting on high IO/load)
[1/50] stefan: starting containers
stefan: starting nginx (was: exited)
stefan: starting mysql (was: exited)
[2/50] john: starting containers
john: starting apache (was: exited)
...

• --report - send the Daily Usage Report email (if email alerts are enabled).
• --action=<name> --title=<title> --message=<message> - log a custom notification to OpenAdmin > Notifications (and email/webhook) if notifications are enabled for that action.

Example:

opencli sentinel --action=admin_api --title="OpenAdmin API is on" --message="API access is now on."

# opencli sentinel --action=admin_api --title="OpenAdmin API is on" --message="API access is now on."
[!] Notifications are disabled for action: admin_api

Nothing is printed when the notification is logged. The message above is shown only when notifications for that action are disabled in notifications.ini.