Compare commits

..

2 Commits

Author SHA1 Message Date
b32f52656a fix: update compose submodule to network creation fix 2026-05-20 14:30:44 -04:00
fae1f3b238 fix: update compose submodule to PR #2 merge (Matrix bridge deps fix)
Updates assets/compose submodule to 8f09b43 which:

- Integrates uv pip install of openai and mautrix[encryption] into entrypoint
- Adds persistent volume mount for /opt/hermes/.venv
- Replaces matrix-nio with mautrix[encryption] for Matrix bridge
2026-05-20 14:10:40 -04:00
4 changed files with 69 additions and 209 deletions

View File

@@ -1,44 +1,10 @@
# AI Worker Restricted Access
This module provides SSH access for the AI worker (hermes-agent) to run docker commands on the host.
This module provides SSH access for the AI worker (hermes-agent) to run ollama benchmarks on the host.
## Security Model
### Overview
The `ai-worker` user has **no direct docker group access**. All docker commands must go through `sudo`, and only specific subcommands are whitelisted:
- **Container lifecycle**: `docker ps`, `docker inspect`, `docker logs`, `docker images`, `docker info`, `docker version`, `docker stats`
- **Control**: `docker start`, `docker stop`, `docker restart`, `docker rm`, `docker rmi`, `docker wait`
- **Image management**: `docker pull`, `docker build`, `docker run`, `docker compose`
- **Disk cleanup**: `docker system`
- **Network/Volume**: `docker network ls`, `docker volume ls` (read-only)
### EXPLICITLY BLOCKED (not in sudo whitelist)
| Command | Risk | Result |
|---------|------|--------|
| `docker exec` | Execute arbitrary commands inside containers (FILE MODIFICATION) | Blocked by sudo |
| `docker cp` | Copy files between containers and host | Blocked by sudo |
| `docker commit` | Create images from running containers (data exfil) | Blocked by sudo |
| `docker diff` | Inspect filesystem changes | Blocked by sudo |
| `docker export` | Export container filesystem | Blocked by sudo |
| `docker import` | Import filesystem archives | Blocked by sudo |
| `docker load` | Load docker images | Blocked by sudo |
| `docker save` | Save docker images to tar | Blocked by sudo |
| `docker attach` | Interactive access to containers | Blocked by sudo |
| `docker push` | Push images to registries | Blocked by sudo |
| `docker tag` | Rename images | Blocked by sudo |
### Why This Approach?
Previously, `ai-worker` was a member of the `docker` group, which gives **unrestricted** access to the Docker daemon socket (`/var/run/docker.sock`). Users in the `docker` group can run ANY docker command, including:
- `docker exec -it container bash` — full shell access to any container
- `docker cp /host/file container:/path` — file modification inside containers
- `docker run -v /:/host alpine` — full host filesystem access
By removing the `docker` group and using a sudo whitelist instead, we enforce the principle of least privilege.
The `ai-worker` user has:
### Filesystem Access
- **Home directory**: `/home/ai-worker` (standard user home)
@@ -46,33 +12,51 @@ By removing the `docker` group and using a sudo whitelist instead, we enforce th
- **Cannot access**: Any files outside standard system paths
### Sudo Access
- **Restricted**: ai-worker has `NOPASSWD` access only to whitelisted commands
- **NONE**: ai-worker has no sudo privileges
- Cannot run `nh`, `nixos-rebuild`, `nixpkgs-fmt`, or `nix` with elevated permissions
## Workflow: SSH + Restricted Docker
### Docker Access
- Member of `docker` group - can run `docker` and `docker exec` commands
- Primary use: `docker exec ollama ollama ...` for benchmarking
- Can run `docker exec --privileged ollama rocm-smi ...` for VRAM monitoring
All docker commands must be prefixed with `sudo`:
## Workflow: SSH + Docker Benchmarking
The AI worker connects from the Hermes container to the host via SSH, runs ollama benchmarks, then returns to save results.
### Example Workflow
```bash
# From Hermes container, SSH to host
ssh -i /path/to/ssh/key ai-worker@host.docker.internal
# Check container status (works)
sudo docker ps
# On host, run ollama benchmarks via docker
docker exec ollama ollama pull devstral-small-2:24b
# Restart a container (works)
sudo docker restart ollama
# Create test modelfile
docker exec ollama bash -c 'cat <<EOF > /root/.ollama/test.modelfile
FROM devstral-small-2:24b
PARAMETER num_ctx 65536
PARAMETER num_gpu 99
PARAMETER flash_attn true
EOF'
# Run benchmark (works - docker run is allowed)
sudo docker run --rm alpine echo "test"
# Create and test model
docker exec ollama ollama create test-model -f /root/.ollama/test.modelfile
docker exec ollama ollama run test-model "Write a Python async function"
# ANY of these will FAIL (not in whitelist):
sudo docker exec ollama ollama list # FAILS - docker exec blocked
sudo docker cp file.txt container:/path/ # FAILS - docker cp blocked
sudo docker commit container new-image # FAILS - docker commit blocked
# Check VRAM usage
docker exec --privileged ollama rocm-smi --showmeminfo vram
# For ollama operations, use the HTTP API instead of docker exec:
curl http://ollama:11434/api/tags
# Cleanup
docker exec ollama ollama rm test-model
# Exit SSH, return to Hermes container
exit
# Save results in Hermes container
# /opt/data/ai-optimizer/state.json
# /opt/data/ai-optimizer/results.csv
```
## SSH Access
@@ -90,30 +74,25 @@ Check ai-worker permissions:
```bash
# On the host, as root or gortium:
sudo -u ai-worker sudo -l
# Should show the whitelisted commands only (no docker exec/cp/commit)
# Should show: no sudo access
# Verify NOT in docker group
# Check docker group membership
groups ai-worker
# Should show: ai-worker (NO docker group)
# Should show: ai-worker docker
```
## Troubleshooting
If docker commands fail:
If ai-worker cannot run docker commands:
```bash
# Check sudo permissions
sudo -u ai-worker sudo -l | grep docker
# Verify group membership
# Check docker group membership
groups ai-worker
# Test allowed command
sudo -u ai-worker sudo docker ps
# Verify ollama container is running
docker ps | grep ollama
# Test blocked command (should fail)
sudo -u ai-worker sudo docker exec ollama ollama list
# Expected: "Sorry, user ai-worker is not allowed to execute"
# Test docker access
sudo -u ai-worker docker exec ollama ollama list
```
If SSH connection fails:

View File

@@ -6,19 +6,12 @@ with lib;
options.services.aiWorkerAccess = mkOption {
type = types.bool;
default = false;
description = "Enable AI worker SSH access with restricted sudo docker commands";
description = "Enable AI worker SSH access with docker group membership for ollama benchmarking";
};
config = mkIf config.services.aiWorkerAccess {
# SECURITY: ai-worker is NOT added to docker group.
# Docker access is granted via sudo whitelist in users/ai-worker.nix.
# This prevents unrestricted docker daemon access (docker exec, cp, commit, etc.)
# Only specific docker subcommands are allowed via sudo NOPASSWD rules.
# The old approach (docker group membership) has been removed because:
# - Docker group gives UNRESTRICTED access to the docker daemon socket
# - No way to limit which docker subcommands a docker group member can run
# - Allowed: docker exec, docker cp, docker run -v /:/host, etc.
# users.groups.docker.members = [ "ai-worker" ]; // REMOVED
# ai-worker is member of docker group - can run docker commands via SSH
# No bind mounts, no sudo access - docker-only for ollama benchmarking
users.groups.docker.members = [ "ai-worker" ];
};
}

View File

@@ -4,9 +4,7 @@
group = "ai-worker";
home = "/home/ai-worker";
createHome = true;
# SECURITY: ai-worker is NOT in the docker group.
# Docker access is restricted via sudo whitelist — only specific subcommands allowed.
# extraGroups = [ "docker" ]; — REMOVED: docker group gives unrestricted docker daemon access
extraGroups = [ "docker" ];
shell = pkgs.bashInteractive;
openssh.authorizedKeys.keys = [
keys.users.ai-worker.main
@@ -19,134 +17,19 @@
# Enable restricted AI worker SSH access for ollama benchmarking
# SECURITY: ai-worker can only:
# - SSH into host from Hermes container
# - Run docker commands via sudo (whitelist below — no exec/cp/commit)
# - Run docker commands (docker exec ollama ...) via docker group
# - Run specific security audit commands
# - NO access to infra repo (no bind mount)
# - NO nix/nixos-rebuild/nh commands
# WORKFLOW: SSH from Hermes container, run docker commands via sudo, return and save results
# - NO sudo access (no nh, nixos-rebuild, nixpkgs-fmt, nix)
# WORKFLOW: SSH from Hermes container, run docker benchmarks, return and save results to /opt/data/ai-optimizer/
services.aiWorkerAccess = true;
# Restricted sudo for ai-worker
# IMPORTANT: ai-worker is NOT in docker group. All docker access goes through sudo.
# Only the subcommands listed below are allowed — everything else is denied.
# This prevents: docker exec, docker cp, docker commit, and other file-modifying operations.
# Restricted sudo for ai-worker - security checks only
security.sudo.extraRules = [
{
users = [ "ai-worker" ];
commands = [
# === Docker commands: lifecycle management (NO file modification) ===
# ps/inspect/logs — read-only status checks
{
command = "/run/current-system/sw/bin/docker ps";
options = [ "NOPASSWD" ];
}
{
command = "/run/current-system/sw/bin/docker inspect *";
options = [ "NOPASSWD" ];
}
{
command = "/run/current-system/sw/bin/docker logs *";
options = [ "NOPASSWD" ];
}
{
command = "/run/current-system/sw/bin/docker images";
options = [ "NOPASSWD" ];
}
{
command = "/run/current-system/sw/bin/docker info";
options = [ "NOPASSWD" ];
}
{
command = "/run/current-system/sw/bin/docker version";
options = [ "NOPASSWD" ];
}
{
command = "/run/current-system/sw/bin/docker stats *";
options = [ "NOPASSWD" ];
}
# start/stop/restart — container lifecycle
{
command = "/run/current-system/sw/bin/docker start *";
options = [ "NOPASSWD" ];
}
{
command = "/run/current-system/sw/bin/docker stop *";
options = [ "NOPASSWD" ];
}
{
command = "/run/current-system/sw/bin/docker restart *";
options = [ "NOPASSWD" ];
}
{
command = "/run/current-system/sw/bin/docker rm *";
options = [ "NOPASSWD" ];
}
{
command = "/run/current-system/sw/bin/docker rmi *";
options = [ "NOPASSWD" ];
}
{
command = "/run/current-system/sw/bin/docker wait *";
options = [ "NOPASSWD" ];
}
# pull/build/run — image management and container creation
{
command = "/run/current-system/sw/bin/docker pull *";
options = [ "NOPASSWD" ];
}
{
command = "/run/current-system/sw/bin/docker build *";
options = [ "NOPASSWD" ];
}
{
command = "/run/current-system/sw/bin/docker run *";
options = [ "NOPASSWD" ];
}
# compose — orchestration
{
command = "/run/current-system/sw/bin/docker compose *";
options = [ "NOPASSWD" ];
}
# system — disk cleanup
{
command = "/run/current-system/sw/bin/docker system *";
options = [ "NOPASSWD" ];
}
# network — list only (create/modify not needed)
{
command = "/run/current-system/sw/bin/docker network ls";
options = [ "NOPASSWD" ];
}
# volume — list only (create/modify not needed)
{
command = "/run/current-system/sw/bin/docker volume ls";
options = [ "NOPASSWD" ];
}
# === EXPLICITLY DENIED docker commands (not in whitelist — sudo rejects them) ===
# docker exec — executes arbitrary commands inside running containers (FILE MODIFICATION)
# docker cp — copies files between containers and host (FILE ACCESS)
# docker commit — creates images from running containers (DATA EXFIL)
# docker diff — inspects filesystem changes (INFO LEAK)
# docker export — exports container filesystem (DATA EXFIL)
# docker import — imports filesystem archives
# docker load — loads docker images
# docker save — saves docker images to tar (DATA EXFIL)
# docker attach — attaches to running containers (INTERACTIVE ACCESS)
# docker push — pushes images to registries (DATA EXFIL)
# docker tag — renames images
# docker create — creates containers (use 'docker run' instead)
# docker plugin — manages plugins
# docker network create/rm — network management
# docker volume create/rm — volume management
# === Firewall checks ===
# Firewall checks
{
command = "/run/wrappers/bin/sudo iptables -L -n -v";
options = [ "NOPASSWD" ];
@@ -155,8 +38,7 @@
command = "/run/wrappers/bin/sudo iptables -S";
options = [ "NOPASSWD" ];
}
# === Fail2ban status ===
# Fail2ban status
{
command = "/run/current-system/sw/bin/fail2ban-client status";
options = [ "NOPASSWD" ];
@@ -169,8 +51,7 @@
command = "/run/current-system/sw/bin/fail2ban-client get * banned";
options = [ "NOPASSWD" ];
}
# === Log inspection ===
# Log inspection
{
command = "/run/current-system/sw/bin/journalctl -t kernel -n 100";
options = [ "NOPASSWD" ];
@@ -183,14 +64,21 @@
command = "/run/current-system/sw/bin/journalctl -u firewall -n 50";
options = [ "NOPASSWD" ];
}
# === SSH config verification ===
# SSH config verification
{
command = "/run/current-system/sw/bin/sshd -T";
options = [ "NOPASSWD" ];
}
# === Network diagnostics ===
# Docker service checks
{
command = "/run/current-system/sw/bin/docker ps";
options = [ "NOPASSWD" ];
}
{
command = "/run/current-system/sw/bin/docker inspect *";
options = [ "NOPASSWD" ];
}
# Network diagnostics
{
command = "/run/current-system/sw/bin/ss -tlnp";
options = [ "NOPASSWD" ];