Autarch/docs/setec_manager_plan.md
2026-03-12 20:51:38 -07:00

24 KiB

Setec App Manager — Architecture Plan

A lightweight Plesk/cPanel replacement built in Go, designed to work with AUTARCH

By darkHal Security Group & Setec Security Labs


1. What Is This?

Setec App Manager is a standalone Go application that turns a bare Debian 13 VPS into a fully managed web hosting platform. It provides:

  • A web dashboard (its own HTTP server on port 9090) for managing the VPS
  • Multi-domain hosting with Nginx reverse proxy management
  • Git-based deployment (clone, pull, restart)
  • SSL/TLS automation via Let's Encrypt (ACME)
  • AUTARCH-native integration — first-class support for deploying and managing AUTARCH instances
  • System administration — users, firewall, packages, monitoring, backups
  • Float Mode backend — WebSocket bridge for AUTARCH Cloud Edition USB passthrough

It is NOT a general-purpose hosting panel. It is purpose-built for running AUTARCH and supporting web applications on a single VPS, with the lightest possible footprint.


2. Technology Stack

Component Choice Rationale
Language Go 1.22+ Single binary, no runtime deps, fast
Web framework net/http + chi router Lightweight, stdlib-based
Templates Go html/template Built-in, secure, fast
Database SQLite (via modernc.org/sqlite) Zero-config, embedded, pure Go
Reverse proxy Nginx (managed configs) Battle-tested, performant
SSL certbot / ACME (golang.org/x/crypto/acme) Let's Encrypt automation
Auth bcrypt + JWT sessions Compatible with AUTARCH's credential format
Firewall ufw / iptables (via exec) Standard Debian tooling
Process mgmt systemd (unit generation) Native Debian service management
WebSocket gorilla/websocket For Float Mode USB bridge + live logs

3. Directory Structure

/opt/setec-manager/
├── setec-manager              # Single Go binary
├── config.yaml                # Manager configuration
├── data/
│   ├── setec.db               # SQLite database (sites, users, logs, jobs)
│   ├── credentials.json       # Admin credentials (bcrypt)
│   └── acme/                  # Let's Encrypt account + certs
├── templates/                 # Embedded HTML templates (via embed.FS)
├── static/                    # Embedded CSS/JS assets
└── nginx/
    ├── sites-available/       # Generated per-domain configs
    └── snippets/              # Shared SSL/proxy snippets

Managed directories on the VPS:

/var/www/                      # Web applications root
├── autarch/                   # AUTARCH instance (cloned from git)
├── example.com/               # Static site or app
└── api.example.com/           # Another app
/etc/nginx/
├── sites-available/           # Setec-generated Nginx configs
├── sites-enabled/             # Symlinks to active sites
└── snippets/
    ├── ssl-params.conf        # Shared SSL settings
    └── proxy-params.conf      # Shared proxy headers
/etc/systemd/system/
├── setec-manager.service      # Manager itself
├── autarch-web.service        # AUTARCH web service
├── autarch-dns.service        # AUTARCH DNS service
└── app-*.service              # Per-app service units

4. Database Schema

-- Sites / domains managed by the panel
CREATE TABLE sites (
    id            INTEGER PRIMARY KEY AUTOINCREMENT,
    domain        TEXT NOT NULL UNIQUE,
    aliases       TEXT DEFAULT '',              -- comma-separated alt domains
    app_type      TEXT NOT NULL DEFAULT 'static', -- 'static', 'reverse_proxy', 'autarch', 'python', 'node'
    app_root      TEXT NOT NULL,                -- /var/www/domain.com
    app_port      INTEGER DEFAULT 0,           -- backend port (for reverse proxy)
    app_entry     TEXT DEFAULT '',              -- entry point (e.g., autarch_web.py, server.js)
    git_repo      TEXT DEFAULT '',              -- git clone URL
    git_branch    TEXT DEFAULT 'main',
    ssl_enabled   BOOLEAN DEFAULT FALSE,
    ssl_cert_path TEXT DEFAULT '',
    ssl_key_path  TEXT DEFAULT '',
    ssl_auto      BOOLEAN DEFAULT TRUE,        -- auto Let's Encrypt
    enabled       BOOLEAN DEFAULT TRUE,
    created_at    DATETIME DEFAULT CURRENT_TIMESTAMP,
    updated_at    DATETIME DEFAULT CURRENT_TIMESTAMP
);

-- System users (for SSH/SFTP access)
CREATE TABLE system_users (
    id            INTEGER PRIMARY KEY AUTOINCREMENT,
    username      TEXT NOT NULL UNIQUE,
    uid           INTEGER,
    home_dir      TEXT,
    shell         TEXT DEFAULT '/bin/bash',
    created_at    DATETIME DEFAULT CURRENT_TIMESTAMP
);

-- Manager users (web panel login)
CREATE TABLE manager_users (
    id            INTEGER PRIMARY KEY AUTOINCREMENT,
    username      TEXT NOT NULL UNIQUE,
    password_hash TEXT NOT NULL,
    role          TEXT DEFAULT 'admin',          -- 'admin', 'viewer'
    force_change  BOOLEAN DEFAULT FALSE,
    last_login    DATETIME,
    created_at    DATETIME DEFAULT CURRENT_TIMESTAMP
);

-- Deployment history
CREATE TABLE deployments (
    id            INTEGER PRIMARY KEY AUTOINCREMENT,
    site_id       INTEGER REFERENCES sites(id),
    action        TEXT NOT NULL,                 -- 'clone', 'pull', 'restart', 'ssl_renew'
    status        TEXT DEFAULT 'pending',        -- 'pending', 'running', 'success', 'failed'
    output        TEXT DEFAULT '',
    started_at    DATETIME,
    finished_at   DATETIME
);

-- Scheduled jobs (SSL renewal, backups, git pull)
CREATE TABLE cron_jobs (
    id            INTEGER PRIMARY KEY AUTOINCREMENT,
    site_id       INTEGER REFERENCES sites(id),  -- NULL for system jobs
    job_type      TEXT NOT NULL,                 -- 'ssl_renew', 'backup', 'git_pull', 'restart'
    schedule      TEXT NOT NULL,                 -- cron expression
    enabled       BOOLEAN DEFAULT TRUE,
    last_run      DATETIME,
    next_run      DATETIME
);

-- Firewall rules
CREATE TABLE firewall_rules (
    id            INTEGER PRIMARY KEY AUTOINCREMENT,
    direction     TEXT DEFAULT 'in',             -- 'in', 'out'
    protocol      TEXT DEFAULT 'tcp',
    port          TEXT NOT NULL,                 -- '80', '443', '8181', '80,443', '1000:2000'
    source        TEXT DEFAULT 'any',
    action        TEXT DEFAULT 'allow',          -- 'allow', 'deny'
    comment       TEXT DEFAULT '',
    enabled       BOOLEAN DEFAULT TRUE
);

-- Float Mode sessions (AUTARCH Cloud Edition)
CREATE TABLE float_sessions (
    id            TEXT PRIMARY KEY,              -- UUID session token
    user_id       INTEGER REFERENCES manager_users(id),
    client_ip     TEXT,
    client_agent  TEXT,                          -- browser user-agent
    usb_bridge    BOOLEAN DEFAULT FALSE,         -- USB passthrough active
    connected_at  DATETIME DEFAULT CURRENT_TIMESTAMP,
    last_ping     DATETIME,
    expires_at    DATETIME
);

-- Backups
CREATE TABLE backups (
    id            INTEGER PRIMARY KEY AUTOINCREMENT,
    site_id       INTEGER REFERENCES sites(id),  -- NULL for full system backup
    backup_type   TEXT DEFAULT 'site',           -- 'site', 'database', 'full'
    file_path     TEXT NOT NULL,
    size_bytes    INTEGER DEFAULT 0,
    created_at    DATETIME DEFAULT CURRENT_TIMESTAMP
);

5. Web Dashboard Routes

5.1 Authentication

GET  /login                   → Login page
POST /login                   → Authenticate (returns JWT cookie)
POST /logout                  → Clear session
GET  /api/auth/status         → Current user info

5.2 Dashboard

GET  /                        → Dashboard overview (system stats, sites, services)
GET  /api/system/info         → CPU, RAM, disk, uptime, load
GET  /api/system/processes    → Top processes by resource usage

5.3 Site Management

GET  /sites                   → Site list page
GET  /sites/new               → New site form
POST /sites                   → Create site (clone repo, generate nginx config, enable)
GET  /sites/:id               → Site detail / edit
PUT  /sites/:id               → Update site config
DELETE /sites/:id             → Remove site (disable nginx, optionally delete files)
POST /sites/:id/deploy        → Git pull + restart
POST /sites/:id/restart       → Restart app service
POST /sites/:id/stop          → Stop app service
POST /sites/:id/start         → Start app service
GET  /sites/:id/logs          → View app logs (journalctl stream)
GET  /sites/:id/logs/stream   → SSE live log stream

5.4 AUTARCH Management

POST /autarch/install         → Clone from git, setup venv, install deps
POST /autarch/update          → Git pull + pip install + restart
GET  /autarch/status          → Service status, version, config
POST /autarch/start           → Start AUTARCH web + DNS
POST /autarch/stop            → Stop all AUTARCH services
POST /autarch/restart         → Restart all AUTARCH services
GET  /autarch/config          → Read autarch_settings.conf
PUT  /autarch/config          → Update autarch_settings.conf
POST /autarch/dns/build       → Build DNS server from source

5.5 SSL / Certificates

GET  /ssl                     → Certificate overview
POST /ssl/:domain/issue       → Issue Let's Encrypt cert (ACME)
POST /ssl/:domain/renew       → Renew cert
POST /ssl/:domain/upload      → Upload custom cert + key
DELETE /ssl/:domain           → Remove cert
GET  /api/ssl/status          → All cert statuses + expiry dates

5.6 Nginx Management

GET  /nginx/status            → Nginx service status + config test
POST /nginx/reload            → Reload nginx (graceful)
POST /nginx/restart           → Restart nginx
GET  /nginx/config/:domain    → View generated config
PUT  /nginx/config/:domain    → Edit config (with validation)
POST /nginx/test              → nginx -t (config syntax check)

5.7 Firewall

GET  /firewall                → Rule list + status
POST /firewall/rules          → Add rule
DELETE /firewall/rules/:id    → Remove rule
POST /firewall/enable         → Enable firewall (ufw enable)
POST /firewall/disable        → Disable firewall
GET  /api/firewall/status     → Current rules + status JSON

5.8 System Users

GET  /users                   → System user list
POST /users                   → Create system user (useradd)
DELETE /users/:id             → Remove system user
POST /users/:id/password      → Reset password
POST /users/:id/ssh-key       → Add SSH public key

5.9 Panel Users

GET  /panel/users             → Manager user list
POST /panel/users             → Create panel user
PUT  /panel/users/:id         → Update (role, password)
DELETE /panel/users/:id       → Remove

5.10 Backups

GET  /backups                 → Backup list
POST /backups/site/:id        → Backup specific site (tar.gz)
POST /backups/full            → Full system backup
POST /backups/:id/restore     → Restore from backup
DELETE /backups/:id           → Delete backup file
GET  /backups/:id/download    → Download backup archive

5.11 Monitoring

GET  /monitor                 → System monitoring page
GET  /api/monitor/cpu         → CPU usage (1s sample)
GET  /api/monitor/memory      → Memory usage
GET  /api/monitor/disk        → Disk usage per mount
GET  /api/monitor/network     → Network I/O stats
GET  /api/monitor/services    → Service status list
WS   /api/monitor/live        → WebSocket live stats stream (1s interval)

5.12 Float Mode Backend

POST /float/register          → Register Float client (returns session token)
WS   /float/bridge/:session   → WebSocket USB bridge (binary frames)
GET  /float/sessions          → Active Float sessions
DELETE /float/sessions/:id    → Disconnect Float session
POST /float/usb/enumerate     → List USB devices on connected client
POST /float/usb/open          → Open USB device on client
POST /float/usb/close         → Close USB device on client
POST /float/usb/transfer      → USB bulk/interrupt transfer via bridge

5.13 Logs

GET  /logs                    → Log viewer page
GET  /api/logs/system         → System logs (journalctl)
GET  /api/logs/nginx          → Nginx access + error logs
GET  /api/logs/setec          → Manager logs
GET  /api/logs/stream         → SSE live log stream (filterable)

6. Core Features Detail

6.1 Site Deployment Flow

When creating a new site:

1. User submits: domain, git_repo (optional), app_type, app_port
2. Manager:
   a. Creates /var/www/<domain>/
   b. If git_repo: git clone <repo> /var/www/<domain>
   c. If python app: creates venv, pip install -r requirements.txt
   d. If node app: npm install
   e. Generates Nginx config from template
   f. Writes to /etc/nginx/sites-available/<domain>
   g. Symlinks to sites-enabled/
   h. If ssl_auto: runs ACME cert issuance
   i. Generates systemd unit for the app
   j. Starts the app service
   k. Reloads nginx
   l. Records deployment in database

6.2 AUTARCH Install Flow

1. git clone https://github.com/DigijEth/autarch.git /var/www/autarch
2. chown -R autarch:autarch /var/www/autarch
3. python3 -m venv /var/www/autarch/venv
4. /var/www/autarch/venv/bin/pip install -r /var/www/autarch/requirements.txt
5. npm install (in /var/www/autarch for hardware JS bundles)
6. bash /var/www/autarch/scripts/build-hw-libs.sh
7. Copy default autarch_settings.conf → update web.host/port, web.secret_key
8. Generate systemd units (autarch-web, autarch-dns)
9. Generate Nginx reverse proxy config (domain → localhost:8181)
10. Issue SSL cert
11. Enable + start services
12. Record deployment

6.3 Nginx Config Templates

Reverse Proxy (AUTARCH / Python / Node):

server {
    listen 80;
    server_name {{.Domain}} {{.Aliases}};

    location /.well-known/acme-challenge/ {
        root /var/www/certbot;
    }

    location / {
        return 301 https://$host$request_uri;
    }
}

server {
    listen 443 ssl http2;
    server_name {{.Domain}} {{.Aliases}};

    ssl_certificate     {{.SSLCertPath}};
    ssl_certificate_key {{.SSLKeyPath}};
    include snippets/ssl-params.conf;

    location / {
        proxy_pass https://127.0.0.1:{{.AppPort}};
        include snippets/proxy-params.conf;
    }

    # WebSocket support (for AUTARCH SSE/WebSocket)
    location /api/ {
        proxy_pass https://127.0.0.1:{{.AppPort}};
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_read_timeout 86400;
        include snippets/proxy-params.conf;
    }
}

Static Site:

server {
    listen 443 ssl http2;
    server_name {{.Domain}};
    root {{.AppRoot}};
    index index.html;

    ssl_certificate     {{.SSLCertPath}};
    ssl_certificate_key {{.SSLKeyPath}};
    include snippets/ssl-params.conf;

    location / {
        try_files $uri $uri/ =404;
    }
}

6.4 Firewall Default Rules

On first setup, Setec Manager installs these ufw rules:

ufw default deny incoming
ufw default allow outgoing
ufw allow 22/tcp    comment "SSH"
ufw allow 80/tcp    comment "HTTP"
ufw allow 443/tcp   comment "HTTPS"
ufw allow 9090/tcp  comment "Setec Manager"
ufw allow 8181/tcp  comment "AUTARCH Web"
ufw allow 53        comment "AUTARCH DNS"
ufw enable

6.5 Float Mode USB Bridge

The Float Mode bridge is the backend half of AUTARCH Cloud Edition's USB passthrough. It works as follows:

┌──────────────────┐     WebSocket      ┌──────────────────┐
│  User's Browser  │◄──────────────────►│  Setec Manager   │
│  (AUTARCH CE)    │                    │  (VPS)           │
└────────┬─────────┘                    └────────┬─────────┘
         │                                       │
         │  Float Applet                         │  USB Commands
         │  (runs on user's PC)                  │  forwarded to
         │                                       │  AUTARCH modules
┌────────▼─────────┐                    ┌────────▼─────────┐
│  WebSocket Client │                    │  AUTARCH Backend │
│  + USB Access     │                    │  (hardware.py    │
│  (native app)     │                    │   equivalent)    │
└────────┬─────────┘                    └──────────────────┘
         │
    ┌────▼────┐
    │ USB Hub │ ← Physical devices (phones, ESP32, etc.)
    └─────────┘

Protocol:

  1. Float applet on user's PC opens WebSocket to wss://domain/float/bridge/<session>
  2. Manager authenticates session token
  3. Binary WebSocket frames carry USB commands and data:
    • Frame type byte: 0x01 = enumerate, 0x02 = open, 0x03 = close, 0x04 = transfer
    • Payload: device descriptor, endpoint, data
  4. Manager translates USB operations into AUTARCH hardware module calls
  5. Results flow back over the same WebSocket

This is the server side only. The client applet is designed in autarch_float.md.


7. Go Package Structure

services/setec-manager/
├── cmd/
│   └── main.go                    # Entry point, flag parsing
├── internal/
│   ├── server/
│   │   ├── server.go              # HTTP server setup, middleware, router
│   │   ├── auth.go                # JWT auth, login/logout handlers
│   │   └── middleware.go          # Logging, auth check, CORS
│   ├── handlers/
│   │   ├── dashboard.go           # Dashboard + system info
│   │   ├── sites.go               # Site CRUD + deployment
│   │   ├── autarch.go             # AUTARCH-specific management
│   │   ├── ssl.go                 # Certificate management
│   │   ├── nginx.go               # Nginx config + control
│   │   ├── firewall.go            # ufw rule management
│   │   ├── users.go               # System + panel user management
│   │   ├── backups.go             # Backup/restore operations
│   │   ├── monitor.go             # System monitoring + WebSocket stream
│   │   ├── logs.go                # Log viewer + SSE stream
│   │   └── float.go               # Float Mode WebSocket bridge
│   ├── nginx/
│   │   ├── config.go              # Nginx config generation
│   │   ├── templates.go           # Go templates for nginx configs
│   │   └── control.go             # nginx reload/restart/test
│   ├── acme/
│   │   └── acme.go                # Let's Encrypt ACME client
│   ├── deploy/
│   │   ├── git.go                 # Git clone/pull operations
│   │   ├── python.go              # Python venv + pip setup
│   │   ├── node.go                # npm install
│   │   └── systemd.go             # Service unit generation + control
│   ├── system/
│   │   ├── info.go                # CPU, RAM, disk, network stats
│   │   ├── firewall.go            # ufw wrapper
│   │   ├── users.go               # useradd/userdel/passwd wrappers
│   │   └── packages.go            # apt wrapper
│   ├── db/
│   │   ├── db.go                  # SQLite connection + migrations
│   │   ├── sites.go               # Site queries
│   │   ├── users.go               # User queries
│   │   ├── deployments.go         # Deployment history queries
│   │   ├── backups.go             # Backup queries
│   │   └── float.go               # Float session queries
│   ├── float/
│   │   ├── bridge.go              # WebSocket USB bridge protocol
│   │   ├── session.go             # Session management
│   │   └── protocol.go            # Binary frame protocol definitions
│   └── config/
│       └── config.go              # YAML config loader
├── web/
│   ├── templates/                 # HTML templates (embedded)
│   │   ├── base.html
│   │   ├── login.html
│   │   ├── dashboard.html
│   │   ├── sites.html
│   │   ├── site_detail.html
│   │   ├── site_new.html
│   │   ├── autarch.html
│   │   ├── ssl.html
│   │   ├── nginx.html
│   │   ├── firewall.html
│   │   ├── users.html
│   │   ├── backups.html
│   │   ├── monitor.html
│   │   ├── logs.html
│   │   └── float.html
│   └── static/                    # CSS/JS assets (embedded)
│       ├── css/style.css
│       └── js/app.js
├── build.sh                       # Build script
├── go.mod
├── config.yaml                    # Default config
└── README.md

8. Configuration (config.yaml)

server:
  host: "0.0.0.0"
  port: 9090
  tls: true
  cert: "/opt/setec-manager/data/acme/manager.crt"
  key: "/opt/setec-manager/data/acme/manager.key"

database:
  path: "/opt/setec-manager/data/setec.db"

nginx:
  sites_available: "/etc/nginx/sites-available"
  sites_enabled: "/etc/nginx/sites-enabled"
  snippets: "/etc/nginx/snippets"
  webroot: "/var/www"
  certbot_webroot: "/var/www/certbot"

acme:
  email: ""                        # Let's Encrypt registration email
  staging: false                   # Use LE staging for testing
  account_dir: "/opt/setec-manager/data/acme"

autarch:
  install_dir: "/var/www/autarch"
  git_repo: "https://github.com/DigijEth/autarch.git"
  git_branch: "main"
  web_port: 8181
  dns_port: 53

float:
  enabled: false
  max_sessions: 10
  session_ttl: "24h"

backups:
  dir: "/opt/setec-manager/data/backups"
  max_age_days: 30
  max_count: 50

logging:
  level: "info"
  file: "/var/log/setec-manager.log"
  max_size_mb: 100
  max_backups: 3

9. Build Targets

Part 1:  Core server, auth, dashboard, site CRUD, Nginx config gen,
         AUTARCH install/deploy, systemd management
         (~4,000 lines)

Part 2:  SSL/ACME automation, firewall management, system users,
         backup/restore, system monitoring
         (~3,500 lines)

Part 3:  Float Mode WebSocket bridge, live log streaming,
         deployment history, scheduled jobs (cron), web UI polish
         (~3,500 lines)

Part 4:  Web UI templates + CSS + JS, full frontend for all features
         (~3,000 lines Go templates + 2,000 lines CSS/JS)

Total estimated: ~16,000 lines

10. Security Considerations

  • Manager runs as root (required for nginx, systemd, useradd)
  • Web panel protected by bcrypt + JWT with short-lived tokens
  • All subprocess calls use exec.Command() with argument arrays (no shell injection)
  • Nginx configs validated with nginx -t before reload
  • ACME challenges served from dedicated webroot (no app interference)
  • Float Mode sessions require authentication + have TTL
  • USB bridge frames validated for protocol compliance
  • SQLite database file permissions: 0600
  • Credentials file permissions: 0600
  • All user-supplied domains validated against DNS before cert issuance
  • Rate limiting on login attempts (5 per minute per IP)

11. First-Run Bootstrap

When setec-manager is run for the first time on a fresh Debian 13 VPS:

1. Detect if first run (no config.yaml or empty database)
2. Interactive TUI setup:
   a. Set admin username + password
   b. Set manager domain (or IP)
   c. Set email for Let's Encrypt
   d. Configure AUTARCH auto-install (y/n)
3. System setup:
   a. apt update && apt install -y nginx certbot python3 python3-venv git ufw
   b. Generate Nginx base config + snippets
   c. Configure ufw default rules
   d. Enable ufw
4. If AUTARCH auto-install:
   a. Clone from git
   b. Full AUTARCH setup (venv, pip, npm, build)
   c. Generate + install systemd units
   d. Generate Nginx reverse proxy
   e. Issue SSL cert
   f. Start AUTARCH
5. Start Setec Manager web dashboard
6. Print access URL