Autarch/master_plan.md

AUTARCH Evolution Plan

From CLI Framework to Full-Stack Security Platform

Context

AUTARCH is currently a CLI-only Python security framework at /home/snake/dh_framework/ with 20+ modules across 6 categories (Defense, Offense, Counter, Analyze, OSINT, Simulate), LLM integration (llama.cpp/transformers/Claude), Metasploit RPC integration, and an OSINT sites database with 7,200+ sites.

There is also a UPnP port forwarding script at /home/snake/wg_setec/upnp-renew.sh (using miniupnpc) that forwards ports 443/TCP, 51820/UDP, 8080/TCP but has no cron job installed despite being intended to run every 12 hours.

The goal is to evolve AUTARCH into a portable, cross-platform security platform with a web dashboard, advanced OSINT search engine, Wireshark integration, UPnP management, and Docker packaging -- while preserving the existing CLI interface.

---

Phase 0: Backup & New Working Directory

What: Copy the codebase to ~/autarch, excluding models (9GB).

Steps:

1. mkdir ~/autarch
2. rsync -av --exclude='models/' --exclude='snoop/' --exclude='__pycache__/' /home/snake/dh_framework/ ~/autarch/
3. Verify the copy: ls ~/autarch/ and du -sh ~/autarch/
4. Switch working directory to ~/autarch
5. git init in the new directory for version control going forward

Files: No code changes. Pure copy operation.

---

Phase 1: UPnP Manager Integration

What: Integrate UPnP port forwarding management into AUTARCH with a config-driven approach and cron job management.

New files:

• core/upnp.py -- UPnP manager class wrapping upnpc (miniupnpc CLI)
• modules/upnp_manager.py -- CLI menu module (CATEGORY: defense)

Changes to existing files:

• core/config.py -- Add [upnp] config section with port mappings, internal IP, refresh interval
• core/menu.py -- No changes needed (module auto-discovered)

core/upnp.py design:

class UPnPManager:
    def list_mappings()        # upnpc -l
    def add_mapping(internal_ip, internal_port, external_port, protocol, description)
    def remove_mapping(external_port, protocol)
    def refresh_all()          # Re-add all configured mappings
    def get_external_ip()      # upnpc -e
    def install_cron(interval_hours)   # Write crontab entry
    def uninstall_cron()       # Remove crontab entry
    def get_cron_status()      # Check if cron is active
    def load_mappings_from_config()
    def save_mappings_to_config()

Config section:

[upnp]
enabled = true
internal_ip = 10.0.0.26
refresh_hours = 12
mappings = 443:TCP,51820:UDP,8080:TCP

CLI module menu:

UPnP Port Manager
  [1] Show Current Mappings
  [2] Add Port Mapping
  [3] Remove Port Mapping
  [4] Refresh All Mappings
  [5] Show External IP
  [6] Cron Job Settings (currently: every 12h / not installed)
  [7] Edit Internal IP
  [0] Back

Cron management: Uses subprocess to read/write crontab via crontab -l and crontab - pipe. Adds entry like:

0 */12 * * * /usr/bin/python3 /home/snake/autarch/autarch.py --upnp-refresh > /dev/null 2>&1

Also add to autarch.py: --upnp-refresh CLI flag that calls UPnPManager.refresh_all() and exits (for cron use).

---

Phase 2: Web UI Dashboard (Flask)

What: Add a Flask-based web dashboard running alongside the CLI. Reuse patterns from wg_setec (dark theme, session auth, template inheritance).

New directory structure:

~/autarch/
├── web/
│   ├── __init__.py
│   ├── app.py              # Flask app factory
│   ├── auth.py             # Login/session auth (from wg_setec pattern)
│   ├── routes/
│   │   ├── __init__.py
│   │   ├── dashboard.py    # Main dashboard
│   │   ├── defense.py      # Defense tools page
│   │   ├── offense.py      # Offense tools page
│   │   ├── counter.py      # Counter-intel page
│   │   ├── analyze.py      # Analysis page
│   │   ├── osint.py        # OSINT search engine
│   │   ├── simulate.py     # Simulation page
│   │   ├── settings.py     # Settings management
│   │   ├── upnp.py         # UPnP management page
│   │   └── wireshark.py    # Wireshark page (Phase 4)
│   ├── templates/
│   │   ├── base.html       # Master template (dark theme, sidebar nav)
│   │   ├── login.html
│   │   ├── dashboard.html  # Overview with status cards
│   │   ├── defense.html
│   │   ├── offense.html
│   │   ├── counter.html
│   │   ├── analyze.html
│   │   ├── osint.html      # Advanced search UI
│   │   ├── simulate.html
│   │   ├── settings.html
│   │   ├── upnp.html
│   │   └── wireshark.html
│   └── static/
│       ├── css/style.css   # Dark theme (reuse wg_setec CSS variables)
│       └── js/app.js       # Vanilla JS for interactions

Dashboard landing page shows:

• System status cards (LLM loaded, MSF connected, UPnP active, services running)
• Quick action buttons for each category
• Recent activity/scan results
• Server info (hostname, IP, uptime)

Each category page provides:

• Module listing with descriptions
• Quick-launch buttons that trigger module functions via API
• Results display area
• Settings specific to that category

Flask app startup:

• Add --web flag to autarch.py to start web server
• Add --web-port PORT (default 8080)
• Web server runs in background thread or separate process
• Can run CLI and web simultaneously

Authentication:

• Session-based with bcrypt (same as wg_setec)
• Default admin/admin, forced password change on first login
• Config section [web] for port, host, secret_key

Key reusable patterns from wg_setec:

• CSS variables dark theme (/home/snake/wg_setec/static/css/style.css)
• Template inheritance (/home/snake/wg_setec/templates/base.html)
• Auth decorator (/home/snake/wg_setec/auth.py)
• Flash messages, status dots, data tables

---

Phase 3: OSINT Search Engine (Web)

What: Build an advanced OSINT search interface in the web UI that surpasses the CLI capabilities.

Route: web/routes/osint.py Template: web/templates/osint.html

Search interface features:

• Search box with type selector (username / email / phone / domain / IP)
• Category multi-select checkboxes (filter site categories)
• NSFW toggle
• Scan depth selector (Quick 100 / Standard 500 / Full 7000+)
• Advanced options panel:
  • Thread count slider
  • Timeout setting
  • User-agent selection
  • Proxy support field
  • Export format (JSON/HTML/CSV)

Real-time results (Server-Sent Events):

• GET /osint/search/stream?username=X&... -- SSE endpoint
• Progressive results as sites are checked
• Live progress bar (sites checked / total)
• Results appear in cards grouped by status (good/maybe/bad)
• Filter/sort results client-side

Results display:

• Card layout per found profile: site name, URL (clickable), confidence %, category badge
• Expandable detail: detection method, page title, response info
• Bulk actions: save to dossier, export, open all in new tabs
• Summary statistics bar chart

Backend integration:

• Reuses existing modules/recon.py scanning functions
• Reuses core/sites_db.py for site queries
• New web/tasks.py for async scan management (threading, not Celery -- keep it simple)

Dossier management page:

• List existing dossiers
• Create new dossier
• Link search results to dossier
• View/edit dossier details
• Export dossier as report

---

Phase 4: Wireshark Module (tshark + pyshark)

What: Add packet capture and analysis capabilities using tshark CLI and pyshark Python library.

New files:

• core/wireshark.py -- tshark/pyshark wrapper class
• modules/wireshark.py -- CLI menu module (CATEGORY: analyze)
• web/routes/wireshark.py -- Web dashboard page

core/wireshark.py design:

class WiresharkManager:
    def list_interfaces()          # tshark -D
    def start_capture(interface, filter, duration, output_file)
    def stop_capture()
    def get_capture_stats()
    def read_pcap(filepath)        # pyshark FileCapture
    def live_capture(interface, filter, callback)  # pyshark LiveCapture
    def get_protocol_hierarchy()   # Protocol distribution
    def extract_conversations()    # IP conversations
    def extract_dns_queries()      # DNS analysis
    def extract_http_requests()    # HTTP analysis
    def extract_credentials()      # Plain-text credential detection
    def apply_display_filter(capture, filter_string)
    def export_packets(capture, format)  # JSON/CSV export

CLI module menu:

Wireshark / Packet Analysis
  [1] List Interfaces
  [2] Start Live Capture
  [3] Open PCAP File
  [4] Protocol Analysis
  [5] Conversation Analysis
  [6] DNS Query Analysis
  [7] HTTP Traffic Analysis
  [8] Credential Detection
  [9] Export Results
  [0] Back

Web UI features:

• Interface selector dropdown
• Capture filter input (BPF syntax)
• Start/stop capture buttons
• Live packet stream (SSE)
• Protocol distribution pie chart
• Conversation table
• Packet detail view (expandable tree)
• PCAP file upload and analysis

Dependencies: pyshark, system tshark (part of Wireshark package)

---

Phase 5: Path Portability & Windows Support

What: Make all paths relative/configurable, remove hardcoded paths, add Windows compatibility.

Key changes:

1. Path resolution system (core/paths.py):

import platform, os
from pathlib import Path

def get_app_dir():
    """Returns the application root directory"""
    return Path(__file__).parent.parent

def get_data_dir():
    return get_app_dir() / 'data'

def get_config_path():
    return get_app_dir() / 'autarch_settings.conf'

def get_results_dir():
    return get_app_dir() / 'results'

def is_windows():
    return platform.system() == 'Windows'

def get_platform():
    return platform.system().lower()  # 'linux', 'windows', 'darwin'

2. Audit all files for hardcoded paths:

   • core/config.py -- WG paths, model paths
   • core/msf.py -- msfrpcd path
   • core/upnp.py -- upnpc path
   • modules/*.py -- any /home/snake/ references
   • Replace all with paths.get_*() calls or config-driven paths

3. Platform-specific tool detection:

   • Check for tshark, nmap, upnpc, wg, msfrpcd availability
   • Graceful degradation when tools missing (disable features, show install instructions)
   • Windows: use where instead of which

4. Cross-platform subprocess calls:

   • Use shutil.which() for tool discovery
   • Use pathlib.Path everywhere instead of string concatenation
   • Handle Windows vs Unix shell differences

5. Windows-specific considerations:

   • nmap available on Windows (nmap.org installer)
   • tshark available on Windows (Wireshark installer)
   • upnpc available on Windows (miniupnpc builds)
   • Metasploit available via WSL or native Windows build
   • WireGuard available on Windows
   • No sudo -- use ctypes.windll.shell32.IsUserAnAdmin() check

---

Phase 6: Docker Packaging

What: Create Docker configuration for portable deployment with all dependencies bundled.

New files:

~/autarch/
├── Dockerfile
├── docker-compose.yml
├── .dockerignore
└── scripts/
    ├── entrypoint.sh      # Container startup script
    └── install-tools.sh   # Install nmap, tshark, upnpc, etc.

Dockerfile approach:

• Base: python:3.11-slim
• Install system tools: nmap, tshark, miniupnpc, wireguard-tools
• Install Python deps from requirements.txt
• Copy application code
• Expose ports: 8080 (web UI), 55553 (MSF RPC passthrough)
• Volume mounts: /data (persistent), /models (LLM models)
• Entrypoint: can start CLI, web, or both

docker-compose.yml:

services:
  autarch:
    build: .
    ports:
      - "8080:8080"
    volumes:
      - ./data:/app/data
      - ./models:/app/models
      - ./results:/app/results
    environment:
      - AUTARCH_MODE=web  # or 'cli' or 'both'
    network_mode: host  # Needed for network scanning tools

requirements.txt (consolidated):

flask
bcrypt
pyshark
qrcode
Pillow
requests
llama-cpp-python  # optional
transformers      # optional
torch             # optional
anthropic         # optional
msgpack

---

Execution Order

Step	Phase	Effort	Description
1	Phase 0	5 min	Backup & create ~/autarch
2	Phase 1	1 session	UPnP manager + cron job management
3	Phase 2	2-3 sessions	Flask web dashboard skeleton + all pages
4	Phase 3	1-2 sessions	OSINT search engine with SSE streaming
5	Phase 4	1-2 sessions	Wireshark/tshark integration
6	Phase 5	1-2 sessions	Path portability + Windows support
7	Phase 6	1 session	Docker packaging

Each phase is self-contained and testable before moving to the next.

---

Verification Plan

After each phase:

1. Phase 0: ls ~/autarch/ matches expected structure, python3 ~/autarch/autarch.py --help works
2. Phase 1: UPnP menu accessible, upnpc -l shows mappings, cron installs/uninstalls correctly
3. Phase 2: python3 autarch.py --web starts Flask on :8080, login works, all pages render, CLI still works independently
4. Phase 3: OSINT search returns results in web UI, SSE streaming shows live progress, results match CLI output
5. Phase 4: tshark -D lists interfaces from web UI, capture start/stop works, PCAP analysis shows results
6. Phase 5: No hardcoded paths remain (grep -r '/home/snake' . returns nothing), app runs from any directory
7. Phase 6: docker-compose up starts the app, web UI accessible, tools available inside container

---

Critical Files Reference

Existing File	Role	Reuse For
/home/snake/wg_setec/static/css/style.css	Dark theme CSS	Web UI base theme
/home/snake/wg_setec/templates/base.html	Template inheritance	Web UI base template
/home/snake/wg_setec/auth.py	bcrypt session auth	Web UI authentication
/home/snake/wg_setec/upnp-renew.sh	UPnP port script	Reference for port mappings
core/config.py	Config handler	Add [upnp], [web] sections
core/sites_db.py	Sites DB (7200+ sites)	OSINT search backend
modules/recon.py	OSINT scanner (2193 lines)	OSINT search engine backend
core/menu.py	Module discovery	Auto-discover new modules
autarch.py	Entry point	Add --web, --upnp-refresh flags