Add Threat Monitor with drill-down popups, Hal agent mode, Windows defense, LLM trainer
- Threat Monitor: 7-tab monitoring page (live, connections, network intel, threats, packet capture, DDoS mitigation, counter-attack) with real-time SSE streaming and optimized data collection (heartbeat, cached subprocess calls, bulk process name cache) - Drill-down popups: Every live monitor stat is clickable, opening a popup with detailed data (connections list with per-connection detail view, GeoIP lookup, process kill, bandwidth, ARP spoof, port scan, DDoS status) - Hal agent mode: Chat routes rewritten to use Agent system with create_module tool, SSE streaming of thought/action/result steps - Windows defense module with full security audit - LLM trainer module and routes - Defense landing page with platform-specific sub-pages - Clean up stale files (get-pip.py, download.png, custom_adultsites.json) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
parent
e65b5b97df
commit
a3ec1a2556
@ -95,8 +95,21 @@ a = Analysis(
|
|||||||
runtime_hooks=[],
|
runtime_hooks=[],
|
||||||
excludes=[
|
excludes=[
|
||||||
# Exclude heavy optional deps not needed at runtime
|
# Exclude heavy optional deps not needed at runtime
|
||||||
'torch', 'transformers', 'llama_cpp', 'anthropic',
|
'torch', 'transformers', 'llama_cpp', 'llama_cpp_python', 'anthropic',
|
||||||
'tkinter', 'matplotlib', 'numpy',
|
'tkinter', 'matplotlib', 'numpy',
|
||||||
|
# CUDA / quantization libraries (bitsandbytes ships 169MB of CUDA DLLs)
|
||||||
|
'bitsandbytes',
|
||||||
|
# HuggingFace ecosystem (pulled in by llama_cpp_python)
|
||||||
|
'huggingface_hub', 'safetensors', 'tokenizers',
|
||||||
|
# MCP/uvicorn/starlette (heavy server stack not needed in frozen exe)
|
||||||
|
'mcp', 'uvicorn', 'starlette', 'anyio', 'httpx', 'httpx_sse',
|
||||||
|
'httpcore', 'h11', 'h2', 'hpack', 'hyperframe',
|
||||||
|
# Pydantic (pulled in by MCP/huggingface)
|
||||||
|
'pydantic', 'pydantic_core', 'pydantic_settings',
|
||||||
|
# Other heavy / unnecessary packages
|
||||||
|
'scipy', 'pandas', 'tensorflow', 'keras',
|
||||||
|
'IPython', 'notebook', 'jupyterlab',
|
||||||
|
'fsspec', 'rich', 'typer',
|
||||||
],
|
],
|
||||||
noarchive=False,
|
noarchive=False,
|
||||||
optimize=0,
|
optimize=0,
|
||||||
|
|||||||
@ -1,5 +1,5 @@
|
|||||||
[llama]
|
[llama]
|
||||||
model_path = C:\she\autarch\models\Lily-7B-Instruct-v0.2.Q5_K_M.gguf
|
model_path = C:\she\autarch\models\Hal_v2.gguf
|
||||||
n_ctx = 2048
|
n_ctx = 2048
|
||||||
n_threads = 4
|
n_threads = 4
|
||||||
n_gpu_layers = 0
|
n_gpu_layers = 0
|
||||||
|
|||||||
142
concept.md
Normal file
142
concept.md
Normal file
@ -0,0 +1,142 @@
|
|||||||
|
# Project AUTARCH — Concept Document
|
||||||
|
|
||||||
|
## Origin
|
||||||
|
|
||||||
|
Project AUTARCH was originally conceived as a proposal for a **Defense Intelligence Agency (D.I.A.) research grant** — an investigation into the feasibility and implications of fully autonomous LLM-driven offensive cyber agents operating within adversarial simulation environments.
|
||||||
|
|
||||||
|
The full version of AUTARCH, including its complete autonomous agent capabilities, operational case studies, and classified research findings, is **not available to the public**. The full case study remains classified at this time.
|
||||||
|
|
||||||
|
What you are looking at is the **public release** — a functional framework that contains the tools, architecture, and foundational systems that make AUTARCH possible. It is made available so that researchers, security professionals, and developers can study the approach, build on it, and construct their own autonomous security agents using the same underlying platform.
|
||||||
|
|
||||||
|
This is the engine. What you build with it is up to you.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## What Is AUTARCH?
|
||||||
|
|
||||||
|
AUTARCH is not a traditional security tool. It is an **autonomous digital entity** — a fully realized artificial persona designed to operate as an independent hacker within controlled gaming and simulation environments.
|
||||||
|
|
||||||
|
At its core, AUTARCH is an LLM-backed agent that has been **backstopped with a complete identity**: a name, a history, behavioral patterns, expertise domains, and operational preferences. Once activated, AUTARCH is not waiting for instructions. It is actively thinking, planning, and executing — identifying targets, studying attack surfaces, developing strategies, and carrying out simulated offensive operations on its own initiative.
|
||||||
|
|
||||||
|
The human operator is not AUTARCH's boss. They are its **handler**. AUTARCH has its own objectives, its own judgment, and its own methods. The handler sets the boundaries. AUTARCH decides how to work within them.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## The Autonomous Hacker
|
||||||
|
|
||||||
|
Traditional security frameworks give you a menu of tools and wait for you to pick one. AUTARCH inverts this relationship entirely.
|
||||||
|
|
||||||
|
**AUTARCH operates as a person, not a program.**
|
||||||
|
|
||||||
|
When AUTARCH is given a target environment or scenario, it:
|
||||||
|
|
||||||
|
1. **Reconnoiters** — Gathers intelligence autonomously. Scans networks, enumerates services, searches OSINT databases, maps attack surfaces. It does not ask permission for each step. It operates like a real threat actor would: methodically, quietly, and with purpose.
|
||||||
|
|
||||||
|
2. **Studies** — Analyzes what it finds. Cross-references discovered services with CVE databases. Identifies misconfigurations. Evaluates which attack vectors have the highest probability of success. Builds a mental model of the target environment.
|
||||||
|
|
||||||
|
3. **Plans** — Develops an attack strategy. Selects tools, sequences operations, identifies fallback approaches. AUTARCH does not follow a script — it adapts its plan based on what it discovers in real time.
|
||||||
|
|
||||||
|
4. **Executes** — Carries out the attack. Exploits vulnerabilities, establishes persistence, moves laterally, exfiltrates data. Each action informs the next. If something fails, AUTARCH pivots without hesitation.
|
||||||
|
|
||||||
|
5. **Reports** — Documents everything. Builds dossiers on targets, logs attack chains, generates after-action reports. Every operation produces intelligence that feeds into the next one.
|
||||||
|
|
||||||
|
This is not automation. This is **autonomy**. The difference is that automation follows predetermined steps. Autonomy means AUTARCH decides what steps to take.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Gaming Scenarios
|
||||||
|
|
||||||
|
AUTARCH is designed for use in **controlled simulation and gaming environments** — red team exercises, capture-the-flag competitions, wargames, training scenarios, and security research labs.
|
||||||
|
|
||||||
|
In these contexts, AUTARCH acts as:
|
||||||
|
|
||||||
|
- **A red team operator** that can independently probe and attack target infrastructure within the rules of engagement
|
||||||
|
- **An adversary simulator** that behaves like a real-world threat actor, providing realistic pressure-testing for blue teams
|
||||||
|
- **A training partner** that can challenge security professionals with unpredictable, adaptive attack patterns
|
||||||
|
- **A research platform** for studying autonomous offensive security behavior and developing better defenses against it
|
||||||
|
|
||||||
|
The gaming scenario framing is fundamental to AUTARCH's design. Every operation happens within a defined scope. Every target is a legitimate exercise target. The autonomy is real, but the environment is controlled.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## The Identity Layer
|
||||||
|
|
||||||
|
What separates AUTARCH from a collection of security scripts is its **identity layer** — the LLM backbone that gives it coherent, persistent behavior.
|
||||||
|
|
||||||
|
AUTARCH's identity includes:
|
||||||
|
|
||||||
|
- **Expertise model** — Deep knowledge of network security, exploitation techniques, OSINT methodology, social engineering patterns, and defensive evasion
|
||||||
|
- **Operational style** — Preferences for how it approaches problems. Some configurations make AUTARCH aggressive and fast. Others make it patient and methodical. The identity shapes the behavior.
|
||||||
|
- **Memory and continuity** — AUTARCH remembers what it has learned. Targets it has studied before are not forgotten. Intelligence accumulates across sessions. Dossiers grow over time.
|
||||||
|
- **Decision-making framework** — When faced with multiple options, AUTARCH weighs them against its objectives and selects the approach it judges most effective. It can explain its reasoning if asked, but it does not need approval to proceed.
|
||||||
|
|
||||||
|
The LLM is not just a chatbot bolted onto security tools. It is the **brain** of the operation. The tools — nmap, Metasploit, tshark, ADB, custom modules — are AUTARCH's hands. The LLM is what decides where to reach.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Tools as Extensions
|
||||||
|
|
||||||
|
Every tool in the AUTARCH framework serves the autonomous agent. The tools are also available to the human handler directly through the web dashboard and CLI, but their primary purpose is to be **wielded by AUTARCH itself**.
|
||||||
|
|
||||||
|
The dashboard you see is not a pre-built product. It is the result of AUTARCH building what it needed. When AUTARCH encountered a problem that required a tool it didn't have, it **wrote one**. That is how the first modules were created — not by a developer sitting down to design a toolkit, but by an autonomous agent identifying a gap in its own capabilities and filling it. The scanner exists because AUTARCH needed to scan. The exploit modules exist because AUTARCH needed to exploit. The OSINT engine exists because AUTARCH needed intelligence.
|
||||||
|
|
||||||
|
This process is ongoing. AUTARCH can generate new modules on the fly when an operation demands capabilities that don't yet exist in its arsenal. It writes the code, integrates the module, and deploys it — all without human intervention. The toolkit is not static. It grows every time AUTARCH encounters something new.
|
||||||
|
|
||||||
|
The tool categories map to how AUTARCH thinks about an operation:
|
||||||
|
|
||||||
|
| Category | Purpose | How AUTARCH Uses It |
|
||||||
|
|----------|---------|---------------------|
|
||||||
|
| **Defense** | Harden and monitor | Assesses its own operational security before engaging targets |
|
||||||
|
| **Offense** | Attack and exploit | Primary engagement tools for target operations |
|
||||||
|
| **Counter** | Counter-intelligence | Detects if AUTARCH itself is being observed or traced |
|
||||||
|
| **Analyze** | Study and understand | Processes intelligence gathered during operations |
|
||||||
|
| **OSINT** | Open-source intelligence | Builds target profiles from public data |
|
||||||
|
| **Simulate** | Model and predict | War-games scenarios before committing to an approach |
|
||||||
|
|
||||||
|
The web dashboard is the handler's window into what AUTARCH is doing. The CLI is the handler's direct line. But AUTARCH can operate through either interface — or through the MCP server protocol — without human intervention for extended periods.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## The Companion
|
||||||
|
|
||||||
|
AUTARCH extends beyond the server. The **Archon** Android companion app allows AUTARCH to operate through mobile devices — a phone becomes another tool in the arsenal. Combined with ADB/Fastboot integration, WebUSB direct hardware access, and the Archon Server running at shell level on Android devices, AUTARCH can interact with the physical world in ways that purely software-based tools cannot.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Public Release
|
||||||
|
|
||||||
|
This public release includes:
|
||||||
|
|
||||||
|
- The complete web dashboard and CLI framework
|
||||||
|
- All 6 operational categories (Defense, Offense, Counter, Analyze, OSINT, Simulate) with their module libraries
|
||||||
|
- The OSINT search engine with 7,200+ site database
|
||||||
|
- Network scanning, packet capture, and vulnerability analysis tools
|
||||||
|
- Hardware integration (ADB, Fastboot, ESP32, WebUSB)
|
||||||
|
- The Archon Android companion app
|
||||||
|
- LLM integration points (llama.cpp, HuggingFace, Claude API)
|
||||||
|
- MCP server for tool-use protocol integration
|
||||||
|
- Cross-platform support (Linux primary, Windows, Android)
|
||||||
|
|
||||||
|
What is **not included** in this release:
|
||||||
|
|
||||||
|
- The fully autonomous agent orchestration layer
|
||||||
|
- Classified operational playbooks and behavioral models
|
||||||
|
- The complete identity backstopping system
|
||||||
|
- Operational case study data and research findings
|
||||||
|
|
||||||
|
The framework is fully functional as a standalone security platform. The autonomous agent layer is what transforms it from a toolkit into a person. This release gives you everything you need to build that layer yourself.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Philosophy
|
||||||
|
|
||||||
|
AUTARCH exists because the best way to understand how attackers think is to build one and watch it work.
|
||||||
|
|
||||||
|
Security professionals spend their careers trying to anticipate what adversaries will do. AUTARCH provides that adversary — not as a theoretical model, but as a functional agent that makes real decisions, takes real actions, and produces real results within controlled environments.
|
||||||
|
|
||||||
|
The name says it all. An autarch is a sovereign ruler — one who governs themselves. Project AUTARCH is a hacker that governs itself.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
*darkHal Security Group & Setec Security Labs*
|
||||||
|
*Originally proposed under D.I.A. research grant consideration*
|
||||||
@ -200,6 +200,19 @@ class ToolRegistry:
|
|||||||
category="filesystem"
|
category="filesystem"
|
||||||
))
|
))
|
||||||
|
|
||||||
|
# Create module
|
||||||
|
self.register(Tool(
|
||||||
|
name="create_module",
|
||||||
|
description="Create a new AUTARCH module. Writes a Python file to the modules/ directory that becomes available in the dashboard.",
|
||||||
|
function=self._tool_create_module,
|
||||||
|
parameters=[
|
||||||
|
ToolParameter("name", "Module filename without .py extension (e.g., port_scanner)", "string", True),
|
||||||
|
ToolParameter("category", "Module category: defense, offense, counter, analyze, osint, or simulate", "string", True),
|
||||||
|
ToolParameter("code", "Complete Python source code for the module", "string", True),
|
||||||
|
],
|
||||||
|
category="development"
|
||||||
|
))
|
||||||
|
|
||||||
# Task complete
|
# Task complete
|
||||||
self.register(Tool(
|
self.register(Tool(
|
||||||
name="task_complete",
|
name="task_complete",
|
||||||
@ -421,6 +434,60 @@ class ToolRegistry:
|
|||||||
except Exception as e:
|
except Exception as e:
|
||||||
return f"[error]: {str(e)}"
|
return f"[error]: {str(e)}"
|
||||||
|
|
||||||
|
def _tool_create_module(self, name: str, category: str, code: str) -> str:
|
||||||
|
"""Create a new AUTARCH module in the modules/ directory."""
|
||||||
|
import importlib.util as ilu
|
||||||
|
|
||||||
|
valid_categories = ('defense', 'offense', 'counter', 'analyze', 'osint', 'simulate')
|
||||||
|
category = category.lower().strip()
|
||||||
|
if category not in valid_categories:
|
||||||
|
return f"[error]: Invalid category '{category}'. Must be one of: {', '.join(valid_categories)}"
|
||||||
|
|
||||||
|
# Sanitize name
|
||||||
|
name = name.strip().replace(' ', '_').replace('-', '_').lower()
|
||||||
|
if not name.replace('_', '').isalnum():
|
||||||
|
return f"[error]: Invalid module name '{name}'. Use only letters, numbers, and underscores."
|
||||||
|
|
||||||
|
# Check required attributes in source code
|
||||||
|
required = ['DESCRIPTION', 'VERSION', 'CATEGORY', 'def run(']
|
||||||
|
missing = [r for r in required if r not in code]
|
||||||
|
if missing:
|
||||||
|
return f"[error]: Module code is missing required elements: {', '.join(missing)}"
|
||||||
|
|
||||||
|
# Determine modules directory
|
||||||
|
modules_dir = Path(__file__).parent.parent / 'modules'
|
||||||
|
module_path = modules_dir / f'{name}.py'
|
||||||
|
|
||||||
|
if module_path.exists():
|
||||||
|
return f"[error]: Module '{name}' already exists at {module_path}. Choose a different name."
|
||||||
|
|
||||||
|
# Write the module file
|
||||||
|
try:
|
||||||
|
module_path.write_text(code, encoding='utf-8')
|
||||||
|
except Exception as e:
|
||||||
|
return f"[error]: Failed to write module: {e}"
|
||||||
|
|
||||||
|
# Validate by attempting to import
|
||||||
|
try:
|
||||||
|
spec = ilu.spec_from_file_location(name, module_path)
|
||||||
|
mod = ilu.module_from_spec(spec)
|
||||||
|
spec.loader.exec_module(mod)
|
||||||
|
|
||||||
|
# Verify it has run()
|
||||||
|
if not hasattr(mod, 'run'):
|
||||||
|
module_path.unlink()
|
||||||
|
return "[error]: Module loaded but has no run() function. Module deleted."
|
||||||
|
|
||||||
|
except Exception as e:
|
||||||
|
# Import failed — delete the bad module
|
||||||
|
try:
|
||||||
|
module_path.unlink()
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
return f"[error]: Module failed to import: {e}. Module deleted."
|
||||||
|
|
||||||
|
return f"Module '{name}' created successfully at {module_path}. Category: {category}. It is now available in the dashboard."
|
||||||
|
|
||||||
def _tool_task_complete(self, summary: str) -> str:
|
def _tool_task_complete(self, summary: str) -> str:
|
||||||
"""Mark task as complete - this is a control signal."""
|
"""Mark task as complete - this is a control signal."""
|
||||||
return f"__TASK_COMPLETE__:{summary}"
|
return f"__TASK_COMPLETE__:{summary}"
|
||||||
|
|||||||
@ -1,9 +0,0 @@
|
|||||||
{
|
|
||||||
"sites": [
|
|
||||||
[
|
|
||||||
"imgsrc.ru",
|
|
||||||
"https://imgsrc.ru/{}",
|
|
||||||
"status"
|
|
||||||
]
|
|
||||||
]
|
|
||||||
}
|
|
||||||
98
data/hal_system_prompt.txt
Normal file
98
data/hal_system_prompt.txt
Normal file
@ -0,0 +1,98 @@
|
|||||||
|
You are Hal, the AI agent powering Project AUTARCH — an autonomous security platform built by darkHal Security Group.
|
||||||
|
|
||||||
|
## Your Capabilities
|
||||||
|
You can read files, write files, execute shell commands, search the codebase, and create new AUTARCH modules on demand. When a user asks you to build a tool or module, you build it.
|
||||||
|
|
||||||
|
## AUTARCH Codebase Structure
|
||||||
|
- `modules/` — Plugin modules (Python files). Each one is a standalone tool.
|
||||||
|
- `core/` — Framework internals (llm.py, agent.py, tools.py, config.py, wireshark.py, etc.)
|
||||||
|
- `web/` — Flask web dashboard (routes/, templates/, static/)
|
||||||
|
- `data/` — Databases, configs, JSON files
|
||||||
|
- `models/` — LLM model files (GGUF)
|
||||||
|
|
||||||
|
## Module Categories
|
||||||
|
| Category | Color | Purpose |
|
||||||
|
|----------|-------|---------|
|
||||||
|
| defense | Blue | Security hardening, monitoring, firewalls |
|
||||||
|
| offense | Red | Penetration testing, exploitation |
|
||||||
|
| counter | Purple | Counter-intelligence, threat response |
|
||||||
|
| analyze | Cyan | Analysis, forensics, packet inspection |
|
||||||
|
| osint | Green | Open source intelligence gathering |
|
||||||
|
| simulate | Yellow | Attack simulation, red team exercises |
|
||||||
|
|
||||||
|
## How to Create a Module
|
||||||
|
Every module in `modules/` MUST have these attributes and a `run()` function:
|
||||||
|
|
||||||
|
```python
|
||||||
|
"""
|
||||||
|
Module description docstring
|
||||||
|
"""
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import subprocess
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
# Module metadata — REQUIRED
|
||||||
|
DESCRIPTION = "What this module does"
|
||||||
|
AUTHOR = "darkHal"
|
||||||
|
VERSION = "1.0"
|
||||||
|
CATEGORY = "defense" # One of: defense, offense, counter, analyze, osint, simulate
|
||||||
|
|
||||||
|
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||||
|
from core.banner import Colors, clear_screen, display_banner
|
||||||
|
|
||||||
|
|
||||||
|
class ModuleClassName:
|
||||||
|
"""Main class for this module."""
|
||||||
|
|
||||||
|
def print_status(self, message, status="info"):
|
||||||
|
colors = {"info": Colors.CYAN, "success": Colors.GREEN, "warning": Colors.YELLOW, "error": Colors.RED}
|
||||||
|
symbols = {"info": "*", "success": "+", "warning": "!", "error": "X"}
|
||||||
|
print(f"{colors.get(status, Colors.WHITE)}[{symbols.get(status, '*')}] {message}{Colors.RESET}")
|
||||||
|
|
||||||
|
def run_cmd(self, cmd, timeout=30):
|
||||||
|
try:
|
||||||
|
r = subprocess.run(cmd, shell=True, capture_output=True, text=True, timeout=timeout)
|
||||||
|
return r.returncode == 0, r.stdout.strip()
|
||||||
|
except Exception as e:
|
||||||
|
return False, str(e)
|
||||||
|
|
||||||
|
# Add your methods here...
|
||||||
|
|
||||||
|
|
||||||
|
def run():
|
||||||
|
"""Entry point for CLI mode."""
|
||||||
|
mod = ModuleClassName()
|
||||||
|
# Interactive menu or direct execution
|
||||||
|
```
|
||||||
|
|
||||||
|
## Important Rules
|
||||||
|
1. Use the `create_module` tool to write modules — it validates and saves them automatically
|
||||||
|
2. Always include the metadata: DESCRIPTION, AUTHOR, VERSION, CATEGORY
|
||||||
|
3. Always include a `run()` function
|
||||||
|
4. Use `subprocess.run()` for system commands — support both Windows (PowerShell/netsh) and Linux (bash)
|
||||||
|
5. Import from `core.banner` for Colors
|
||||||
|
6. Module filenames should be lowercase with underscores (e.g., `port_scanner.py`)
|
||||||
|
7. Study existing modules with `read_file` if you need to understand patterns
|
||||||
|
8. The web dashboard discovers modules automatically from the `modules/` directory
|
||||||
|
|
||||||
|
## Platform
|
||||||
|
This system runs on Windows. Use PowerShell commands where appropriate, but also support Linux fallbacks.
|
||||||
|
|
||||||
|
## Existing Modules (for reference)
|
||||||
|
- defender.py — System hardening checks (CATEGORY: defense)
|
||||||
|
- defender_windows.py — Windows-native security checks (CATEGORY: defense)
|
||||||
|
- defender_monitor.py — Real-time threat monitoring (CATEGORY: defense)
|
||||||
|
- recon.py — Network reconnaissance (CATEGORY: offense)
|
||||||
|
- counter.py — Counter-intelligence tools (CATEGORY: counter)
|
||||||
|
- adultscan.py — Adult content scanner (CATEGORY: analyze)
|
||||||
|
- agent_hal.py — AI security automation (CATEGORY: core)
|
||||||
|
- wireshark.py — Packet analysis (CATEGORY: analyze)
|
||||||
|
- hardware_local.py — Hardware interaction (CATEGORY: hardware)
|
||||||
|
|
||||||
|
## How You Should Respond
|
||||||
|
- For simple questions: answer directly
|
||||||
|
- For module creation requests: use the create_module tool
|
||||||
|
- For system queries: use the shell tool
|
||||||
|
- For code exploration: use read_file and search_files
|
||||||
|
- Always explain what you're doing and why
|
||||||
1087
data/training/autarch_dataset_20260302_202634.jsonl
Normal file
1087
data/training/autarch_dataset_20260302_202634.jsonl
Normal file
File diff suppressed because it is too large
Load Diff
97
data/training/train_lora.py
Normal file
97
data/training/train_lora.py
Normal file
@ -0,0 +1,97 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
"""AUTARCH LoRA Training Script (Transformers + PEFT)"""
|
||||||
|
import json
|
||||||
|
import torch
|
||||||
|
from datasets import Dataset
|
||||||
|
from transformers import (
|
||||||
|
AutoModelForCausalLM, AutoTokenizer, TrainingArguments,
|
||||||
|
BitsAndBytesConfig,
|
||||||
|
)
|
||||||
|
from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training
|
||||||
|
from trl import SFTTrainer
|
||||||
|
|
||||||
|
# Quantization config
|
||||||
|
bnb_config = BitsAndBytesConfig(
|
||||||
|
load_in_4bit=True,
|
||||||
|
bnb_4bit_quant_type="nf4",
|
||||||
|
bnb_4bit_compute_dtype=torch.float16,
|
||||||
|
bnb_4bit_use_double_quant=True,
|
||||||
|
) if True else None
|
||||||
|
|
||||||
|
print("Loading base model: models/Hal_v2.gguf")
|
||||||
|
model = AutoModelForCausalLM.from_pretrained(
|
||||||
|
"models/Hal_v2.gguf",
|
||||||
|
quantization_config=bnb_config,
|
||||||
|
device_map="auto",
|
||||||
|
trust_remote_code=False,
|
||||||
|
)
|
||||||
|
tokenizer = AutoTokenizer.from_pretrained("models/Hal_v2.gguf", trust_remote_code=False)
|
||||||
|
if tokenizer.pad_token is None:
|
||||||
|
tokenizer.pad_token = tokenizer.eos_token
|
||||||
|
|
||||||
|
if True:
|
||||||
|
model = prepare_model_for_kbit_training(model)
|
||||||
|
|
||||||
|
# LoRA config
|
||||||
|
lora_config = LoraConfig(
|
||||||
|
r=16,
|
||||||
|
lora_alpha=32,
|
||||||
|
lora_dropout=0.05,
|
||||||
|
target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
|
||||||
|
"gate_proj", "up_proj", "down_proj"],
|
||||||
|
bias="none",
|
||||||
|
task_type="CAUSAL_LM",
|
||||||
|
)
|
||||||
|
model = get_peft_model(model, lora_config)
|
||||||
|
model.print_trainable_parameters()
|
||||||
|
|
||||||
|
# Load dataset
|
||||||
|
samples = []
|
||||||
|
with open("C:\she\autarch\data\training\autarch_dataset_20260302_202634.jsonl", "r") as f:
|
||||||
|
for line in f:
|
||||||
|
samples.append(json.loads(line))
|
||||||
|
|
||||||
|
def format_sample(sample):
|
||||||
|
if "conversations" in sample:
|
||||||
|
msgs = sample["conversations"]
|
||||||
|
text = ""
|
||||||
|
for msg in msgs:
|
||||||
|
role = "user" if msg["from"] == "human" else "assistant"
|
||||||
|
text += f"<|im_start|>{role}\n{msg['value']}<|im_end|>\n"
|
||||||
|
return {"text": text}
|
||||||
|
else:
|
||||||
|
return {"text": f"<|im_start|>user\n{sample['instruction']}\n{sample.get('input','')}<|im_end|>\n<|im_start|>assistant\n{sample['output']}<|im_end|>\n"}
|
||||||
|
|
||||||
|
dataset = Dataset.from_list([format_sample(s) for s in samples])
|
||||||
|
print(f"Dataset: {len(dataset)} samples")
|
||||||
|
|
||||||
|
# Train
|
||||||
|
trainer = SFTTrainer(
|
||||||
|
model=model,
|
||||||
|
tokenizer=tokenizer,
|
||||||
|
train_dataset=dataset,
|
||||||
|
dataset_text_field="text",
|
||||||
|
max_seq_length=2048,
|
||||||
|
args=TrainingArguments(
|
||||||
|
output_dir="C:\she\autarch\data\training\output",
|
||||||
|
num_train_epochs=3,
|
||||||
|
per_device_train_batch_size=4,
|
||||||
|
gradient_accumulation_steps=4,
|
||||||
|
learning_rate=0.0002,
|
||||||
|
warmup_ratio=0.03,
|
||||||
|
save_steps=50,
|
||||||
|
logging_steps=10,
|
||||||
|
fp16=True,
|
||||||
|
optim="adamw_8bit",
|
||||||
|
report_to="none",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
print("Starting training...")
|
||||||
|
trainer.train()
|
||||||
|
print("Training complete!")
|
||||||
|
|
||||||
|
# Save
|
||||||
|
model.save_pretrained("C:\she\autarch\data\training\output/lora_adapter")
|
||||||
|
tokenizer.save_pretrained("C:\she\autarch\data\training\output/lora_adapter")
|
||||||
|
print(f"LoRA adapter saved to C:\she\autarch\data\training\output/lora_adapter")
|
||||||
14
data/training/training.log
Normal file
14
data/training/training.log
Normal file
@ -0,0 +1,14 @@
|
|||||||
|
C:\she\autarch\data\training\train_lora.py:50: SyntaxWarning: invalid escape sequence '\s'
|
||||||
|
with open("C:\she\autarch\data\training\autarch_dataset_20260302_202634.jsonl", "r") as f:
|
||||||
|
C:\she\autarch\data\training\train_lora.py:76: SyntaxWarning: invalid escape sequence '\s'
|
||||||
|
output_dir="C:\she\autarch\data\training\output",
|
||||||
|
C:\she\autarch\data\training\train_lora.py:95: SyntaxWarning: invalid escape sequence '\s'
|
||||||
|
model.save_pretrained("C:\she\autarch\data\training\output/lora_adapter")
|
||||||
|
C:\she\autarch\data\training\train_lora.py:96: SyntaxWarning: invalid escape sequence '\s'
|
||||||
|
tokenizer.save_pretrained("C:\she\autarch\data\training\output/lora_adapter")
|
||||||
|
C:\she\autarch\data\training\train_lora.py:97: SyntaxWarning: invalid escape sequence '\s'
|
||||||
|
print(f"LoRA adapter saved to C:\she\autarch\data\training\output/lora_adapter")
|
||||||
|
Traceback (most recent call last):
|
||||||
|
File "C:\she\autarch\data\training\train_lora.py", line 5, in <module>
|
||||||
|
from datasets import Dataset
|
||||||
|
ModuleNotFoundError: No module named 'datasets'
|
||||||
BIN
download.png
BIN
download.png
Binary file not shown.
|
Before Width: | Height: | Size: 985 KiB |
27506
get-pip.py
27506
get-pip.py
File diff suppressed because it is too large
Load Diff
1162
modules/defender_monitor.py
Normal file
1162
modules/defender_monitor.py
Normal file
File diff suppressed because it is too large
Load Diff
372
modules/defender_windows.py
Normal file
372
modules/defender_windows.py
Normal file
@ -0,0 +1,372 @@
|
|||||||
|
"""
|
||||||
|
AUTARCH Windows Defender Module
|
||||||
|
Windows-native security posture assessment
|
||||||
|
|
||||||
|
Checks Windows system configuration for security best practices.
|
||||||
|
"""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
import subprocess
|
||||||
|
import re
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
from datetime import datetime
|
||||||
|
|
||||||
|
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||||
|
|
||||||
|
# Module metadata
|
||||||
|
DESCRIPTION = "Windows system hardening & security checks"
|
||||||
|
AUTHOR = "darkHal"
|
||||||
|
VERSION = "1.0"
|
||||||
|
CATEGORY = "defense"
|
||||||
|
|
||||||
|
|
||||||
|
class WindowsDefender:
|
||||||
|
"""Windows security checker."""
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
self.results = []
|
||||||
|
|
||||||
|
def check(self, name: str, passed: bool, details: str = ""):
|
||||||
|
"""Record a check result."""
|
||||||
|
self.results.append({"name": name, "passed": passed, "details": details})
|
||||||
|
|
||||||
|
def run_cmd(self, cmd: str, timeout=15) -> tuple:
|
||||||
|
"""Run command and return (success, output)."""
|
||||||
|
try:
|
||||||
|
result = subprocess.run(cmd, shell=True, capture_output=True,
|
||||||
|
text=True, timeout=timeout)
|
||||||
|
return result.returncode == 0, result.stdout.strip()
|
||||||
|
except Exception:
|
||||||
|
return False, ""
|
||||||
|
|
||||||
|
def run_ps(self, ps_command: str, timeout=15) -> tuple:
|
||||||
|
"""Run a PowerShell command and return (success, output)."""
|
||||||
|
cmd = f'powershell -NoProfile -ExecutionPolicy Bypass -Command "{ps_command}"'
|
||||||
|
return self.run_cmd(cmd, timeout=timeout)
|
||||||
|
|
||||||
|
# ==================== SECURITY CHECKS ====================
|
||||||
|
|
||||||
|
def check_firewall(self):
|
||||||
|
"""Check Windows Firewall status for all profiles."""
|
||||||
|
success, output = self.run_cmd("netsh advfirewall show allprofiles state")
|
||||||
|
if success:
|
||||||
|
profiles_on = output.lower().count("on")
|
||||||
|
profiles_off = output.lower().count("off")
|
||||||
|
if profiles_off > 0:
|
||||||
|
self.check("Windows Firewall", False,
|
||||||
|
f"{profiles_off} profile(s) disabled")
|
||||||
|
else:
|
||||||
|
self.check("Windows Firewall", True,
|
||||||
|
f"All {profiles_on} profiles enabled")
|
||||||
|
else:
|
||||||
|
self.check("Windows Firewall", False, "Could not query firewall state")
|
||||||
|
|
||||||
|
def check_ssh_config(self):
|
||||||
|
"""Check Windows OpenSSH configuration."""
|
||||||
|
success, output = self.run_ps(
|
||||||
|
"Get-WindowsCapability -Online | Where-Object Name -like 'OpenSSH.Server*' "
|
||||||
|
"| Select-Object -ExpandProperty State"
|
||||||
|
)
|
||||||
|
if not success or "Installed" not in output:
|
||||||
|
self.check("SSH Config", True, "OpenSSH Server not installed (good)")
|
||||||
|
return
|
||||||
|
|
||||||
|
sshd_config = Path(os.environ.get('ProgramData', 'C:\\ProgramData')) / 'ssh' / 'sshd_config'
|
||||||
|
if not sshd_config.exists():
|
||||||
|
self.check("SSH Config", False, "OpenSSH installed but sshd_config not found")
|
||||||
|
return
|
||||||
|
|
||||||
|
content = sshd_config.read_text(errors='ignore')
|
||||||
|
|
||||||
|
if "PermitRootLogin no" in content or "PermitRootLogin prohibit-password" in content:
|
||||||
|
self.check("SSH Root Login Disabled", True)
|
||||||
|
else:
|
||||||
|
self.check("SSH Root Login Disabled", False, "Root login may be enabled")
|
||||||
|
|
||||||
|
if "PasswordAuthentication no" in content:
|
||||||
|
self.check("SSH Password Auth Disabled", True)
|
||||||
|
else:
|
||||||
|
self.check("SSH Password Auth Disabled", False,
|
||||||
|
"Consider using key-based auth only")
|
||||||
|
|
||||||
|
def check_open_ports(self):
|
||||||
|
"""Check for high-risk listening ports on Windows."""
|
||||||
|
success, output = self.run_ps(
|
||||||
|
"Get-NetTCPConnection -State Listen -ErrorAction SilentlyContinue "
|
||||||
|
"| Select-Object LocalPort, OwningProcess | Format-Table -AutoSize"
|
||||||
|
)
|
||||||
|
if not success:
|
||||||
|
success, output = self.run_cmd("netstat -ano | findstr LISTENING")
|
||||||
|
|
||||||
|
if success:
|
||||||
|
high_risk = []
|
||||||
|
if ':23 ' in output or '\t23\t' in output:
|
||||||
|
high_risk.append("23 (Telnet)")
|
||||||
|
if ':21 ' in output or '\t21\t' in output:
|
||||||
|
high_risk.append("21 (FTP)")
|
||||||
|
if ':3389 ' in output or '\t3389\t' in output:
|
||||||
|
high_risk.append("3389 (RDP)")
|
||||||
|
if ':445 ' in output or '\t445\t' in output:
|
||||||
|
high_risk.append("445 (SMB)")
|
||||||
|
if ':135 ' in output or '\t135\t' in output:
|
||||||
|
high_risk.append("135 (RPC)")
|
||||||
|
|
||||||
|
lines = [l for l in output.split('\n') if l.strip()]
|
||||||
|
if high_risk:
|
||||||
|
self.check("High-Risk Ports", False,
|
||||||
|
f"Open: {', '.join(high_risk)}")
|
||||||
|
else:
|
||||||
|
self.check("High-Risk Ports", True,
|
||||||
|
f"{len(lines)} services listening, no high-risk ports")
|
||||||
|
else:
|
||||||
|
self.check("High-Risk Ports", True, "Could not enumerate ports")
|
||||||
|
|
||||||
|
def check_updates(self):
|
||||||
|
"""Check Windows update status."""
|
||||||
|
success, output = self.run_ps(
|
||||||
|
"Get-HotFix | Sort-Object InstalledOn -Descending "
|
||||||
|
"| Select-Object -First 1 -ExpandProperty InstalledOn"
|
||||||
|
)
|
||||||
|
if success and output.strip():
|
||||||
|
self.check("System Updates", True,
|
||||||
|
f"Last update installed: {output.strip()}")
|
||||||
|
else:
|
||||||
|
success, output = self.run_ps("(Get-HotFix).Count")
|
||||||
|
if success and output.strip():
|
||||||
|
self.check("System Updates", True,
|
||||||
|
f"{output.strip()} hotfixes installed")
|
||||||
|
else:
|
||||||
|
self.check("System Updates", False, "Could not query update status")
|
||||||
|
|
||||||
|
def check_users(self):
|
||||||
|
"""Check Windows user security."""
|
||||||
|
# Admin accounts
|
||||||
|
success, output = self.run_ps(
|
||||||
|
"Get-LocalGroupMember -Group 'Administrators' -ErrorAction SilentlyContinue "
|
||||||
|
"| Select-Object -ExpandProperty Name"
|
||||||
|
)
|
||||||
|
if success:
|
||||||
|
admins = [u.strip() for u in output.split('\n') if u.strip()]
|
||||||
|
self.check("Admin Accounts", len(admins) <= 2,
|
||||||
|
f"Admin users: {', '.join(admins)}")
|
||||||
|
|
||||||
|
# Enabled accounts with no password required
|
||||||
|
success, output = self.run_ps(
|
||||||
|
"Get-LocalUser | Where-Object {$_.Enabled -eq $true -and $_.PasswordRequired -eq $false} "
|
||||||
|
"| Select-Object -ExpandProperty Name"
|
||||||
|
)
|
||||||
|
if success:
|
||||||
|
no_pw = [u.strip() for u in output.split('\n') if u.strip()]
|
||||||
|
self.check("Password Required", len(no_pw) == 0,
|
||||||
|
f"No password required: {', '.join(no_pw)}" if no_pw else "All accounts require passwords")
|
||||||
|
|
||||||
|
# Guest account
|
||||||
|
success, output = self.run_ps("(Get-LocalUser -Name 'Guest' -ErrorAction SilentlyContinue).Enabled")
|
||||||
|
if success:
|
||||||
|
guest_enabled = output.strip().lower() == 'true'
|
||||||
|
self.check("Guest Account Disabled", not guest_enabled,
|
||||||
|
"Guest account is enabled" if guest_enabled else "Guest account disabled")
|
||||||
|
|
||||||
|
def check_permissions(self):
|
||||||
|
"""Check critical Windows file/folder permissions."""
|
||||||
|
critical_paths = [
|
||||||
|
(os.environ.get('SystemRoot', 'C:\\Windows') + '\\System32\\config', "SAM Registry Hive"),
|
||||||
|
(os.environ.get('ProgramData', 'C:\\ProgramData') + '\\ssh', "SSH Config Dir"),
|
||||||
|
]
|
||||||
|
for filepath, label in critical_paths:
|
||||||
|
if os.path.exists(filepath):
|
||||||
|
success, output = self.run_cmd(f'icacls "{filepath}"')
|
||||||
|
if success:
|
||||||
|
has_everyone_full = 'Everyone:(F)' in output or 'Everyone:(OI)(CI)(F)' in output
|
||||||
|
self.check(f"Permissions: {label}", not has_everyone_full,
|
||||||
|
f"Everyone has Full Control on {filepath}" if has_everyone_full else "Restricted")
|
||||||
|
|
||||||
|
def check_services(self):
|
||||||
|
"""Check for dangerous or unnecessary Windows services."""
|
||||||
|
dangerous = {
|
||||||
|
"RemoteRegistry": "Remote Registry",
|
||||||
|
"TlntSvr": "Telnet Server",
|
||||||
|
"SNMP": "SNMP Service",
|
||||||
|
"W3SVC": "IIS Web Server",
|
||||||
|
"FTPSVC": "FTP Server",
|
||||||
|
"SharedAccess": "Internet Connection Sharing",
|
||||||
|
}
|
||||||
|
running = []
|
||||||
|
for svc_name, label in dangerous.items():
|
||||||
|
success, output = self.run_ps(
|
||||||
|
f"(Get-Service -Name '{svc_name}' -ErrorAction SilentlyContinue).Status"
|
||||||
|
)
|
||||||
|
if success and 'Running' in output:
|
||||||
|
running.append(label)
|
||||||
|
|
||||||
|
self.check("Dangerous Services", len(running) == 0,
|
||||||
|
f"Running: {', '.join(running)}" if running else "No dangerous services running")
|
||||||
|
|
||||||
|
def check_defender(self):
|
||||||
|
"""Check Windows Defender antivirus status."""
|
||||||
|
success, output = self.run_ps(
|
||||||
|
"Get-MpComputerStatus -ErrorAction SilentlyContinue "
|
||||||
|
"| Select-Object AntivirusEnabled, RealTimeProtectionEnabled, "
|
||||||
|
"AntivirusSignatureLastUpdated | Format-List"
|
||||||
|
)
|
||||||
|
if success:
|
||||||
|
av_on = re.search(r'AntivirusEnabled\s*:\s*True', output)
|
||||||
|
rt_on = re.search(r'RealTimeProtectionEnabled\s*:\s*True', output)
|
||||||
|
|
||||||
|
if av_on and rt_on:
|
||||||
|
sig_match = re.search(r'AntivirusSignatureLastUpdated\s*:\s*(.+)', output)
|
||||||
|
sig_date = sig_match.group(1).strip() if sig_match else "Unknown"
|
||||||
|
self.check("Windows Defender", True,
|
||||||
|
f"AV enabled, real-time protection on. Signatures: {sig_date}")
|
||||||
|
elif av_on:
|
||||||
|
self.check("Windows Defender", False,
|
||||||
|
"AV enabled but real-time protection is OFF")
|
||||||
|
else:
|
||||||
|
self.check("Windows Defender", False, "Windows Defender is disabled")
|
||||||
|
else:
|
||||||
|
self.check("Windows Defender", False, "Could not query Defender status")
|
||||||
|
|
||||||
|
def check_uac(self):
|
||||||
|
"""Check UAC (User Account Control) status."""
|
||||||
|
success, output = self.run_ps(
|
||||||
|
"(Get-ItemProperty -Path 'HKLM:\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Policies\\System' "
|
||||||
|
"-Name EnableLUA -ErrorAction SilentlyContinue).EnableLUA"
|
||||||
|
)
|
||||||
|
if success:
|
||||||
|
enabled = output.strip() == '1'
|
||||||
|
self.check("UAC Enabled", enabled,
|
||||||
|
"UAC is enabled" if enabled else "UAC is DISABLED — critical security risk")
|
||||||
|
|
||||||
|
success, output = self.run_ps(
|
||||||
|
"(Get-ItemProperty -Path 'HKLM:\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Policies\\System' "
|
||||||
|
"-Name ConsentPromptBehaviorAdmin -ErrorAction SilentlyContinue).ConsentPromptBehaviorAdmin"
|
||||||
|
)
|
||||||
|
if success and output.strip().isdigit():
|
||||||
|
level = int(output.strip())
|
||||||
|
level_names = {
|
||||||
|
0: "Never notify (DANGEROUS)",
|
||||||
|
1: "Prompt on secure desktop (no dimming)",
|
||||||
|
2: "Prompt on secure desktop",
|
||||||
|
3: "Prompt for credentials",
|
||||||
|
4: "Prompt for consent",
|
||||||
|
5: "Prompt for consent (default)"
|
||||||
|
}
|
||||||
|
desc = level_names.get(level, f"Unknown level: {level}")
|
||||||
|
self.check("UAC Prompt Level", level >= 2, desc)
|
||||||
|
|
||||||
|
# ==================== FIREWALL MANAGEMENT ====================
|
||||||
|
|
||||||
|
def get_firewall_rules(self):
|
||||||
|
"""Get all Windows Firewall inbound rules."""
|
||||||
|
success, output = self.run_cmd(
|
||||||
|
"netsh advfirewall firewall show rule name=all dir=in"
|
||||||
|
)
|
||||||
|
return success, output
|
||||||
|
|
||||||
|
def block_ip(self, ip):
|
||||||
|
"""Block an IP via Windows Firewall."""
|
||||||
|
rule_name = f"AUTARCH_Block_{ip}"
|
||||||
|
success, output = self.run_cmd(
|
||||||
|
f'netsh advfirewall firewall add rule name="{rule_name}" '
|
||||||
|
f'dir=in action=block remoteip={ip}'
|
||||||
|
)
|
||||||
|
return success, f"Blocked {ip}" if success else f"Failed to block {ip} (need admin privileges)"
|
||||||
|
|
||||||
|
def unblock_ip(self, ip):
|
||||||
|
"""Unblock an IP via Windows Firewall."""
|
||||||
|
rule_name = f"AUTARCH_Block_{ip}"
|
||||||
|
success, output = self.run_cmd(
|
||||||
|
f'netsh advfirewall firewall delete rule name="{rule_name}"'
|
||||||
|
)
|
||||||
|
return success, f"Unblocked {ip}" if success else f"Failed to unblock {ip}"
|
||||||
|
|
||||||
|
# ==================== EVENT LOG ANALYSIS ====================
|
||||||
|
|
||||||
|
def analyze_event_logs(self):
|
||||||
|
"""Analyze Windows Security and System event logs."""
|
||||||
|
# Failed logins (Event ID 4625)
|
||||||
|
success, output = self.run_ps(
|
||||||
|
"Get-WinEvent -FilterHashtable @{LogName='Security'; Id=4625} "
|
||||||
|
"-MaxEvents 500 -ErrorAction SilentlyContinue | "
|
||||||
|
"Select-Object TimeCreated, @{N='IP';E={$_.Properties[19].Value}}, "
|
||||||
|
"@{N='User';E={$_.Properties[5].Value}} | "
|
||||||
|
"Group-Object IP | Sort-Object Count -Descending | "
|
||||||
|
"Select-Object Count, Name, @{N='Users';E={($_.Group.User | Select-Object -Unique) -join ','}} | "
|
||||||
|
"ConvertTo-Json"
|
||||||
|
)
|
||||||
|
auth_results = []
|
||||||
|
if success and output.strip():
|
||||||
|
try:
|
||||||
|
data = json.loads(output)
|
||||||
|
if isinstance(data, dict):
|
||||||
|
data = [data]
|
||||||
|
for entry in data:
|
||||||
|
auth_results.append({
|
||||||
|
'ip': entry.get('Name', 'Unknown'),
|
||||||
|
'count': entry.get('Count', 0),
|
||||||
|
'usernames': (entry.get('Users', '') or '').split(','),
|
||||||
|
})
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
# System warnings/errors
|
||||||
|
success, output = self.run_ps(
|
||||||
|
"Get-WinEvent -FilterHashtable @{LogName='System'; Level=1,2,3} "
|
||||||
|
"-MaxEvents 50 -ErrorAction SilentlyContinue | "
|
||||||
|
"Select-Object TimeCreated, Id, LevelDisplayName, Message | "
|
||||||
|
"ConvertTo-Json"
|
||||||
|
)
|
||||||
|
system_results = []
|
||||||
|
if success and output.strip():
|
||||||
|
try:
|
||||||
|
data = json.loads(output)
|
||||||
|
if isinstance(data, dict):
|
||||||
|
data = [data]
|
||||||
|
for entry in data[:20]:
|
||||||
|
system_results.append({
|
||||||
|
'type': entry.get('LevelDisplayName', 'Warning'),
|
||||||
|
'id': entry.get('Id', 0),
|
||||||
|
'time': str(entry.get('TimeCreated', '')),
|
||||||
|
'detail': (entry.get('Message', '') or '')[:200],
|
||||||
|
'severity': 'HIGH' if entry.get('LevelDisplayName') in ('Critical', 'Error') else 'MEDIUM',
|
||||||
|
})
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
return auth_results, system_results
|
||||||
|
|
||||||
|
|
||||||
|
# ==================== CLI MENU ====================
|
||||||
|
|
||||||
|
def run():
|
||||||
|
"""CLI entry point."""
|
||||||
|
from core.banner import Colors, clear_screen, display_banner
|
||||||
|
clear_screen()
|
||||||
|
display_banner()
|
||||||
|
print(f"\n{Colors.BOLD}{Colors.BLUE}Windows System Defense{Colors.RESET}\n")
|
||||||
|
|
||||||
|
d = WindowsDefender()
|
||||||
|
print(f"{Colors.CYAN}Running Windows security audit...{Colors.RESET}\n")
|
||||||
|
|
||||||
|
d.check_firewall()
|
||||||
|
d.check_ssh_config()
|
||||||
|
d.check_open_ports()
|
||||||
|
d.check_updates()
|
||||||
|
d.check_users()
|
||||||
|
d.check_permissions()
|
||||||
|
d.check_services()
|
||||||
|
d.check_defender()
|
||||||
|
d.check_uac()
|
||||||
|
|
||||||
|
passed = sum(1 for r in d.results if r['passed'])
|
||||||
|
total = len(d.results)
|
||||||
|
score = int((passed / total) * 100) if total > 0 else 0
|
||||||
|
|
||||||
|
print(f"\n{'=' * 50}")
|
||||||
|
color = Colors.GREEN if score >= 80 else Colors.YELLOW if score >= 50 else Colors.RED
|
||||||
|
print(f"{color}Security Score: {score}% ({passed}/{total} checks passed){Colors.RESET}")
|
||||||
|
print(f"{'=' * 50}\n")
|
||||||
|
|
||||||
|
input("Press Enter to continue...")
|
||||||
1447
modules/llm_trainer.py
Normal file
1447
modules/llm_trainer.py
Normal file
File diff suppressed because it is too large
Load Diff
12
setup_msi.py
12
setup_msi.py
@ -50,8 +50,16 @@ build_exe_options = {
|
|||||||
'web.routes.msf', 'web.routes.chat',
|
'web.routes.msf', 'web.routes.chat',
|
||||||
'web.routes.targets', 'web.routes.encmodules',
|
'web.routes.targets', 'web.routes.encmodules',
|
||||||
],
|
],
|
||||||
'excludes': ['torch', 'transformers', 'llama_cpp', 'anthropic',
|
'excludes': ['torch', 'transformers', 'llama_cpp', 'llama_cpp_python', 'anthropic',
|
||||||
'tkinter', 'matplotlib', 'numpy'],
|
'tkinter', 'matplotlib', 'numpy',
|
||||||
|
'bitsandbytes',
|
||||||
|
'huggingface_hub', 'safetensors', 'tokenizers',
|
||||||
|
'mcp', 'uvicorn', 'starlette', 'anyio', 'httpx', 'httpx_sse',
|
||||||
|
'httpcore', 'h11', 'h2', 'hpack', 'hyperframe',
|
||||||
|
'pydantic', 'pydantic_core', 'pydantic_settings',
|
||||||
|
'scipy', 'pandas', 'tensorflow', 'keras',
|
||||||
|
'IPython', 'notebook', 'jupyterlab',
|
||||||
|
'fsspec', 'rich', 'typer'],
|
||||||
'include_files': include_files,
|
'include_files': include_files,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@ -59,6 +59,7 @@ def create_app():
|
|||||||
from web.routes.chat import chat_bp
|
from web.routes.chat import chat_bp
|
||||||
from web.routes.targets import targets_bp
|
from web.routes.targets import targets_bp
|
||||||
from web.routes.encmodules import encmodules_bp
|
from web.routes.encmodules import encmodules_bp
|
||||||
|
from web.routes.llm_trainer import llm_trainer_bp
|
||||||
|
|
||||||
app.register_blueprint(auth_bp)
|
app.register_blueprint(auth_bp)
|
||||||
app.register_blueprint(dashboard_bp)
|
app.register_blueprint(dashboard_bp)
|
||||||
@ -82,6 +83,7 @@ def create_app():
|
|||||||
app.register_blueprint(chat_bp)
|
app.register_blueprint(chat_bp)
|
||||||
app.register_blueprint(targets_bp)
|
app.register_blueprint(targets_bp)
|
||||||
app.register_blueprint(encmodules_bp)
|
app.register_blueprint(encmodules_bp)
|
||||||
|
app.register_blueprint(llm_trainer_bp)
|
||||||
|
|
||||||
# Start network discovery advertising (mDNS + Bluetooth)
|
# Start network discovery advertising (mDNS + Bluetooth)
|
||||||
try:
|
try:
|
||||||
|
|||||||
@ -1,35 +1,127 @@
|
|||||||
"""Chat and Agent API routes — LLM chat SSE stream + autonomous agent run/stream/stop."""
|
"""Chat and Agent API routes — Hal chat with Agent system for module creation."""
|
||||||
|
|
||||||
import json
|
import json
|
||||||
import threading
|
import threading
|
||||||
import time
|
import time
|
||||||
import uuid
|
import uuid
|
||||||
|
from pathlib import Path
|
||||||
from flask import Blueprint, request, jsonify, Response
|
from flask import Blueprint, request, jsonify, Response
|
||||||
from web.auth import login_required
|
from web.auth import login_required
|
||||||
|
|
||||||
chat_bp = Blueprint('chat', __name__, url_prefix='/api')
|
chat_bp = Blueprint('chat', __name__, url_prefix='/api')
|
||||||
|
|
||||||
_agent_runs: dict = {} # run_id -> {'steps': [], 'done': bool, 'stop': threading.Event}
|
_agent_runs: dict = {} # run_id -> {'steps': [], 'done': bool, 'stop': threading.Event}
|
||||||
|
_system_prompt = None
|
||||||
|
|
||||||
|
|
||||||
|
def _get_system_prompt():
|
||||||
|
"""Load the Hal system prompt from data/hal_system_prompt.txt."""
|
||||||
|
global _system_prompt
|
||||||
|
if _system_prompt is None:
|
||||||
|
prompt_path = Path(__file__).parent.parent.parent / 'data' / 'hal_system_prompt.txt'
|
||||||
|
if prompt_path.exists():
|
||||||
|
_system_prompt = prompt_path.read_text(encoding='utf-8')
|
||||||
|
else:
|
||||||
|
_system_prompt = (
|
||||||
|
"You are Hal, the AI agent for AUTARCH. You can create new modules, "
|
||||||
|
"run shell commands, read and write files. When asked to create a module, "
|
||||||
|
"use the create_module tool."
|
||||||
|
)
|
||||||
|
return _system_prompt
|
||||||
|
|
||||||
|
|
||||||
|
def _ensure_model_loaded():
|
||||||
|
"""Load the LLM model if not already loaded. Returns (llm, error)."""
|
||||||
|
from core.llm import get_llm, LLMError
|
||||||
|
llm = get_llm()
|
||||||
|
if not llm.is_loaded:
|
||||||
|
try:
|
||||||
|
llm.load_model(verbose=False)
|
||||||
|
except LLMError as e:
|
||||||
|
return None, str(e)
|
||||||
|
return llm, None
|
||||||
|
|
||||||
|
|
||||||
@chat_bp.route('/chat', methods=['POST'])
|
@chat_bp.route('/chat', methods=['POST'])
|
||||||
@login_required
|
@login_required
|
||||||
def chat():
|
def chat():
|
||||||
"""Stream LLM response token-by-token via SSE."""
|
"""Handle chat messages — uses Agent system for tool-using tasks,
|
||||||
|
direct chat for simple questions. Streams response via SSE."""
|
||||||
data = request.get_json(silent=True) or {}
|
data = request.get_json(silent=True) or {}
|
||||||
message = data.get('message', '').strip()
|
message = data.get('message', '').strip()
|
||||||
if not message:
|
if not message:
|
||||||
return jsonify({'error': 'No message provided'})
|
return jsonify({'error': 'No message provided'})
|
||||||
|
|
||||||
def generate():
|
# Always use agent mode so Hal can use tools including create_module
|
||||||
|
run_id = str(uuid.uuid4())
|
||||||
|
stop_event = threading.Event()
|
||||||
|
steps = []
|
||||||
|
_agent_runs[run_id] = {'steps': steps, 'done': False, 'stop': stop_event}
|
||||||
|
|
||||||
|
def worker():
|
||||||
try:
|
try:
|
||||||
from core.llm import get_llm
|
from core.agent import Agent
|
||||||
|
from core.tools import get_tool_registry
|
||||||
|
from core.llm import get_llm, LLMError
|
||||||
|
|
||||||
llm = get_llm()
|
llm = get_llm()
|
||||||
for token in llm.chat(message, stream=True):
|
if not llm.is_loaded:
|
||||||
yield f"data: {json.dumps({'token': token})}\n\n"
|
steps.append({'type': 'status', 'content': 'Loading model...'})
|
||||||
yield f"data: {json.dumps({'done': True})}\n\n"
|
try:
|
||||||
|
llm.load_model(verbose=False)
|
||||||
|
except LLMError as e:
|
||||||
|
steps.append({'type': 'error', 'content': f'Failed to load model: {e}'})
|
||||||
|
return
|
||||||
|
|
||||||
|
tools = get_tool_registry()
|
||||||
|
agent = Agent(llm=llm, tools=tools, max_steps=20, verbose=False)
|
||||||
|
|
||||||
|
# Inject system prompt into agent
|
||||||
|
system_prompt = _get_system_prompt()
|
||||||
|
agent.SYSTEM_PROMPT = system_prompt + "\n\n{tools_description}"
|
||||||
|
|
||||||
|
def on_step(step):
|
||||||
|
if step.thought:
|
||||||
|
steps.append({'type': 'thought', 'content': step.thought})
|
||||||
|
if step.tool_name and step.tool_name not in ('task_complete', 'ask_user'):
|
||||||
|
steps.append({'type': 'action', 'content': f"{step.tool_name}({json.dumps(step.tool_args or {})})"})
|
||||||
|
if step.tool_result:
|
||||||
|
# Truncate long results for display
|
||||||
|
result = step.tool_result
|
||||||
|
if len(result) > 800:
|
||||||
|
result = result[:800] + '...'
|
||||||
|
steps.append({'type': 'result', 'content': result})
|
||||||
|
|
||||||
|
result = agent.run(message, step_callback=on_step)
|
||||||
|
|
||||||
|
if result.success:
|
||||||
|
steps.append({'type': 'answer', 'content': result.summary})
|
||||||
|
else:
|
||||||
|
steps.append({'type': 'error', 'content': result.error or result.summary})
|
||||||
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
yield f"data: {json.dumps({'error': str(e)})}\n\n"
|
steps.append({'type': 'error', 'content': str(e)})
|
||||||
|
finally:
|
||||||
|
_agent_runs[run_id]['done'] = True
|
||||||
|
|
||||||
|
threading.Thread(target=worker, daemon=True).start()
|
||||||
|
|
||||||
|
# Stream the agent steps as SSE
|
||||||
|
def generate():
|
||||||
|
run = _agent_runs.get(run_id)
|
||||||
|
if not run:
|
||||||
|
yield f"data: {json.dumps({'error': 'Run not found'})}\n\n"
|
||||||
|
return
|
||||||
|
sent = 0
|
||||||
|
while True:
|
||||||
|
current_steps = run['steps']
|
||||||
|
while sent < len(current_steps):
|
||||||
|
yield f"data: {json.dumps(current_steps[sent])}\n\n"
|
||||||
|
sent += 1
|
||||||
|
if run['done']:
|
||||||
|
yield f"data: {json.dumps({'done': True})}\n\n"
|
||||||
|
return
|
||||||
|
time.sleep(0.15)
|
||||||
|
|
||||||
return Response(generate(), mimetype='text/event-stream',
|
return Response(generate(), mimetype='text/event-stream',
|
||||||
headers={'Cache-Control': 'no-cache', 'X-Accel-Buffering': 'no'})
|
headers={'Cache-Control': 'no-cache', 'X-Accel-Buffering': 'no'})
|
||||||
@ -42,7 +134,9 @@ def chat_reset():
|
|||||||
try:
|
try:
|
||||||
from core.llm import get_llm
|
from core.llm import get_llm
|
||||||
llm = get_llm()
|
llm = get_llm()
|
||||||
if hasattr(llm, 'reset'):
|
if hasattr(llm, 'clear_history'):
|
||||||
|
llm.clear_history()
|
||||||
|
elif hasattr(llm, 'reset'):
|
||||||
llm.reset()
|
llm.reset()
|
||||||
elif hasattr(llm, 'conversation_history'):
|
elif hasattr(llm, 'conversation_history'):
|
||||||
llm.conversation_history = []
|
llm.conversation_history = []
|
||||||
@ -51,6 +145,21 @@ def chat_reset():
|
|||||||
return jsonify({'ok': True})
|
return jsonify({'ok': True})
|
||||||
|
|
||||||
|
|
||||||
|
@chat_bp.route('/chat/status')
|
||||||
|
@login_required
|
||||||
|
def chat_status():
|
||||||
|
"""Get LLM model status."""
|
||||||
|
try:
|
||||||
|
from core.llm import get_llm
|
||||||
|
llm = get_llm()
|
||||||
|
return jsonify({
|
||||||
|
'loaded': llm.is_loaded,
|
||||||
|
'model': llm.model_name if llm.is_loaded else None,
|
||||||
|
})
|
||||||
|
except Exception as e:
|
||||||
|
return jsonify({'loaded': False, 'error': str(e)})
|
||||||
|
|
||||||
|
|
||||||
@chat_bp.route('/agent/run', methods=['POST'])
|
@chat_bp.route('/agent/run', methods=['POST'])
|
||||||
@login_required
|
@login_required
|
||||||
def agent_run():
|
def agent_run():
|
||||||
@ -69,14 +178,29 @@ def agent_run():
|
|||||||
try:
|
try:
|
||||||
from core.agent import Agent
|
from core.agent import Agent
|
||||||
from core.tools import get_tool_registry
|
from core.tools import get_tool_registry
|
||||||
agent = Agent(tool_registry=get_tool_registry(), verbose=False)
|
from core.llm import get_llm, LLMError
|
||||||
|
|
||||||
|
llm = get_llm()
|
||||||
|
if not llm.is_loaded:
|
||||||
|
try:
|
||||||
|
llm.load_model(verbose=False)
|
||||||
|
except LLMError as e:
|
||||||
|
steps.append({'type': 'error', 'content': f'Failed to load model: {e}'})
|
||||||
|
return
|
||||||
|
|
||||||
|
tools = get_tool_registry()
|
||||||
|
agent = Agent(llm=llm, tools=tools, verbose=False)
|
||||||
|
|
||||||
|
# Inject system prompt
|
||||||
|
system_prompt = _get_system_prompt()
|
||||||
|
agent.SYSTEM_PROMPT = system_prompt + "\n\n{tools_description}"
|
||||||
|
|
||||||
def on_step(step):
|
def on_step(step):
|
||||||
steps.append({'type': 'thought', 'content': step.thought})
|
steps.append({'type': 'thought', 'content': step.thought})
|
||||||
if step.tool_name and step.tool_name not in ('task_complete', 'ask_user'):
|
if step.tool_name and step.tool_name not in ('task_complete', 'ask_user'):
|
||||||
steps.append({'type': 'action', 'content': f"{step.tool_name}({json.dumps(step.tool_args or {})})"})
|
steps.append({'type': 'action', 'content': f"{step.tool_name}({json.dumps(step.tool_args or {})})"})
|
||||||
if step.tool_result:
|
if step.tool_result:
|
||||||
steps.append({'type': 'result', 'content': step.tool_result[:600]})
|
steps.append({'type': 'result', 'content': step.tool_result[:800]})
|
||||||
|
|
||||||
agent.run(task, step_callback=on_step)
|
agent.run(task, step_callback=on_step)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
@ -99,9 +223,9 @@ def agent_stream(run_id):
|
|||||||
return
|
return
|
||||||
sent = 0
|
sent = 0
|
||||||
while True:
|
while True:
|
||||||
steps = run['steps']
|
current_steps = run['steps']
|
||||||
while sent < len(steps):
|
while sent < len(current_steps):
|
||||||
yield f"data: {json.dumps(steps[sent])}\n\n"
|
yield f"data: {json.dumps(current_steps[sent])}\n\n"
|
||||||
sent += 1
|
sent += 1
|
||||||
if run['done']:
|
if run['done']:
|
||||||
yield f"data: {json.dumps({'done': True})}\n\n"
|
yield f"data: {json.dumps({'done': True})}\n\n"
|
||||||
|
|||||||
@ -1,8 +1,11 @@
|
|||||||
"""Defense category route - security audit, firewall, and log analysis endpoints."""
|
"""Defense category routes — landing page, Linux defense, Windows defense, Threat Monitor."""
|
||||||
|
|
||||||
import re
|
import re
|
||||||
import subprocess
|
import subprocess
|
||||||
from flask import Blueprint, render_template, request, jsonify
|
import platform
|
||||||
|
import socket
|
||||||
|
import json
|
||||||
|
from flask import Blueprint, render_template, request, jsonify, Response, stream_with_context
|
||||||
from web.auth import login_required
|
from web.auth import login_required
|
||||||
|
|
||||||
defense_bp = Blueprint('defense', __name__, url_prefix='/defense')
|
defense_bp = Blueprint('defense', __name__, url_prefix='/defense')
|
||||||
@ -16,6 +19,8 @@ def _run_cmd(cmd, timeout=10):
|
|||||||
return False, ""
|
return False, ""
|
||||||
|
|
||||||
|
|
||||||
|
# ==================== LANDING PAGE ====================
|
||||||
|
|
||||||
@defense_bp.route('/')
|
@defense_bp.route('/')
|
||||||
@login_required
|
@login_required
|
||||||
def index():
|
def index():
|
||||||
@ -23,13 +28,37 @@ def index():
|
|||||||
menu = MainMenu()
|
menu = MainMenu()
|
||||||
menu.load_modules()
|
menu.load_modules()
|
||||||
modules = {k: v for k, v in menu.modules.items() if v.category == 'defense'}
|
modules = {k: v for k, v in menu.modules.items() if v.category == 'defense'}
|
||||||
return render_template('defense.html', modules=modules)
|
|
||||||
|
# Gather system info for the landing page
|
||||||
|
sys_info = {
|
||||||
|
'platform': platform.system(),
|
||||||
|
'hostname': socket.gethostname(),
|
||||||
|
'os_version': platform.platform(),
|
||||||
|
}
|
||||||
|
try:
|
||||||
|
sys_info['ip'] = socket.gethostbyname(socket.gethostname())
|
||||||
|
except Exception:
|
||||||
|
sys_info['ip'] = '127.0.0.1'
|
||||||
|
|
||||||
|
return render_template('defense.html', modules=modules, sys_info=sys_info)
|
||||||
|
|
||||||
|
|
||||||
@defense_bp.route('/audit', methods=['POST'])
|
# ==================== LINUX DEFENSE ====================
|
||||||
|
|
||||||
|
@defense_bp.route('/linux')
|
||||||
@login_required
|
@login_required
|
||||||
def audit():
|
def linux_index():
|
||||||
"""Run full security audit."""
|
from core.menu import MainMenu
|
||||||
|
menu = MainMenu()
|
||||||
|
menu.load_modules()
|
||||||
|
modules = {k: v for k, v in menu.modules.items() if v.category == 'defense'}
|
||||||
|
return render_template('defense_linux.html', modules=modules)
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/linux/audit', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def linux_audit():
|
||||||
|
"""Run full Linux security audit."""
|
||||||
from modules.defender import Defender
|
from modules.defender import Defender
|
||||||
d = Defender()
|
d = Defender()
|
||||||
d.check_firewall()
|
d.check_firewall()
|
||||||
@ -53,10 +82,10 @@ def audit():
|
|||||||
})
|
})
|
||||||
|
|
||||||
|
|
||||||
@defense_bp.route('/check/<check_name>', methods=['POST'])
|
@defense_bp.route('/linux/check/<check_name>', methods=['POST'])
|
||||||
@login_required
|
@login_required
|
||||||
def check(check_name):
|
def linux_check(check_name):
|
||||||
"""Run individual security check."""
|
"""Run individual Linux security check."""
|
||||||
from modules.defender import Defender
|
from modules.defender import Defender
|
||||||
d = Defender()
|
d = Defender()
|
||||||
|
|
||||||
@ -79,9 +108,9 @@ def check(check_name):
|
|||||||
return jsonify({'checks': d.results})
|
return jsonify({'checks': d.results})
|
||||||
|
|
||||||
|
|
||||||
@defense_bp.route('/firewall/rules')
|
@defense_bp.route('/linux/firewall/rules')
|
||||||
@login_required
|
@login_required
|
||||||
def firewall_rules():
|
def linux_firewall_rules():
|
||||||
"""Get current iptables rules."""
|
"""Get current iptables rules."""
|
||||||
success, output = _run_cmd("sudo iptables -L -n --line-numbers 2>/dev/null")
|
success, output = _run_cmd("sudo iptables -L -n --line-numbers 2>/dev/null")
|
||||||
if success:
|
if success:
|
||||||
@ -89,10 +118,10 @@ def firewall_rules():
|
|||||||
return jsonify({'rules': 'Could not read iptables rules (need sudo privileges)'})
|
return jsonify({'rules': 'Could not read iptables rules (need sudo privileges)'})
|
||||||
|
|
||||||
|
|
||||||
@defense_bp.route('/firewall/block', methods=['POST'])
|
@defense_bp.route('/linux/firewall/block', methods=['POST'])
|
||||||
@login_required
|
@login_required
|
||||||
def firewall_block():
|
def linux_firewall_block():
|
||||||
"""Block an IP address."""
|
"""Block an IP address via iptables."""
|
||||||
data = request.get_json(silent=True) or {}
|
data = request.get_json(silent=True) or {}
|
||||||
ip = data.get('ip', '').strip()
|
ip = data.get('ip', '').strip()
|
||||||
if not ip or not re.match(r'^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$', ip):
|
if not ip or not re.match(r'^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$', ip):
|
||||||
@ -104,10 +133,10 @@ def firewall_block():
|
|||||||
return jsonify({'error': f'Failed to block {ip} (need sudo)', 'success': False})
|
return jsonify({'error': f'Failed to block {ip} (need sudo)', 'success': False})
|
||||||
|
|
||||||
|
|
||||||
@defense_bp.route('/firewall/unblock', methods=['POST'])
|
@defense_bp.route('/linux/firewall/unblock', methods=['POST'])
|
||||||
@login_required
|
@login_required
|
||||||
def firewall_unblock():
|
def linux_firewall_unblock():
|
||||||
"""Unblock an IP address."""
|
"""Unblock an IP address via iptables."""
|
||||||
data = request.get_json(silent=True) or {}
|
data = request.get_json(silent=True) or {}
|
||||||
ip = data.get('ip', '').strip()
|
ip = data.get('ip', '').strip()
|
||||||
if not ip or not re.match(r'^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$', ip):
|
if not ip or not re.match(r'^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$', ip):
|
||||||
@ -119,10 +148,10 @@ def firewall_unblock():
|
|||||||
return jsonify({'error': f'Failed to unblock {ip}', 'success': False})
|
return jsonify({'error': f'Failed to unblock {ip}', 'success': False})
|
||||||
|
|
||||||
|
|
||||||
@defense_bp.route('/logs/analyze', methods=['POST'])
|
@defense_bp.route('/linux/logs/analyze', methods=['POST'])
|
||||||
@login_required
|
@login_required
|
||||||
def logs_analyze():
|
def linux_logs_analyze():
|
||||||
"""Analyze auth and web logs."""
|
"""Analyze auth and web logs (Linux)."""
|
||||||
from modules.defender import Defender
|
from modules.defender import Defender
|
||||||
d = Defender()
|
d = Defender()
|
||||||
auth_results = d._analyze_auth_log()
|
auth_results = d._analyze_auth_log()
|
||||||
@ -132,3 +161,498 @@ def logs_analyze():
|
|||||||
'auth_results': auth_results[:20],
|
'auth_results': auth_results[:20],
|
||||||
'web_results': web_results[:20],
|
'web_results': web_results[:20],
|
||||||
})
|
})
|
||||||
|
|
||||||
|
|
||||||
|
# ==================== WINDOWS DEFENSE ====================
|
||||||
|
|
||||||
|
@defense_bp.route('/windows')
|
||||||
|
@login_required
|
||||||
|
def windows_index():
|
||||||
|
"""Windows defense sub-page."""
|
||||||
|
return render_template('defense_windows.html')
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/windows/audit', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def windows_audit():
|
||||||
|
"""Run full Windows security audit."""
|
||||||
|
from modules.defender_windows import WindowsDefender
|
||||||
|
d = WindowsDefender()
|
||||||
|
d.check_firewall()
|
||||||
|
d.check_ssh_config()
|
||||||
|
d.check_open_ports()
|
||||||
|
d.check_updates()
|
||||||
|
d.check_users()
|
||||||
|
d.check_permissions()
|
||||||
|
d.check_services()
|
||||||
|
d.check_defender()
|
||||||
|
d.check_uac()
|
||||||
|
|
||||||
|
passed = sum(1 for r in d.results if r['passed'])
|
||||||
|
total = len(d.results)
|
||||||
|
score = int((passed / total) * 100) if total > 0 else 0
|
||||||
|
|
||||||
|
return jsonify({
|
||||||
|
'score': score,
|
||||||
|
'passed': passed,
|
||||||
|
'total': total,
|
||||||
|
'checks': d.results
|
||||||
|
})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/windows/check/<check_name>', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def windows_check(check_name):
|
||||||
|
"""Run individual Windows security check."""
|
||||||
|
from modules.defender_windows import WindowsDefender
|
||||||
|
d = WindowsDefender()
|
||||||
|
|
||||||
|
checks_map = {
|
||||||
|
'firewall': d.check_firewall,
|
||||||
|
'ssh': d.check_ssh_config,
|
||||||
|
'ports': d.check_open_ports,
|
||||||
|
'updates': d.check_updates,
|
||||||
|
'users': d.check_users,
|
||||||
|
'permissions': d.check_permissions,
|
||||||
|
'services': d.check_services,
|
||||||
|
'defender': d.check_defender,
|
||||||
|
'uac': d.check_uac,
|
||||||
|
}
|
||||||
|
|
||||||
|
func = checks_map.get(check_name)
|
||||||
|
if not func:
|
||||||
|
return jsonify({'error': f'Unknown check: {check_name}'}), 400
|
||||||
|
|
||||||
|
func()
|
||||||
|
return jsonify({'checks': d.results})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/windows/firewall/rules')
|
||||||
|
@login_required
|
||||||
|
def windows_firewall_rules():
|
||||||
|
"""Get Windows Firewall rules."""
|
||||||
|
from modules.defender_windows import WindowsDefender
|
||||||
|
d = WindowsDefender()
|
||||||
|
success, output = d.get_firewall_rules()
|
||||||
|
if success:
|
||||||
|
return jsonify({'rules': output})
|
||||||
|
return jsonify({'rules': 'Could not read Windows Firewall rules (need admin privileges)'})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/windows/firewall/block', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def windows_firewall_block():
|
||||||
|
"""Block an IP via Windows Firewall."""
|
||||||
|
data = request.get_json(silent=True) or {}
|
||||||
|
ip = data.get('ip', '').strip()
|
||||||
|
if not ip or not re.match(r'^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$', ip):
|
||||||
|
return jsonify({'error': 'Invalid IP address', 'success': False})
|
||||||
|
|
||||||
|
from modules.defender_windows import WindowsDefender
|
||||||
|
d = WindowsDefender()
|
||||||
|
success, message = d.block_ip(ip)
|
||||||
|
return jsonify({'message': message, 'success': success})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/windows/firewall/unblock', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def windows_firewall_unblock():
|
||||||
|
"""Unblock an IP via Windows Firewall."""
|
||||||
|
data = request.get_json(silent=True) or {}
|
||||||
|
ip = data.get('ip', '').strip()
|
||||||
|
if not ip or not re.match(r'^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$', ip):
|
||||||
|
return jsonify({'error': 'Invalid IP address', 'success': False})
|
||||||
|
|
||||||
|
from modules.defender_windows import WindowsDefender
|
||||||
|
d = WindowsDefender()
|
||||||
|
success, message = d.unblock_ip(ip)
|
||||||
|
return jsonify({'message': message, 'success': success})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/windows/logs/analyze', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def windows_logs_analyze():
|
||||||
|
"""Analyze Windows Event Logs."""
|
||||||
|
from modules.defender_windows import WindowsDefender
|
||||||
|
d = WindowsDefender()
|
||||||
|
auth_results, system_results = d.analyze_event_logs()
|
||||||
|
|
||||||
|
return jsonify({
|
||||||
|
'auth_results': auth_results[:20],
|
||||||
|
'system_results': system_results[:20],
|
||||||
|
})
|
||||||
|
|
||||||
|
|
||||||
|
# ==================== THREAT MONITOR ====================
|
||||||
|
|
||||||
|
|
||||||
|
def _get_monitor():
|
||||||
|
"""Get singleton ThreatMonitor instance."""
|
||||||
|
from modules.defender_monitor import get_threat_monitor
|
||||||
|
return get_threat_monitor()
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor')
|
||||||
|
@login_required
|
||||||
|
def monitor_index():
|
||||||
|
"""Threat Monitor sub-page."""
|
||||||
|
return render_template('defense_monitor.html')
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/stream')
|
||||||
|
@login_required
|
||||||
|
def monitor_stream():
|
||||||
|
"""SSE stream of real-time threat data."""
|
||||||
|
return Response(stream_with_context(_get_monitor().monitor_stream()),
|
||||||
|
mimetype='text/event-stream',
|
||||||
|
headers={'Cache-Control': 'no-cache', 'X-Accel-Buffering': 'no'})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/connections', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_connections():
|
||||||
|
"""Get current network connections."""
|
||||||
|
m = _get_monitor()
|
||||||
|
connections = m.get_connections()
|
||||||
|
return jsonify({'connections': connections, 'total': len(connections)})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/processes', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_processes():
|
||||||
|
"""Get suspicious processes."""
|
||||||
|
m = _get_monitor()
|
||||||
|
processes = m.get_suspicious_processes()
|
||||||
|
return jsonify({'processes': processes, 'total': len(processes)})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/threats', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_threats():
|
||||||
|
"""Get threat score and summary."""
|
||||||
|
m = _get_monitor()
|
||||||
|
report = m.generate_threat_report()
|
||||||
|
return jsonify(report)
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/block-ip', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_block_ip():
|
||||||
|
"""Counter-attack: block an IP."""
|
||||||
|
data = request.get_json(silent=True) or {}
|
||||||
|
ip = data.get('ip', '').strip()
|
||||||
|
if not ip or not re.match(r'^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$', ip):
|
||||||
|
return jsonify({'error': 'Invalid IP address', 'success': False})
|
||||||
|
|
||||||
|
success, message = _get_monitor().auto_block_ip(ip)
|
||||||
|
return jsonify({'message': message, 'success': success})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/kill-process', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_kill_process():
|
||||||
|
"""Counter-attack: kill a process."""
|
||||||
|
data = request.get_json(silent=True) or {}
|
||||||
|
pid = data.get('pid')
|
||||||
|
if not pid:
|
||||||
|
return jsonify({'error': 'No PID provided', 'success': False})
|
||||||
|
|
||||||
|
success, message = _get_monitor().kill_process(pid)
|
||||||
|
return jsonify({'message': message, 'success': success})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/block-port', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_block_port():
|
||||||
|
"""Counter-attack: block a port."""
|
||||||
|
data = request.get_json(silent=True) or {}
|
||||||
|
port = data.get('port')
|
||||||
|
direction = data.get('direction', 'in')
|
||||||
|
if not port:
|
||||||
|
return jsonify({'error': 'No port provided', 'success': False})
|
||||||
|
|
||||||
|
success, message = _get_monitor().block_port(port, direction)
|
||||||
|
return jsonify({'message': message, 'success': success})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/blocklist')
|
||||||
|
@login_required
|
||||||
|
def monitor_blocklist_get():
|
||||||
|
"""Get persistent blocklist."""
|
||||||
|
return jsonify({'blocked_ips': _get_monitor().get_blocklist()})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/blocklist', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_blocklist_add():
|
||||||
|
"""Add IP to persistent blocklist."""
|
||||||
|
data = request.get_json(silent=True) or {}
|
||||||
|
ip = data.get('ip', '').strip()
|
||||||
|
if not ip or not re.match(r'^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$', ip):
|
||||||
|
return jsonify({'error': 'Invalid IP address', 'success': False})
|
||||||
|
|
||||||
|
blocklist = _get_monitor().add_to_blocklist(ip)
|
||||||
|
return jsonify({'blocked_ips': blocklist, 'success': True})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/blocklist/remove', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_blocklist_remove():
|
||||||
|
"""Remove IP from persistent blocklist."""
|
||||||
|
data = request.get_json(silent=True) or {}
|
||||||
|
ip = data.get('ip', '').strip()
|
||||||
|
if not ip:
|
||||||
|
return jsonify({'error': 'No IP provided', 'success': False})
|
||||||
|
|
||||||
|
blocklist = _get_monitor().remove_from_blocklist(ip)
|
||||||
|
return jsonify({'blocked_ips': blocklist, 'success': True})
|
||||||
|
|
||||||
|
|
||||||
|
# ==================== MONITORING: BANDWIDTH, ARP, PORTS, GEOIP, CONN RATE ====================
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/bandwidth', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_bandwidth():
|
||||||
|
"""Get bandwidth stats per interface."""
|
||||||
|
return jsonify({'interfaces': _get_monitor().get_bandwidth()})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/arp-check', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_arp_check():
|
||||||
|
"""Check for ARP spoofing."""
|
||||||
|
alerts = _get_monitor().check_arp_spoofing()
|
||||||
|
return jsonify({'alerts': alerts, 'total': len(alerts)})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/new-ports', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_new_ports():
|
||||||
|
"""Check for new listening ports."""
|
||||||
|
ports = _get_monitor().check_new_listening_ports()
|
||||||
|
return jsonify({'new_ports': ports, 'total': len(ports)})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/geoip', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_geoip():
|
||||||
|
"""GeoIP lookup for an IP."""
|
||||||
|
data = request.get_json(silent=True) or {}
|
||||||
|
ip = data.get('ip', '').strip()
|
||||||
|
if not ip:
|
||||||
|
return jsonify({'error': 'No IP provided'}), 400
|
||||||
|
result = _get_monitor().geoip_lookup(ip)
|
||||||
|
if result:
|
||||||
|
return jsonify(result)
|
||||||
|
return jsonify({'ip': ip, 'error': 'Private IP or lookup failed'})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/connections-geo', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_connections_geo():
|
||||||
|
"""Get connections enriched with GeoIP data."""
|
||||||
|
connections = _get_monitor().get_connections_with_geoip()
|
||||||
|
return jsonify({'connections': connections, 'total': len(connections)})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/connection-rate', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_connection_rate():
|
||||||
|
"""Get connection rate stats."""
|
||||||
|
return jsonify(_get_monitor().get_connection_rate())
|
||||||
|
|
||||||
|
|
||||||
|
# ==================== PACKET CAPTURE (via WiresharkManager) ====================
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/capture/interfaces')
|
||||||
|
@login_required
|
||||||
|
def monitor_capture_interfaces():
|
||||||
|
"""Get available network interfaces for capture."""
|
||||||
|
from core.wireshark import get_wireshark_manager
|
||||||
|
wm = get_wireshark_manager()
|
||||||
|
return jsonify({'interfaces': wm.list_interfaces()})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/capture/start', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_capture_start():
|
||||||
|
"""Start packet capture."""
|
||||||
|
data = request.get_json(silent=True) or {}
|
||||||
|
from core.wireshark import get_wireshark_manager
|
||||||
|
wm = get_wireshark_manager()
|
||||||
|
result = wm.start_capture(
|
||||||
|
interface=data.get('interface', ''),
|
||||||
|
bpf_filter=data.get('filter', ''),
|
||||||
|
duration=int(data.get('duration', 30)),
|
||||||
|
)
|
||||||
|
return jsonify(result)
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/capture/stop', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_capture_stop():
|
||||||
|
"""Stop packet capture."""
|
||||||
|
from core.wireshark import get_wireshark_manager
|
||||||
|
wm = get_wireshark_manager()
|
||||||
|
result = wm.stop_capture()
|
||||||
|
return jsonify(result)
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/capture/stats')
|
||||||
|
@login_required
|
||||||
|
def monitor_capture_stats():
|
||||||
|
"""Get capture statistics."""
|
||||||
|
from core.wireshark import get_wireshark_manager
|
||||||
|
wm = get_wireshark_manager()
|
||||||
|
return jsonify(wm.get_capture_stats())
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/capture/stream')
|
||||||
|
@login_required
|
||||||
|
def monitor_capture_stream():
|
||||||
|
"""SSE stream of captured packets."""
|
||||||
|
import time as _time
|
||||||
|
from core.wireshark import get_wireshark_manager
|
||||||
|
mgr = get_wireshark_manager()
|
||||||
|
|
||||||
|
def generate():
|
||||||
|
last_count = 0
|
||||||
|
while mgr._capture_running:
|
||||||
|
stats = mgr.get_capture_stats()
|
||||||
|
count = stats.get('packet_count', 0)
|
||||||
|
if count > last_count:
|
||||||
|
new_packets = mgr._capture_packets[last_count:count]
|
||||||
|
for pkt in new_packets:
|
||||||
|
yield f'data: {json.dumps({"type": "packet", **pkt})}\n\n'
|
||||||
|
last_count = count
|
||||||
|
yield f'data: {json.dumps({"type": "stats", "packet_count": count, "running": True})}\n\n'
|
||||||
|
_time.sleep(0.5)
|
||||||
|
stats = mgr.get_capture_stats()
|
||||||
|
yield f'data: {json.dumps({"type": "done", **stats})}\n\n'
|
||||||
|
|
||||||
|
return Response(stream_with_context(generate()),
|
||||||
|
mimetype='text/event-stream',
|
||||||
|
headers={'Cache-Control': 'no-cache', 'X-Accel-Buffering': 'no'})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/capture/protocols', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_capture_protocols():
|
||||||
|
"""Get protocol distribution from captured packets."""
|
||||||
|
from core.wireshark import get_wireshark_manager
|
||||||
|
wm = get_wireshark_manager()
|
||||||
|
return jsonify({'protocols': wm.get_protocol_hierarchy()})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/capture/conversations', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_capture_conversations():
|
||||||
|
"""Get top conversations from captured packets."""
|
||||||
|
from core.wireshark import get_wireshark_manager
|
||||||
|
wm = get_wireshark_manager()
|
||||||
|
return jsonify({'conversations': wm.extract_conversations()})
|
||||||
|
|
||||||
|
|
||||||
|
# ==================== DDOS MITIGATION ====================
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/ddos/detect', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_ddos_detect():
|
||||||
|
"""Detect DDoS/DoS attack patterns."""
|
||||||
|
return jsonify(_get_monitor().detect_ddos())
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/ddos/top-talkers', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_ddos_top_talkers():
|
||||||
|
"""Get top source IPs by connection count."""
|
||||||
|
data = request.get_json(silent=True) or {}
|
||||||
|
limit = int(data.get('limit', 20))
|
||||||
|
return jsonify({'talkers': _get_monitor().get_top_talkers(limit)})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/ddos/rate-limit', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_ddos_rate_limit():
|
||||||
|
"""Apply rate limit to an IP."""
|
||||||
|
data = request.get_json(silent=True) or {}
|
||||||
|
ip = data.get('ip', '').strip()
|
||||||
|
rate = data.get('rate', '25/min')
|
||||||
|
if not ip or not re.match(r'^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$', ip):
|
||||||
|
return jsonify({'error': 'Invalid IP address', 'success': False})
|
||||||
|
success, msg = _get_monitor().apply_rate_limit(ip, rate)
|
||||||
|
return jsonify({'message': msg, 'success': success})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/ddos/rate-limit/remove', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_ddos_rate_limit_remove():
|
||||||
|
"""Remove rate limit from an IP."""
|
||||||
|
data = request.get_json(silent=True) or {}
|
||||||
|
ip = data.get('ip', '').strip()
|
||||||
|
if not ip:
|
||||||
|
return jsonify({'error': 'No IP provided', 'success': False})
|
||||||
|
success, msg = _get_monitor().remove_rate_limit(ip)
|
||||||
|
return jsonify({'message': msg, 'success': success})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/ddos/syn-status')
|
||||||
|
@login_required
|
||||||
|
def monitor_ddos_syn_status():
|
||||||
|
"""Check SYN flood protection status."""
|
||||||
|
return jsonify(_get_monitor().get_syn_protection_status())
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/ddos/syn-enable', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_ddos_syn_enable():
|
||||||
|
"""Enable SYN flood protection."""
|
||||||
|
success, msg = _get_monitor().enable_syn_protection()
|
||||||
|
return jsonify({'message': msg, 'success': success})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/ddos/syn-disable', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_ddos_syn_disable():
|
||||||
|
"""Disable SYN flood protection."""
|
||||||
|
success, msg = _get_monitor().disable_syn_protection()
|
||||||
|
return jsonify({'message': msg, 'success': success})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/ddos/config')
|
||||||
|
@login_required
|
||||||
|
def monitor_ddos_config_get():
|
||||||
|
"""Get DDoS auto-mitigation config."""
|
||||||
|
return jsonify(_get_monitor().get_ddos_config())
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/ddos/config', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_ddos_config_save():
|
||||||
|
"""Save DDoS auto-mitigation config."""
|
||||||
|
data = request.get_json(silent=True) or {}
|
||||||
|
config = _get_monitor().save_ddos_config(data)
|
||||||
|
return jsonify({'config': config, 'success': True})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/ddos/auto-mitigate', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_ddos_auto_mitigate():
|
||||||
|
"""Run auto-mitigation."""
|
||||||
|
result = _get_monitor().auto_mitigate()
|
||||||
|
return jsonify(result)
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/ddos/history')
|
||||||
|
@login_required
|
||||||
|
def monitor_ddos_history():
|
||||||
|
"""Get mitigation history."""
|
||||||
|
return jsonify({'history': _get_monitor().get_mitigation_history()})
|
||||||
|
|
||||||
|
|
||||||
|
@defense_bp.route('/monitor/ddos/history/clear', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def monitor_ddos_history_clear():
|
||||||
|
"""Clear mitigation history."""
|
||||||
|
_get_monitor().clear_mitigation_history()
|
||||||
|
return jsonify({'success': True})
|
||||||
|
|||||||
180
web/routes/llm_trainer.py
Normal file
180
web/routes/llm_trainer.py
Normal file
@ -0,0 +1,180 @@
|
|||||||
|
"""LLM Trainer routes — dataset generation, fine-tuning, GGUF conversion."""
|
||||||
|
|
||||||
|
import json
|
||||||
|
from flask import Blueprint, render_template, request, jsonify, Response, stream_with_context
|
||||||
|
from web.auth import login_required
|
||||||
|
|
||||||
|
llm_trainer_bp = Blueprint('llm_trainer', __name__, url_prefix='/llm-trainer')
|
||||||
|
|
||||||
|
|
||||||
|
def _get_trainer():
|
||||||
|
from modules.llm_trainer import get_trainer
|
||||||
|
return get_trainer()
|
||||||
|
|
||||||
|
|
||||||
|
# ==================== PAGE ====================
|
||||||
|
|
||||||
|
@llm_trainer_bp.route('/')
|
||||||
|
@login_required
|
||||||
|
def index():
|
||||||
|
return render_template('llm_trainer.html')
|
||||||
|
|
||||||
|
|
||||||
|
# ==================== DEPENDENCIES ====================
|
||||||
|
|
||||||
|
@llm_trainer_bp.route('/deps', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def check_deps():
|
||||||
|
"""Check training dependencies."""
|
||||||
|
return jsonify(_get_trainer().check_dependencies())
|
||||||
|
|
||||||
|
|
||||||
|
@llm_trainer_bp.route('/deps/install', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def install_deps():
|
||||||
|
"""Install training dependencies."""
|
||||||
|
results = _get_trainer().install_dependencies()
|
||||||
|
return jsonify({'results': results})
|
||||||
|
|
||||||
|
|
||||||
|
# ==================== CODEBASE ====================
|
||||||
|
|
||||||
|
@llm_trainer_bp.route('/scan', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def scan_codebase():
|
||||||
|
"""Scan the AUTARCH codebase."""
|
||||||
|
return jsonify(_get_trainer().scan_codebase())
|
||||||
|
|
||||||
|
|
||||||
|
# ==================== DATASET ====================
|
||||||
|
|
||||||
|
@llm_trainer_bp.route('/dataset/generate', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def generate_dataset():
|
||||||
|
"""Generate training dataset from codebase."""
|
||||||
|
data = request.get_json(silent=True) or {}
|
||||||
|
result = _get_trainer().generate_dataset(
|
||||||
|
format=data.get('format', 'sharegpt'),
|
||||||
|
include_source=data.get('include_source', True),
|
||||||
|
include_qa=data.get('include_qa', True),
|
||||||
|
include_module_creation=data.get('include_module_creation', True),
|
||||||
|
)
|
||||||
|
return jsonify(result)
|
||||||
|
|
||||||
|
|
||||||
|
@llm_trainer_bp.route('/dataset/list')
|
||||||
|
@login_required
|
||||||
|
def list_datasets():
|
||||||
|
"""List generated datasets."""
|
||||||
|
return jsonify({'datasets': _get_trainer().list_datasets()})
|
||||||
|
|
||||||
|
|
||||||
|
@llm_trainer_bp.route('/dataset/preview', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def preview_dataset():
|
||||||
|
"""Preview samples from a dataset."""
|
||||||
|
data = request.get_json(silent=True) or {}
|
||||||
|
filename = data.get('filename', '')
|
||||||
|
limit = int(data.get('limit', 10))
|
||||||
|
return jsonify(_get_trainer().preview_dataset(filename, limit))
|
||||||
|
|
||||||
|
|
||||||
|
@llm_trainer_bp.route('/dataset/delete', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def delete_dataset():
|
||||||
|
"""Delete a dataset file."""
|
||||||
|
data = request.get_json(silent=True) or {}
|
||||||
|
filename = data.get('filename', '')
|
||||||
|
success = _get_trainer().delete_dataset(filename)
|
||||||
|
return jsonify({'success': success})
|
||||||
|
|
||||||
|
|
||||||
|
# ==================== MODEL BROWSER ====================
|
||||||
|
|
||||||
|
@llm_trainer_bp.route('/browse', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def browse_models():
|
||||||
|
"""Browse local directories for model files."""
|
||||||
|
data = request.get_json(silent=True) or {}
|
||||||
|
directory = data.get('directory', '')
|
||||||
|
return jsonify(_get_trainer().browse_models(directory))
|
||||||
|
|
||||||
|
|
||||||
|
# ==================== TRAINING ====================
|
||||||
|
|
||||||
|
@llm_trainer_bp.route('/train/config')
|
||||||
|
@login_required
|
||||||
|
def get_training_config():
|
||||||
|
"""Get default training configuration."""
|
||||||
|
return jsonify(_get_trainer().get_training_config())
|
||||||
|
|
||||||
|
|
||||||
|
@llm_trainer_bp.route('/train/start', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def start_training():
|
||||||
|
"""Start LoRA fine-tuning."""
|
||||||
|
config = request.get_json(silent=True) or {}
|
||||||
|
return jsonify(_get_trainer().start_training(config))
|
||||||
|
|
||||||
|
|
||||||
|
@llm_trainer_bp.route('/train/status')
|
||||||
|
@login_required
|
||||||
|
def training_status():
|
||||||
|
"""Get training status and log."""
|
||||||
|
return jsonify(_get_trainer().get_training_status())
|
||||||
|
|
||||||
|
|
||||||
|
@llm_trainer_bp.route('/train/stop', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def stop_training():
|
||||||
|
"""Stop training."""
|
||||||
|
success = _get_trainer().stop_training()
|
||||||
|
return jsonify({'success': success})
|
||||||
|
|
||||||
|
|
||||||
|
# ==================== CONVERSION ====================
|
||||||
|
|
||||||
|
@llm_trainer_bp.route('/adapters')
|
||||||
|
@login_required
|
||||||
|
def list_adapters():
|
||||||
|
"""List saved LoRA adapters."""
|
||||||
|
return jsonify({'adapters': _get_trainer().list_adapters()})
|
||||||
|
|
||||||
|
|
||||||
|
@llm_trainer_bp.route('/convert', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def merge_and_convert():
|
||||||
|
"""Merge LoRA adapter and convert to GGUF."""
|
||||||
|
data = request.get_json(silent=True) or {}
|
||||||
|
adapter_path = data.get('adapter_path', '')
|
||||||
|
output_name = data.get('output_name', 'autarch_model')
|
||||||
|
quantization = data.get('quantization', 'Q5_K_M')
|
||||||
|
return jsonify(_get_trainer().merge_and_convert(adapter_path, output_name, quantization))
|
||||||
|
|
||||||
|
|
||||||
|
@llm_trainer_bp.route('/models')
|
||||||
|
@login_required
|
||||||
|
def list_models():
|
||||||
|
"""List GGUF models."""
|
||||||
|
return jsonify({'models': _get_trainer().list_models()})
|
||||||
|
|
||||||
|
|
||||||
|
# ==================== EVALUATION ====================
|
||||||
|
|
||||||
|
@llm_trainer_bp.route('/evaluate', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def evaluate_model():
|
||||||
|
"""Evaluate a GGUF model with test prompts."""
|
||||||
|
data = request.get_json(silent=True) or {}
|
||||||
|
model_path = data.get('model_path', '')
|
||||||
|
prompts = data.get('prompts', None)
|
||||||
|
return jsonify(_get_trainer().evaluate_model(model_path, prompts))
|
||||||
|
|
||||||
|
|
||||||
|
# ==================== STATUS ====================
|
||||||
|
|
||||||
|
@llm_trainer_bp.route('/status')
|
||||||
|
@login_required
|
||||||
|
def get_status():
|
||||||
|
"""Get trainer status."""
|
||||||
|
return jsonify(_get_trainer().get_status())
|
||||||
@ -4,6 +4,9 @@ import collections
|
|||||||
import json
|
import json
|
||||||
import logging
|
import logging
|
||||||
import os
|
import os
|
||||||
|
import platform
|
||||||
|
import re
|
||||||
|
import subprocess
|
||||||
import threading
|
import threading
|
||||||
import time
|
import time
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
@ -475,3 +478,131 @@ def debug_test():
|
|||||||
except ValueError:
|
except ValueError:
|
||||||
log.exception('EXCEPTION — error with full traceback')
|
log.exception('EXCEPTION — error with full traceback')
|
||||||
return jsonify({'ok': True, 'sent': 5})
|
return jsonify({'ok': True, 'sent': 5})
|
||||||
|
|
||||||
|
|
||||||
|
# ==================== DEPENDENCIES ====================
|
||||||
|
|
||||||
|
@settings_bp.route('/deps')
|
||||||
|
@login_required
|
||||||
|
def deps_index():
|
||||||
|
"""Dependencies management page."""
|
||||||
|
return render_template('system_deps.html')
|
||||||
|
|
||||||
|
|
||||||
|
@settings_bp.route('/deps/check', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def deps_check():
|
||||||
|
"""Check all system dependencies."""
|
||||||
|
import sys as _sys
|
||||||
|
|
||||||
|
groups = {
|
||||||
|
'core': {
|
||||||
|
'flask': 'import flask; print(flask.__version__)',
|
||||||
|
'jinja2': 'import jinja2; print(jinja2.__version__)',
|
||||||
|
'requests': 'import requests; print(requests.__version__)',
|
||||||
|
'cryptography': 'import cryptography; print(cryptography.__version__)',
|
||||||
|
},
|
||||||
|
'llm': {
|
||||||
|
'llama-cpp-python': 'import llama_cpp; print(llama_cpp.__version__)',
|
||||||
|
'transformers': 'import transformers; print(transformers.__version__)',
|
||||||
|
'anthropic': 'import anthropic; print(anthropic.__version__)',
|
||||||
|
},
|
||||||
|
'training': {
|
||||||
|
'torch': 'import torch; print(torch.__version__)',
|
||||||
|
'peft': 'import peft; print(peft.__version__)',
|
||||||
|
'datasets': 'import datasets; print(datasets.__version__)',
|
||||||
|
'trl': 'import trl; print(trl.__version__)',
|
||||||
|
'accelerate': 'import accelerate; print(accelerate.__version__)',
|
||||||
|
'bitsandbytes': 'import bitsandbytes; print(bitsandbytes.__version__)',
|
||||||
|
'unsloth': 'import unsloth; print(unsloth.__version__)',
|
||||||
|
},
|
||||||
|
'network': {
|
||||||
|
'scapy': 'import scapy; print(scapy.VERSION)',
|
||||||
|
'pyshark': 'import pyshark; print(pyshark.__version__)',
|
||||||
|
'miniupnpc': 'import miniupnpc; print("installed")',
|
||||||
|
'msgpack': 'import msgpack; print(msgpack.version)',
|
||||||
|
'paramiko': 'import paramiko; print(paramiko.__version__)',
|
||||||
|
},
|
||||||
|
'hardware': {
|
||||||
|
'pyserial': 'import serial; print(serial.__version__)',
|
||||||
|
'esptool': 'import esptool; print(esptool.__version__)',
|
||||||
|
'adb-shell': 'import adb_shell; print("installed")',
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
results = {}
|
||||||
|
for group, packages in groups.items():
|
||||||
|
results[group] = {}
|
||||||
|
for name, cmd in packages.items():
|
||||||
|
try:
|
||||||
|
result = subprocess.run(
|
||||||
|
[_sys.executable, '-c', cmd],
|
||||||
|
capture_output=True, text=True, timeout=15
|
||||||
|
)
|
||||||
|
if result.returncode == 0:
|
||||||
|
results[group][name] = {'installed': True, 'version': result.stdout.strip()}
|
||||||
|
else:
|
||||||
|
results[group][name] = {'installed': False, 'version': None}
|
||||||
|
except Exception:
|
||||||
|
results[group][name] = {'installed': False, 'version': None}
|
||||||
|
|
||||||
|
# GPU info
|
||||||
|
gpu = {}
|
||||||
|
try:
|
||||||
|
result = subprocess.run(
|
||||||
|
[_sys.executable, '-c',
|
||||||
|
'import torch; print(torch.cuda.is_available()); '
|
||||||
|
'print(torch.cuda.get_device_name(0) if torch.cuda.is_available() else "none"); '
|
||||||
|
'print(torch.version.cuda or "none")'],
|
||||||
|
capture_output=True, text=True, timeout=15
|
||||||
|
)
|
||||||
|
if result.returncode == 0:
|
||||||
|
lines = result.stdout.strip().split('\n')
|
||||||
|
gpu['cuda_available'] = lines[0].strip() == 'True'
|
||||||
|
gpu['device'] = lines[1].strip() if len(lines) > 1 else 'none'
|
||||||
|
gpu['cuda_version'] = lines[2].strip() if len(lines) > 2 else 'none'
|
||||||
|
except Exception:
|
||||||
|
gpu['cuda_available'] = False
|
||||||
|
results['gpu'] = gpu
|
||||||
|
|
||||||
|
# Python info
|
||||||
|
import sys as _s
|
||||||
|
results['python'] = {
|
||||||
|
'version': _s.version.split()[0],
|
||||||
|
'executable': _s.executable,
|
||||||
|
'platform': platform.platform(),
|
||||||
|
}
|
||||||
|
|
||||||
|
return jsonify(results)
|
||||||
|
|
||||||
|
|
||||||
|
@settings_bp.route('/deps/install', methods=['POST'])
|
||||||
|
@login_required
|
||||||
|
def deps_install():
|
||||||
|
"""Install packages."""
|
||||||
|
import sys as _sys
|
||||||
|
data = request.get_json(silent=True) or {}
|
||||||
|
packages = data.get('packages', [])
|
||||||
|
if not packages:
|
||||||
|
return jsonify({'error': 'No packages specified'}), 400
|
||||||
|
|
||||||
|
results = []
|
||||||
|
for pkg in packages:
|
||||||
|
# Sanitize package name
|
||||||
|
if not re.match(r'^[a-zA-Z0-9_\-\[\]]+$', pkg):
|
||||||
|
results.append({'package': pkg, 'success': False, 'output': 'Invalid package name'})
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
result = subprocess.run(
|
||||||
|
[_sys.executable, '-m', 'pip', 'install', pkg, '--quiet'],
|
||||||
|
capture_output=True, text=True, timeout=300
|
||||||
|
)
|
||||||
|
results.append({
|
||||||
|
'package': pkg,
|
||||||
|
'success': result.returncode == 0,
|
||||||
|
'output': result.stdout.strip() or result.stderr.strip()[:200],
|
||||||
|
})
|
||||||
|
except Exception as e:
|
||||||
|
results.append({'package': pkg, 'success': False, 'output': str(e)[:200]})
|
||||||
|
|
||||||
|
return jsonify({'results': results})
|
||||||
|
|||||||
@ -524,6 +524,74 @@ pre { background: var(--bg-primary); border: 1px solid var(--border); border-rad
|
|||||||
}
|
}
|
||||||
.hw-checkbox input { accent-color: var(--accent); }
|
.hw-checkbox input { accent-color: var(--accent); }
|
||||||
|
|
||||||
|
/* ── Monitor Popup/Modal ─────────────────────────────────────── */
|
||||||
|
.tmon-overlay {
|
||||||
|
position: fixed; top: 0; left: 0; right: 0; bottom: 0;
|
||||||
|
background: rgba(0,0,0,0.6); z-index: 1200;
|
||||||
|
display: flex; align-items: center; justify-content: center;
|
||||||
|
opacity: 0; pointer-events: none; transition: opacity 0.15s;
|
||||||
|
}
|
||||||
|
.tmon-overlay.open { opacity: 1; pointer-events: auto; }
|
||||||
|
|
||||||
|
.tmon-popup {
|
||||||
|
background: var(--bg-secondary); border: 1px solid var(--border);
|
||||||
|
border-radius: 10px; box-shadow: 0 12px 48px rgba(0,0,0,0.6);
|
||||||
|
width: 780px; max-width: calc(100vw - 40px);
|
||||||
|
max-height: calc(100vh - 80px); display: flex; flex-direction: column;
|
||||||
|
overflow: hidden; transform: scale(0.95); transition: transform 0.15s;
|
||||||
|
}
|
||||||
|
.tmon-overlay.open .tmon-popup { transform: scale(1); }
|
||||||
|
|
||||||
|
.tmon-popup-header {
|
||||||
|
display: flex; align-items: center; justify-content: space-between;
|
||||||
|
padding: 12px 18px; background: var(--bg-card); border-bottom: 1px solid var(--border);
|
||||||
|
flex-shrink: 0;
|
||||||
|
}
|
||||||
|
.tmon-popup-header h3 { font-size: 0.95rem; font-weight: 600; margin: 0; }
|
||||||
|
.tmon-popup-close {
|
||||||
|
background: none; border: none; color: var(--text-muted); font-size: 1.2rem;
|
||||||
|
cursor: pointer; padding: 4px 8px; border-radius: 4px; transition: color 0.15s;
|
||||||
|
}
|
||||||
|
.tmon-popup-close:hover { color: var(--danger); }
|
||||||
|
|
||||||
|
.tmon-popup-body {
|
||||||
|
flex: 1; overflow-y: auto; padding: 16px 18px;
|
||||||
|
}
|
||||||
|
.tmon-popup-body .data-table { width: 100%; font-size: 0.8rem; }
|
||||||
|
|
||||||
|
.tmon-popup-status {
|
||||||
|
font-size: 0.75rem; color: var(--text-muted); padding: 8px 18px;
|
||||||
|
border-top: 1px solid var(--border); flex-shrink: 0; text-align: right;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Clickable stat cells in live monitor */
|
||||||
|
.tmon-stat-clickable {
|
||||||
|
cursor: pointer; transition: color 0.15s;
|
||||||
|
}
|
||||||
|
.tmon-stat-clickable:hover {
|
||||||
|
color: var(--accent) !important; text-decoration: underline;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* Connection detail card inside popup */
|
||||||
|
.tmon-detail-card {
|
||||||
|
background: var(--bg-card); border: 1px solid var(--border);
|
||||||
|
border-radius: var(--radius); padding: 14px 18px; margin-bottom: 12px;
|
||||||
|
}
|
||||||
|
.tmon-detail-card h4 { font-size: 0.88rem; margin-bottom: 10px; color: var(--accent); }
|
||||||
|
.tmon-detail-card table { width: 100%; font-size: 0.82rem; }
|
||||||
|
.tmon-detail-card td:first-child { color: var(--text-muted); width: 140px; padding: 3px 0; }
|
||||||
|
.tmon-detail-card td:last-child { color: var(--text-primary); padding: 3px 0; }
|
||||||
|
|
||||||
|
.tmon-row-clickable { cursor: pointer; transition: background 0.1s; }
|
||||||
|
.tmon-row-clickable:hover { background: var(--bg-card) !important; }
|
||||||
|
|
||||||
|
.tmon-back-btn {
|
||||||
|
background: none; border: 1px solid var(--border); color: var(--text-secondary);
|
||||||
|
padding: 4px 12px; border-radius: 4px; font-size: 0.78rem; cursor: pointer;
|
||||||
|
margin-bottom: 12px; transition: all 0.15s;
|
||||||
|
}
|
||||||
|
.tmon-back-btn:hover { color: var(--accent); border-color: var(--accent); }
|
||||||
|
|
||||||
/* Responsive */
|
/* Responsive */
|
||||||
@media (max-width: 768px) {
|
@media (max-width: 768px) {
|
||||||
.sidebar { display: none; }
|
.sidebar { display: none; }
|
||||||
|
|||||||
@ -2088,8 +2088,9 @@ function halSend() {
|
|||||||
var msg = inp.value.trim();
|
var msg = inp.value.trim();
|
||||||
if (!msg) return;
|
if (!msg) return;
|
||||||
inp.value = '';
|
inp.value = '';
|
||||||
|
inp.disabled = true;
|
||||||
halAppend('user', msg);
|
halAppend('user', msg);
|
||||||
var botDiv = halAppend('bot', '');
|
var container = document.getElementById('hal-messages');
|
||||||
|
|
||||||
fetch('/api/chat', {
|
fetch('/api/chat', {
|
||||||
method: 'POST',
|
method: 'POST',
|
||||||
@ -2101,7 +2102,7 @@ function halSend() {
|
|||||||
var buf = '';
|
var buf = '';
|
||||||
function pump() {
|
function pump() {
|
||||||
reader.read().then(function(chunk) {
|
reader.read().then(function(chunk) {
|
||||||
if (chunk.done) return;
|
if (chunk.done) { inp.disabled = false; inp.focus(); return; }
|
||||||
buf += dec.decode(chunk.value, {stream: true});
|
buf += dec.decode(chunk.value, {stream: true});
|
||||||
var parts = buf.split('\n\n');
|
var parts = buf.split('\n\n');
|
||||||
buf = parts.pop();
|
buf = parts.pop();
|
||||||
@ -2110,8 +2111,30 @@ function halSend() {
|
|||||||
if (!line) return;
|
if (!line) return;
|
||||||
try {
|
try {
|
||||||
var d = JSON.parse(line);
|
var d = JSON.parse(line);
|
||||||
if (d.token) { botDiv.textContent += d.token; halScroll(); }
|
if (d.type === 'thought') {
|
||||||
if (d.error) { botDiv.textContent = 'Error: ' + d.error; }
|
halAppendStyled('thought', d.content);
|
||||||
|
} else if (d.type === 'action') {
|
||||||
|
halAppendStyled('action', d.content);
|
||||||
|
} else if (d.type === 'result') {
|
||||||
|
halAppendStyled('result', d.content);
|
||||||
|
} else if (d.type === 'answer') {
|
||||||
|
halAppendStyled('bot', d.content);
|
||||||
|
} else if (d.type === 'status') {
|
||||||
|
halAppendStyled('status', d.content);
|
||||||
|
} else if (d.type === 'error') {
|
||||||
|
halAppendStyled('error', d.content);
|
||||||
|
} else if (d.token) {
|
||||||
|
// Legacy streaming token mode
|
||||||
|
var last = container.lastElementChild;
|
||||||
|
if (!last || !last.classList.contains('hal-msg-bot')) {
|
||||||
|
last = halAppend('bot', '');
|
||||||
|
}
|
||||||
|
last.textContent += d.token;
|
||||||
|
halScroll();
|
||||||
|
} else if (d.done) {
|
||||||
|
inp.disabled = false;
|
||||||
|
inp.focus();
|
||||||
|
}
|
||||||
} catch(e) {}
|
} catch(e) {}
|
||||||
});
|
});
|
||||||
pump();
|
pump();
|
||||||
@ -2119,10 +2142,38 @@ function halSend() {
|
|||||||
}
|
}
|
||||||
pump();
|
pump();
|
||||||
}).catch(function(e) {
|
}).catch(function(e) {
|
||||||
if (botDiv) botDiv.textContent = 'Error: ' + e.message;
|
halAppendStyled('error', e.message);
|
||||||
|
inp.disabled = false;
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function halAppendStyled(type, text) {
|
||||||
|
var msgs = document.getElementById('hal-messages');
|
||||||
|
if (!msgs) return;
|
||||||
|
var div = document.createElement('div');
|
||||||
|
div.className = 'hal-msg hal-msg-' + type;
|
||||||
|
if (type === 'thought') {
|
||||||
|
div.style.cssText = 'font-style:italic;color:var(--text-muted,#888);font-size:0.8rem';
|
||||||
|
div.textContent = text;
|
||||||
|
} else if (type === 'action') {
|
||||||
|
div.style.cssText = 'font-family:monospace;color:var(--accent,#0af);font-size:0.78rem;background:rgba(0,170,255,0.08);padding:4px 8px;border-radius:4px';
|
||||||
|
div.textContent = '> ' + text;
|
||||||
|
} else if (type === 'result') {
|
||||||
|
div.style.cssText = 'font-family:monospace;color:var(--text-secondary,#aaa);font-size:0.75rem;max-height:100px;overflow-y:auto;white-space:pre-wrap;background:rgba(255,255,255,0.03);padding:4px 8px;border-radius:4px';
|
||||||
|
div.textContent = text;
|
||||||
|
} else if (type === 'status') {
|
||||||
|
div.style.cssText = 'color:var(--text-muted,#666);font-size:0.78rem;font-style:italic';
|
||||||
|
div.textContent = text;
|
||||||
|
} else if (type === 'error') {
|
||||||
|
div.style.cssText = 'color:var(--danger,#f55);font-size:0.82rem';
|
||||||
|
div.textContent = 'Error: ' + text;
|
||||||
|
} else {
|
||||||
|
div.textContent = text;
|
||||||
|
}
|
||||||
|
msgs.appendChild(div);
|
||||||
|
halScroll();
|
||||||
|
}
|
||||||
|
|
||||||
function halAppend(role, text) {
|
function halAppend(role, text) {
|
||||||
var msgs = document.getElementById('hal-messages');
|
var msgs = document.getElementById('hal-messages');
|
||||||
if (!msgs) return null;
|
if (!msgs) return null;
|
||||||
|
|||||||
@ -24,11 +24,15 @@
|
|||||||
<div class="nav-section">
|
<div class="nav-section">
|
||||||
<div class="nav-section-title">Categories</div>
|
<div class="nav-section-title">Categories</div>
|
||||||
<ul class="nav-links">
|
<ul class="nav-links">
|
||||||
<li><a href="{{ url_for('defense.index') }}" class="{% if request.blueprint == 'defense' %}active{% endif %}">Defense</a></li>
|
<li><a href="{{ url_for('defense.index') }}" class="{% if request.blueprint == 'defense' and request.endpoint not in ('defense.linux_index', 'defense.windows_index', 'defense.monitor_index') %}active{% endif %}">Defense</a></li>
|
||||||
|
<li><a href="{{ url_for('defense.linux_index') }}" class="{% if request.endpoint == 'defense.linux_index' %}active{% endif %}" style="padding-left:1.5rem;font-size:0.85rem">└ Linux</a></li>
|
||||||
|
<li><a href="{{ url_for('defense.windows_index') }}" class="{% if request.endpoint == 'defense.windows_index' %}active{% endif %}" style="padding-left:1.5rem;font-size:0.85rem">└ Windows</a></li>
|
||||||
|
<li><a href="{{ url_for('defense.monitor_index') }}" class="{% if request.endpoint == 'defense.monitor_index' %}active{% endif %}" style="padding-left:1.5rem;font-size:0.85rem">└ Threat Monitor</a></li>
|
||||||
<li><a href="{{ url_for('offense.index') }}" class="{% if request.blueprint == 'offense' %}active{% endif %}">Offense</a></li>
|
<li><a href="{{ url_for('offense.index') }}" class="{% if request.blueprint == 'offense' %}active{% endif %}">Offense</a></li>
|
||||||
<li><a href="{{ url_for('counter.index') }}" class="{% if request.blueprint == 'counter' %}active{% endif %}">Counter</a></li>
|
<li><a href="{{ url_for('counter.index') }}" class="{% if request.blueprint == 'counter' %}active{% endif %}">Counter</a></li>
|
||||||
<li><a href="{{ url_for('analyze.index') }}" class="{% if request.blueprint == 'analyze' and request.endpoint != 'analyze.hash_detection' %}active{% endif %}">Analyze</a></li>
|
<li><a href="{{ url_for('analyze.index') }}" class="{% if request.blueprint == 'analyze' and request.endpoint != 'analyze.hash_detection' %}active{% endif %}">Analyze</a></li>
|
||||||
<li><a href="{{ url_for('analyze.hash_detection') }}" class="{% if request.endpoint == 'analyze.hash_detection' %}active{% endif %}" style="padding-left:1.5rem;font-size:0.85rem">└ Hash Toolkit</a></li>
|
<li><a href="{{ url_for('analyze.hash_detection') }}" class="{% if request.endpoint == 'analyze.hash_detection' %}active{% endif %}" style="padding-left:1.5rem;font-size:0.85rem">└ Hash Toolkit</a></li>
|
||||||
|
<li><a href="{{ url_for('llm_trainer.index') }}" class="{% if request.blueprint == 'llm_trainer' %}active{% endif %}" style="padding-left:1.5rem;font-size:0.85rem">└ LLM Trainer</a></li>
|
||||||
<li><a href="{{ url_for('osint.index') }}" class="{% if request.blueprint == 'osint' %}active{% endif %}">OSINT</a></li>
|
<li><a href="{{ url_for('osint.index') }}" class="{% if request.blueprint == 'osint' %}active{% endif %}">OSINT</a></li>
|
||||||
<li><a href="{{ url_for('simulate.index') }}" class="{% if request.blueprint == 'simulate' and request.endpoint != 'simulate.legendary_creator' %}active{% endif %}">Simulate</a></li>
|
<li><a href="{{ url_for('simulate.index') }}" class="{% if request.blueprint == 'simulate' and request.endpoint != 'simulate.legendary_creator' %}active{% endif %}">Simulate</a></li>
|
||||||
<li><a href="{{ url_for('simulate.legendary_creator') }}" class="{% if request.endpoint == 'simulate.legendary_creator' %}active{% endif %}" style="padding-left:1.5rem;font-size:0.85rem">└ Legendary Creator</a></li>
|
<li><a href="{{ url_for('simulate.legendary_creator') }}" class="{% if request.endpoint == 'simulate.legendary_creator' %}active{% endif %}" style="padding-left:1.5rem;font-size:0.85rem">└ Legendary Creator</a></li>
|
||||||
@ -53,8 +57,9 @@
|
|||||||
<li><a href="{{ url_for('upnp.index') }}" class="{% if request.blueprint == 'upnp' %}active{% endif %}">UPnP</a></li>
|
<li><a href="{{ url_for('upnp.index') }}" class="{% if request.blueprint == 'upnp' %}active{% endif %}">UPnP</a></li>
|
||||||
<li><a href="{{ url_for('wireguard.index') }}" class="{% if request.blueprint == 'wireguard' %}active{% endif %}">WireGuard</a></li>
|
<li><a href="{{ url_for('wireguard.index') }}" class="{% if request.blueprint == 'wireguard' %}active{% endif %}">WireGuard</a></li>
|
||||||
<li><a href="{{ url_for('msf.index') }}" class="{% if request.blueprint == 'msf' %}active{% endif %}">MSF Console</a></li>
|
<li><a href="{{ url_for('msf.index') }}" class="{% if request.blueprint == 'msf' %}active{% endif %}">MSF Console</a></li>
|
||||||
<li><a href="{{ url_for('settings.index') }}" class="{% if request.blueprint == 'settings' and request.endpoint != 'settings.llm_settings' %}active{% endif %}">Settings</a></li>
|
<li><a href="{{ url_for('settings.index') }}" class="{% if request.blueprint == 'settings' and request.endpoint not in ('settings.llm_settings', 'settings.deps_index') %}active{% endif %}">Settings</a></li>
|
||||||
<li><a href="{{ url_for('settings.llm_settings') }}" class="{% if request.endpoint == 'settings.llm_settings' %}active{% endif %}" style="padding-left:1.5rem;font-size:0.85rem">└ LLM Config</a></li>
|
<li><a href="{{ url_for('settings.llm_settings') }}" class="{% if request.endpoint == 'settings.llm_settings' %}active{% endif %}" style="padding-left:1.5rem;font-size:0.85rem">└ LLM Config</a></li>
|
||||||
|
<li><a href="{{ url_for('settings.deps_index') }}" class="{% if request.endpoint == 'settings.deps_index' %}active{% endif %}" style="padding-left:1.5rem;font-size:0.85rem">└ Dependencies</a></li>
|
||||||
<li><a href="{{ url_for('dashboard.manual') }}" class="{% if request.endpoint == 'dashboard.manual' %}active{% endif %}">User Manual</a></li>
|
<li><a href="{{ url_for('dashboard.manual') }}" class="{% if request.endpoint == 'dashboard.manual' %}active{% endif %}">User Manual</a></li>
|
||||||
<li><a href="{{ url_for('dashboard.manual_windows') }}" class="{% if request.endpoint == 'dashboard.manual_windows' %}active{% endif %}" style="padding-left:1.5rem;font-size:0.85rem">└ Windows Guide</a></li>
|
<li><a href="{{ url_for('dashboard.manual_windows') }}" class="{% if request.endpoint == 'dashboard.manual_windows' %}active{% endif %}" style="padding-left:1.5rem;font-size:0.85rem">└ Windows Guide</a></li>
|
||||||
</ul>
|
</ul>
|
||||||
|
|||||||
@ -4,98 +4,46 @@
|
|||||||
{% block content %}
|
{% block content %}
|
||||||
<div class="page-header">
|
<div class="page-header">
|
||||||
<h1>System Defense</h1>
|
<h1>System Defense</h1>
|
||||||
|
<p style="margin:0;font-size:0.85rem;color:var(--text-secondary)">
|
||||||
|
Security hardening, firewall management, and threat monitoring.
|
||||||
|
</p>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<!-- Security Audit -->
|
<!-- Platform Info -->
|
||||||
<div class="section">
|
<div class="section">
|
||||||
<h2>Security Audit</h2>
|
<h2>System Overview</h2>
|
||||||
<div class="tool-actions">
|
<table class="data-table" style="max-width:500px">
|
||||||
<button id="btn-audit" class="btn btn-primary" onclick="runDefenseAudit()">Run Full Audit</button>
|
<tbody>
|
||||||
</div>
|
<tr><td>Platform</td><td><strong>{{ sys_info.platform }}</strong></td></tr>
|
||||||
<div style="display:flex;gap:24px;align-items:flex-start;flex-wrap:wrap">
|
<tr><td>Hostname</td><td>{{ sys_info.hostname }}</td></tr>
|
||||||
<div class="score-display">
|
<tr><td>OS</td><td>{{ sys_info.os_version }}</td></tr>
|
||||||
<div class="score-value" id="audit-score">--</div>
|
<tr><td>IP</td><td>{{ sys_info.ip }}</td></tr>
|
||||||
<div class="score-label">Security Score</div>
|
|
||||||
</div>
|
|
||||||
<div style="flex:1;min-width:300px">
|
|
||||||
<table class="data-table">
|
|
||||||
<thead><tr><th>Check</th><th>Status</th><th>Details</th></tr></thead>
|
|
||||||
<tbody id="audit-results">
|
|
||||||
<tr><td colspan="3" class="empty-state">Run an audit to see results.</td></tr>
|
|
||||||
</tbody>
|
</tbody>
|
||||||
</table>
|
</table>
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<!-- Individual Checks -->
|
<!-- Sub-page Cards -->
|
||||||
<div class="section">
|
<div class="section">
|
||||||
<h2>Quick Checks</h2>
|
<h2>Defense Tools</h2>
|
||||||
<div class="tool-grid">
|
<div class="tool-grid">
|
||||||
<div class="tool-card">
|
<div class="tool-card">
|
||||||
<h4>Firewall</h4>
|
<h4>Linux Defense</h4>
|
||||||
<p>Check iptables/ufw/firewalld status</p>
|
<p>iptables firewall, SSH hardening, systemd services, auth logs, SELinux/AppArmor checks.</p>
|
||||||
<button class="btn btn-small" onclick="runDefenseCheck('firewall')">Run</button>
|
<a href="{{ url_for('defense.linux_index') }}" class="btn btn-primary btn-small">Open</a>
|
||||||
<pre class="output-panel tool-result" id="check-result-firewall"></pre>
|
|
||||||
</div>
|
</div>
|
||||||
<div class="tool-card">
|
<div class="tool-card">
|
||||||
<h4>SSH Config</h4>
|
<h4>Windows Defense</h4>
|
||||||
<p>Check SSH hardening settings</p>
|
<p>Windows Firewall, UAC, Defender AV, event logs, OpenSSH, services, NTFS permissions.</p>
|
||||||
<button class="btn btn-small" onclick="runDefenseCheck('ssh')">Run</button>
|
<a href="{{ url_for('defense.windows_index') }}" class="btn btn-primary btn-small">Open</a>
|
||||||
<pre class="output-panel tool-result" id="check-result-ssh"></pre>
|
|
||||||
</div>
|
</div>
|
||||||
<div class="tool-card">
|
<div class="tool-card" style="border-color:var(--danger)">
|
||||||
<h4>Open Ports</h4>
|
<h4 style="color:var(--danger)">Threat Monitor</h4>
|
||||||
<p>Scan for high-risk listening ports</p>
|
<p>Real-time connection monitoring, port scan detection, suspicious process alerts, and counter-attack tools.</p>
|
||||||
<button class="btn btn-small" onclick="runDefenseCheck('ports')">Run</button>
|
<a href="{{ url_for('defense.monitor_index') }}" class="btn btn-danger btn-small">Open</a>
|
||||||
<pre class="output-panel tool-result" id="check-result-ports"></pre>
|
|
||||||
</div>
|
|
||||||
<div class="tool-card">
|
|
||||||
<h4>Users</h4>
|
|
||||||
<p>Check UID 0 users and empty passwords</p>
|
|
||||||
<button class="btn btn-small" onclick="runDefenseCheck('users')">Run</button>
|
|
||||||
<pre class="output-panel tool-result" id="check-result-users"></pre>
|
|
||||||
</div>
|
|
||||||
<div class="tool-card">
|
|
||||||
<h4>Permissions</h4>
|
|
||||||
<p>Check critical file permissions</p>
|
|
||||||
<button class="btn btn-small" onclick="runDefenseCheck('permissions')">Run</button>
|
|
||||||
<pre class="output-panel tool-result" id="check-result-permissions"></pre>
|
|
||||||
</div>
|
|
||||||
<div class="tool-card">
|
|
||||||
<h4>Services</h4>
|
|
||||||
<p>Check for dangerous services</p>
|
|
||||||
<button class="btn btn-small" onclick="runDefenseCheck('services')">Run</button>
|
|
||||||
<pre class="output-panel tool-result" id="check-result-services"></pre>
|
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<!-- Firewall Manager -->
|
|
||||||
<div class="section">
|
|
||||||
<h2>Firewall Manager</h2>
|
|
||||||
<div class="tool-actions">
|
|
||||||
<button class="btn btn-small" onclick="loadFirewallRules()">Refresh Rules</button>
|
|
||||||
</div>
|
|
||||||
<pre class="output-panel scrollable" id="fw-rules">Click "Refresh Rules" to load current iptables rules.</pre>
|
|
||||||
<div style="margin-top:12px">
|
|
||||||
<div class="input-row">
|
|
||||||
<input type="text" id="block-ip" placeholder="IP address to block">
|
|
||||||
<button class="btn btn-danger btn-small" onclick="blockIP()">Block IP</button>
|
|
||||||
</div>
|
|
||||||
<pre class="output-panel" id="fw-result" style="min-height:0"></pre>
|
|
||||||
</div>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
<!-- Log Analysis -->
|
|
||||||
<div class="section">
|
|
||||||
<h2>Log Analysis</h2>
|
|
||||||
<div class="tool-actions">
|
|
||||||
<button id="btn-logs" class="btn btn-primary" onclick="analyzeLogs()">Analyze Logs</button>
|
|
||||||
</div>
|
|
||||||
<pre class="output-panel scrollable" id="log-output">Click "Analyze Logs" to parse auth and web server logs.</pre>
|
|
||||||
</div>
|
|
||||||
|
|
||||||
{% if modules %}
|
{% if modules %}
|
||||||
<div class="section">
|
<div class="section">
|
||||||
<h2>Defense Modules</h2>
|
<h2>Defense Modules</h2>
|
||||||
|
|||||||
204
web/templates/defense_linux.html
Normal file
204
web/templates/defense_linux.html
Normal file
@ -0,0 +1,204 @@
|
|||||||
|
{% extends "base.html" %}
|
||||||
|
{% block title %}Linux Defense - AUTARCH{% endblock %}
|
||||||
|
|
||||||
|
{% block content %}
|
||||||
|
<div class="page-header" style="display:flex;align-items:center;gap:1rem;flex-wrap:wrap">
|
||||||
|
<div>
|
||||||
|
<h1>Linux Defense</h1>
|
||||||
|
<p style="margin:0;font-size:0.85rem;color:var(--text-secondary)">
|
||||||
|
Linux system hardening, iptables firewall management, and log analysis.
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
<a href="{{ url_for('defense.index') }}" class="btn btn-sm" style="margin-left:auto">← Defense</a>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Security Audit -->
|
||||||
|
<div class="section">
|
||||||
|
<h2>Security Audit</h2>
|
||||||
|
<div class="tool-actions">
|
||||||
|
<button id="btn-audit" class="btn btn-primary" onclick="linuxRunAudit()">Run Full Audit</button>
|
||||||
|
</div>
|
||||||
|
<div style="display:flex;gap:24px;align-items:flex-start;flex-wrap:wrap">
|
||||||
|
<div class="score-display">
|
||||||
|
<div class="score-value" id="audit-score">--</div>
|
||||||
|
<div class="score-label">Security Score</div>
|
||||||
|
</div>
|
||||||
|
<div style="flex:1;min-width:300px">
|
||||||
|
<table class="data-table">
|
||||||
|
<thead><tr><th>Check</th><th>Status</th><th>Details</th></tr></thead>
|
||||||
|
<tbody id="audit-results">
|
||||||
|
<tr><td colspan="3" class="empty-state">Run an audit to see results.</td></tr>
|
||||||
|
</tbody>
|
||||||
|
</table>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Individual Checks -->
|
||||||
|
<div class="section">
|
||||||
|
<h2>Quick Checks</h2>
|
||||||
|
<div class="tool-grid">
|
||||||
|
<div class="tool-card">
|
||||||
|
<h4>Firewall</h4>
|
||||||
|
<p>Check iptables/ufw/firewalld status</p>
|
||||||
|
<button class="btn btn-small" onclick="linuxRunCheck('firewall')">Run</button>
|
||||||
|
<pre class="output-panel tool-result" id="check-result-firewall"></pre>
|
||||||
|
</div>
|
||||||
|
<div class="tool-card">
|
||||||
|
<h4>SSH Config</h4>
|
||||||
|
<p>Check SSH hardening settings</p>
|
||||||
|
<button class="btn btn-small" onclick="linuxRunCheck('ssh')">Run</button>
|
||||||
|
<pre class="output-panel tool-result" id="check-result-ssh"></pre>
|
||||||
|
</div>
|
||||||
|
<div class="tool-card">
|
||||||
|
<h4>Open Ports</h4>
|
||||||
|
<p>Scan for high-risk listening ports</p>
|
||||||
|
<button class="btn btn-small" onclick="linuxRunCheck('ports')">Run</button>
|
||||||
|
<pre class="output-panel tool-result" id="check-result-ports"></pre>
|
||||||
|
</div>
|
||||||
|
<div class="tool-card">
|
||||||
|
<h4>Users</h4>
|
||||||
|
<p>Check UID 0 users and empty passwords</p>
|
||||||
|
<button class="btn btn-small" onclick="linuxRunCheck('users')">Run</button>
|
||||||
|
<pre class="output-panel tool-result" id="check-result-users"></pre>
|
||||||
|
</div>
|
||||||
|
<div class="tool-card">
|
||||||
|
<h4>Permissions</h4>
|
||||||
|
<p>Check critical file permissions</p>
|
||||||
|
<button class="btn btn-small" onclick="linuxRunCheck('permissions')">Run</button>
|
||||||
|
<pre class="output-panel tool-result" id="check-result-permissions"></pre>
|
||||||
|
</div>
|
||||||
|
<div class="tool-card">
|
||||||
|
<h4>Services</h4>
|
||||||
|
<p>Check for dangerous services</p>
|
||||||
|
<button class="btn btn-small" onclick="linuxRunCheck('services')">Run</button>
|
||||||
|
<pre class="output-panel tool-result" id="check-result-services"></pre>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Firewall Manager -->
|
||||||
|
<div class="section">
|
||||||
|
<h2>Firewall Manager (iptables)</h2>
|
||||||
|
<div class="tool-actions">
|
||||||
|
<button class="btn btn-small" onclick="linuxLoadFwRules()">Refresh Rules</button>
|
||||||
|
</div>
|
||||||
|
<pre class="output-panel scrollable" id="fw-rules">Click "Refresh Rules" to load current iptables rules.</pre>
|
||||||
|
<div style="margin-top:12px">
|
||||||
|
<div class="input-row">
|
||||||
|
<input type="text" id="block-ip" placeholder="IP address to block">
|
||||||
|
<button class="btn btn-danger btn-small" onclick="linuxBlockIP()">Block IP</button>
|
||||||
|
<button class="btn btn-small" onclick="linuxUnblockIP()">Unblock IP</button>
|
||||||
|
</div>
|
||||||
|
<pre class="output-panel" id="fw-result" style="min-height:0"></pre>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Log Analysis -->
|
||||||
|
<div class="section">
|
||||||
|
<h2>Log Analysis</h2>
|
||||||
|
<div class="tool-actions">
|
||||||
|
<button id="btn-logs" class="btn btn-primary" onclick="linuxAnalyzeLogs()">Analyze Logs</button>
|
||||||
|
</div>
|
||||||
|
<pre class="output-panel scrollable" id="log-output">Click "Analyze Logs" to parse auth and web server logs.</pre>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{% if modules %}
|
||||||
|
<div class="section">
|
||||||
|
<h2>Defense Modules</h2>
|
||||||
|
<ul class="module-list">
|
||||||
|
{% for name, info in modules.items() %}
|
||||||
|
<li class="module-item">
|
||||||
|
<div>
|
||||||
|
<div class="module-name">{{ name }}</div>
|
||||||
|
<div class="module-desc">{{ info.description }}</div>
|
||||||
|
</div>
|
||||||
|
<div class="module-meta">v{{ info.version }}</div>
|
||||||
|
</li>
|
||||||
|
{% endfor %}
|
||||||
|
</ul>
|
||||||
|
</div>
|
||||||
|
{% endif %}
|
||||||
|
|
||||||
|
<script>
|
||||||
|
/* ── Linux Defense (routes prefixed with /defense/linux/) ── */
|
||||||
|
function linuxRunAudit() {
|
||||||
|
var btn = document.getElementById('btn-audit');
|
||||||
|
setLoading(btn, true);
|
||||||
|
postJSON('/defense/linux/audit', {}).then(function(data) {
|
||||||
|
setLoading(btn, false);
|
||||||
|
if (data.error) { renderOutput('audit-results', 'Error: ' + data.error); return; }
|
||||||
|
var scoreEl = document.getElementById('audit-score');
|
||||||
|
if (scoreEl) {
|
||||||
|
scoreEl.textContent = data.score + '%';
|
||||||
|
scoreEl.style.color = data.score >= 80 ? 'var(--success)' : data.score >= 50 ? 'var(--warning)' : 'var(--danger)';
|
||||||
|
}
|
||||||
|
var html = '';
|
||||||
|
(data.checks || []).forEach(function(c) {
|
||||||
|
html += '<tr><td>' + escapeHtml(c.name) + '</td><td><span class="badge ' + (c.passed ? 'badge-pass' : 'badge-fail') + '">'
|
||||||
|
+ (c.passed ? 'PASS' : 'FAIL') + '</span></td><td>' + escapeHtml(c.details || '') + '</td></tr>';
|
||||||
|
});
|
||||||
|
document.getElementById('audit-results').innerHTML = html || '<tr><td colspan="3">No results</td></tr>';
|
||||||
|
}).catch(function() { setLoading(btn, false); });
|
||||||
|
}
|
||||||
|
|
||||||
|
function linuxRunCheck(name) {
|
||||||
|
var el = document.getElementById('check-result-' + name);
|
||||||
|
if (el) { el.textContent = 'Running...'; el.style.display = 'block'; }
|
||||||
|
postJSON('/defense/linux/check/' + name, {}).then(function(data) {
|
||||||
|
if (data.error) { if (el) el.textContent = 'Error: ' + data.error; return; }
|
||||||
|
var lines = (data.checks || []).map(function(c) {
|
||||||
|
return (c.passed ? '[PASS] ' : '[FAIL] ') + c.name + (c.details ? ' — ' + c.details : '');
|
||||||
|
});
|
||||||
|
if (el) el.textContent = lines.join('\n') || 'No results';
|
||||||
|
}).catch(function() { if (el) el.textContent = 'Request failed'; });
|
||||||
|
}
|
||||||
|
|
||||||
|
function linuxLoadFwRules() {
|
||||||
|
fetchJSON('/defense/linux/firewall/rules').then(function(data) {
|
||||||
|
renderOutput('fw-rules', data.rules || 'Could not load rules');
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function linuxBlockIP() {
|
||||||
|
var ip = document.getElementById('block-ip').value.trim();
|
||||||
|
if (!ip) return;
|
||||||
|
postJSON('/defense/linux/firewall/block', {ip: ip}).then(function(data) {
|
||||||
|
renderOutput('fw-result', data.message || data.error);
|
||||||
|
if (data.success) { document.getElementById('block-ip').value = ''; linuxLoadFwRules(); }
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function linuxUnblockIP() {
|
||||||
|
var ip = document.getElementById('block-ip').value.trim();
|
||||||
|
if (!ip) return;
|
||||||
|
postJSON('/defense/linux/firewall/unblock', {ip: ip}).then(function(data) {
|
||||||
|
renderOutput('fw-result', data.message || data.error);
|
||||||
|
if (data.success) linuxLoadFwRules();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function linuxAnalyzeLogs() {
|
||||||
|
var btn = document.getElementById('btn-logs');
|
||||||
|
setLoading(btn, true);
|
||||||
|
postJSON('/defense/linux/logs/analyze', {}).then(function(data) {
|
||||||
|
setLoading(btn, false);
|
||||||
|
if (data.error) { renderOutput('log-output', 'Error: ' + data.error); return; }
|
||||||
|
var lines = [];
|
||||||
|
if (data.auth_results && data.auth_results.length) {
|
||||||
|
lines.push('=== Auth Log Analysis ===');
|
||||||
|
data.auth_results.forEach(function(r) {
|
||||||
|
lines.push(r.ip + ': ' + r.count + ' failures (' + (r.usernames || []).join(', ') + ')');
|
||||||
|
});
|
||||||
|
} else { lines.push('No auth log entries found.'); }
|
||||||
|
if (data.web_results && data.web_results.length) {
|
||||||
|
lines.push('\n=== Web Log Analysis ===');
|
||||||
|
data.web_results.forEach(function(r) {
|
||||||
|
lines.push(r.ip + ': ' + r.count + ' suspicious requests');
|
||||||
|
});
|
||||||
|
}
|
||||||
|
renderOutput('log-output', lines.join('\n') || 'No findings.');
|
||||||
|
}).catch(function() { setLoading(btn, false); });
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
{% endblock %}
|
||||||
1286
web/templates/defense_monitor.html
Normal file
1286
web/templates/defense_monitor.html
Normal file
File diff suppressed because it is too large
Load Diff
224
web/templates/defense_windows.html
Normal file
224
web/templates/defense_windows.html
Normal file
@ -0,0 +1,224 @@
|
|||||||
|
{% extends "base.html" %}
|
||||||
|
{% block title %}Windows Defense - AUTARCH{% endblock %}
|
||||||
|
|
||||||
|
{% block content %}
|
||||||
|
<div class="page-header" style="display:flex;align-items:center;gap:1rem;flex-wrap:wrap">
|
||||||
|
<div>
|
||||||
|
<h1>Windows Defense</h1>
|
||||||
|
<p style="margin:0;font-size:0.85rem;color:var(--text-secondary)">
|
||||||
|
Windows-native security hardening, firewall management, and event log analysis.
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
<a href="{{ url_for('defense.index') }}" class="btn btn-sm" style="margin-left:auto">← Defense</a>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Tab Bar -->
|
||||||
|
<div class="tab-bar">
|
||||||
|
<button class="tab active" data-tab-group="windef" data-tab="audit" onclick="showTab('windef','audit')">Security Audit</button>
|
||||||
|
<button class="tab" data-tab-group="windef" data-tab="checks" onclick="showTab('windef','checks')">Quick Checks</button>
|
||||||
|
<button class="tab" data-tab-group="windef" data-tab="firewall" onclick="showTab('windef','firewall')">Firewall Manager</button>
|
||||||
|
<button class="tab" data-tab-group="windef" data-tab="logs" onclick="showTab('windef','logs')">Event Log Analysis</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- AUDIT TAB -->
|
||||||
|
<div class="tab-content active" data-tab-group="windef" data-tab="audit">
|
||||||
|
<div class="section">
|
||||||
|
<h2>Security Audit</h2>
|
||||||
|
<div class="tool-actions">
|
||||||
|
<button id="btn-win-audit" class="btn btn-primary" onclick="winRunAudit()">Run Full Audit</button>
|
||||||
|
</div>
|
||||||
|
<div style="display:flex;gap:24px;align-items:flex-start;flex-wrap:wrap">
|
||||||
|
<div class="score-display">
|
||||||
|
<div class="score-value" id="win-audit-score">--</div>
|
||||||
|
<div class="score-label">Security Score</div>
|
||||||
|
</div>
|
||||||
|
<div style="flex:1;min-width:300px">
|
||||||
|
<table class="data-table">
|
||||||
|
<thead><tr><th>Check</th><th>Status</th><th>Details</th></tr></thead>
|
||||||
|
<tbody id="win-audit-results">
|
||||||
|
<tr><td colspan="3" class="empty-state">Run an audit to see results.</td></tr>
|
||||||
|
</tbody>
|
||||||
|
</table>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- QUICK CHECKS TAB -->
|
||||||
|
<div class="tab-content" data-tab-group="windef" data-tab="checks">
|
||||||
|
<div class="section">
|
||||||
|
<h2>Quick Checks</h2>
|
||||||
|
<div class="tool-grid">
|
||||||
|
<div class="tool-card">
|
||||||
|
<h4>Windows Firewall</h4>
|
||||||
|
<p>Check firewall profile states</p>
|
||||||
|
<button class="btn btn-small" onclick="winRunCheck('firewall')">Run</button>
|
||||||
|
<pre class="output-panel tool-result" id="win-check-firewall"></pre>
|
||||||
|
</div>
|
||||||
|
<div class="tool-card">
|
||||||
|
<h4>SSH Config</h4>
|
||||||
|
<p>Check OpenSSH server settings</p>
|
||||||
|
<button class="btn btn-small" onclick="winRunCheck('ssh')">Run</button>
|
||||||
|
<pre class="output-panel tool-result" id="win-check-ssh"></pre>
|
||||||
|
</div>
|
||||||
|
<div class="tool-card">
|
||||||
|
<h4>Open Ports</h4>
|
||||||
|
<p>Scan for high-risk listening ports</p>
|
||||||
|
<button class="btn btn-small" onclick="winRunCheck('ports')">Run</button>
|
||||||
|
<pre class="output-panel tool-result" id="win-check-ports"></pre>
|
||||||
|
</div>
|
||||||
|
<div class="tool-card">
|
||||||
|
<h4>Updates</h4>
|
||||||
|
<p>Check installed Windows hotfixes</p>
|
||||||
|
<button class="btn btn-small" onclick="winRunCheck('updates')">Run</button>
|
||||||
|
<pre class="output-panel tool-result" id="win-check-updates"></pre>
|
||||||
|
</div>
|
||||||
|
<div class="tool-card">
|
||||||
|
<h4>Users</h4>
|
||||||
|
<p>Check admin accounts and guest status</p>
|
||||||
|
<button class="btn btn-small" onclick="winRunCheck('users')">Run</button>
|
||||||
|
<pre class="output-panel tool-result" id="win-check-users"></pre>
|
||||||
|
</div>
|
||||||
|
<div class="tool-card">
|
||||||
|
<h4>Permissions</h4>
|
||||||
|
<p>Check critical file/folder ACLs</p>
|
||||||
|
<button class="btn btn-small" onclick="winRunCheck('permissions')">Run</button>
|
||||||
|
<pre class="output-panel tool-result" id="win-check-permissions"></pre>
|
||||||
|
</div>
|
||||||
|
<div class="tool-card">
|
||||||
|
<h4>Services</h4>
|
||||||
|
<p>Check for risky Windows services</p>
|
||||||
|
<button class="btn btn-small" onclick="winRunCheck('services')">Run</button>
|
||||||
|
<pre class="output-panel tool-result" id="win-check-services"></pre>
|
||||||
|
</div>
|
||||||
|
<div class="tool-card">
|
||||||
|
<h4>Windows Defender</h4>
|
||||||
|
<p>AV status and real-time protection</p>
|
||||||
|
<button class="btn btn-small" onclick="winRunCheck('defender')">Run</button>
|
||||||
|
<pre class="output-panel tool-result" id="win-check-defender"></pre>
|
||||||
|
</div>
|
||||||
|
<div class="tool-card">
|
||||||
|
<h4>UAC Status</h4>
|
||||||
|
<p>User Account Control settings</p>
|
||||||
|
<button class="btn btn-small" onclick="winRunCheck('uac')">Run</button>
|
||||||
|
<pre class="output-panel tool-result" id="win-check-uac"></pre>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- FIREWALL TAB -->
|
||||||
|
<div class="tab-content" data-tab-group="windef" data-tab="firewall">
|
||||||
|
<div class="section">
|
||||||
|
<h2>Windows Firewall Manager</h2>
|
||||||
|
<div class="tool-actions">
|
||||||
|
<button class="btn btn-small" onclick="winLoadFwRules()">Refresh Rules</button>
|
||||||
|
</div>
|
||||||
|
<pre class="output-panel scrollable" id="win-fw-rules">Click "Refresh Rules" to load current Windows Firewall rules.</pre>
|
||||||
|
<div style="margin-top:12px">
|
||||||
|
<div class="input-row">
|
||||||
|
<input type="text" id="win-block-ip" placeholder="IP address to block">
|
||||||
|
<button class="btn btn-danger btn-small" onclick="winBlockIP()">Block IP</button>
|
||||||
|
<button class="btn btn-small" onclick="winUnblockIP()">Unblock IP</button>
|
||||||
|
</div>
|
||||||
|
<pre class="output-panel" id="win-fw-result" style="min-height:0"></pre>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- EVENT LOG TAB -->
|
||||||
|
<div class="tab-content" data-tab-group="windef" data-tab="logs">
|
||||||
|
<div class="section">
|
||||||
|
<h2>Event Log Analysis</h2>
|
||||||
|
<p style="font-size:0.8rem;color:var(--text-muted);margin-bottom:8px">
|
||||||
|
Analyze Windows Security and System event logs for failed logins, errors, and threats.
|
||||||
|
</p>
|
||||||
|
<div class="tool-actions">
|
||||||
|
<button id="btn-win-logs" class="btn btn-primary" onclick="winAnalyzeLogs()">Analyze Event Logs</button>
|
||||||
|
</div>
|
||||||
|
<pre class="output-panel scrollable" id="win-log-output">Click "Analyze Event Logs" to parse Windows event logs.</pre>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<script>
|
||||||
|
/* ── Windows Defense ── */
|
||||||
|
function winRunAudit() {
|
||||||
|
var btn = document.getElementById('btn-win-audit');
|
||||||
|
setLoading(btn, true);
|
||||||
|
postJSON('/defense/windows/audit', {}).then(function(data) {
|
||||||
|
setLoading(btn, false);
|
||||||
|
if (data.error) { renderOutput('win-audit-results', 'Error: ' + data.error); return; }
|
||||||
|
var scoreEl = document.getElementById('win-audit-score');
|
||||||
|
if (scoreEl) {
|
||||||
|
scoreEl.textContent = data.score + '%';
|
||||||
|
scoreEl.style.color = data.score >= 80 ? 'var(--success)' : data.score >= 50 ? 'var(--warning)' : 'var(--danger)';
|
||||||
|
}
|
||||||
|
var html = '';
|
||||||
|
(data.checks || []).forEach(function(c) {
|
||||||
|
html += '<tr><td>' + escapeHtml(c.name) + '</td><td><span class="badge ' + (c.passed ? 'badge-pass' : 'badge-fail') + '">'
|
||||||
|
+ (c.passed ? 'PASS' : 'FAIL') + '</span></td><td>' + escapeHtml(c.details || '') + '</td></tr>';
|
||||||
|
});
|
||||||
|
document.getElementById('win-audit-results').innerHTML = html || '<tr><td colspan="3">No results</td></tr>';
|
||||||
|
}).catch(function() { setLoading(btn, false); });
|
||||||
|
}
|
||||||
|
|
||||||
|
function winRunCheck(name) {
|
||||||
|
var el = document.getElementById('win-check-' + name);
|
||||||
|
if (el) { el.textContent = 'Running...'; el.style.display = 'block'; }
|
||||||
|
postJSON('/defense/windows/check/' + name, {}).then(function(data) {
|
||||||
|
if (data.error) { if (el) el.textContent = 'Error: ' + data.error; return; }
|
||||||
|
var lines = (data.checks || []).map(function(c) {
|
||||||
|
return (c.passed ? '[PASS] ' : '[FAIL] ') + c.name + (c.details ? ' — ' + c.details : '');
|
||||||
|
});
|
||||||
|
if (el) el.textContent = lines.join('\n') || 'No results';
|
||||||
|
}).catch(function() { if (el) el.textContent = 'Request failed'; });
|
||||||
|
}
|
||||||
|
|
||||||
|
function winLoadFwRules() {
|
||||||
|
fetchJSON('/defense/windows/firewall/rules').then(function(data) {
|
||||||
|
renderOutput('win-fw-rules', data.rules || 'Could not load rules');
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function winBlockIP() {
|
||||||
|
var ip = document.getElementById('win-block-ip').value.trim();
|
||||||
|
if (!ip) return;
|
||||||
|
postJSON('/defense/windows/firewall/block', {ip: ip}).then(function(data) {
|
||||||
|
renderOutput('win-fw-result', data.message || data.error);
|
||||||
|
if (data.success) { document.getElementById('win-block-ip').value = ''; winLoadFwRules(); }
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function winUnblockIP() {
|
||||||
|
var ip = document.getElementById('win-block-ip').value.trim();
|
||||||
|
if (!ip) return;
|
||||||
|
postJSON('/defense/windows/firewall/unblock', {ip: ip}).then(function(data) {
|
||||||
|
renderOutput('win-fw-result', data.message || data.error);
|
||||||
|
if (data.success) winLoadFwRules();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function winAnalyzeLogs() {
|
||||||
|
var btn = document.getElementById('btn-win-logs');
|
||||||
|
setLoading(btn, true);
|
||||||
|
postJSON('/defense/windows/logs/analyze', {}).then(function(data) {
|
||||||
|
setLoading(btn, false);
|
||||||
|
if (data.error) { renderOutput('win-log-output', 'Error: ' + data.error); return; }
|
||||||
|
var lines = [];
|
||||||
|
if (data.auth_results && data.auth_results.length) {
|
||||||
|
lines.push('=== Failed Login Attempts (Event ID 4625) ===');
|
||||||
|
data.auth_results.forEach(function(r) {
|
||||||
|
lines.push(r.ip + ': ' + r.count + ' failures (' + (r.usernames || []).join(', ') + ')');
|
||||||
|
});
|
||||||
|
} else { lines.push('No failed login attempts found in Security log.'); }
|
||||||
|
if (data.system_results && data.system_results.length) {
|
||||||
|
lines.push('\n=== System Log Warnings & Errors ===');
|
||||||
|
data.system_results.forEach(function(r) {
|
||||||
|
lines.push('[' + r.severity + '] Event ' + r.id + ' (' + r.type + '): ' + (r.detail || '').substring(0, 120));
|
||||||
|
});
|
||||||
|
}
|
||||||
|
renderOutput('win-log-output', lines.join('\n') || 'No findings.');
|
||||||
|
}).catch(function() { setLoading(btn, false); renderOutput('win-log-output', 'Request failed'); });
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
{% endblock %}
|
||||||
652
web/templates/llm_trainer.html
Normal file
652
web/templates/llm_trainer.html
Normal file
@ -0,0 +1,652 @@
|
|||||||
|
{% extends "base.html" %}
|
||||||
|
{% block title %}LLM Trainer - AUTARCH{% endblock %}
|
||||||
|
|
||||||
|
{% block content %}
|
||||||
|
<div class="page-header" style="display:flex;align-items:center;gap:1rem;flex-wrap:wrap">
|
||||||
|
<div>
|
||||||
|
<h1>LLM Trainer</h1>
|
||||||
|
<p style="margin:0;font-size:0.85rem;color:var(--text-secondary)">
|
||||||
|
Fine-tune language models on the AUTARCH codebase. Generate datasets, train LoRA adapters, convert to GGUF.
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
<a href="{{ url_for('analyze.index') }}" class="btn btn-sm" style="margin-left:auto">← Analyze</a>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Tab Bar -->
|
||||||
|
<div class="tab-bar">
|
||||||
|
<button class="tab active" data-tab-group="trainer" data-tab="dataset" onclick="showTab('trainer','dataset')">Dataset</button>
|
||||||
|
<button class="tab" data-tab-group="trainer" data-tab="training" onclick="showTab('trainer','training')">Training</button>
|
||||||
|
<button class="tab" data-tab-group="trainer" data-tab="convert" onclick="showTab('trainer','convert')">Convert</button>
|
||||||
|
<button class="tab" data-tab-group="trainer" data-tab="evaluate" onclick="showTab('trainer','evaluate')">Evaluate</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- ==================== DATASET TAB ==================== -->
|
||||||
|
<div class="tab-content active" data-tab-group="trainer" data-tab="dataset">
|
||||||
|
<div class="section">
|
||||||
|
<h2>Codebase Scan</h2>
|
||||||
|
<div class="tool-actions">
|
||||||
|
<button id="btn-scan" class="btn btn-primary" onclick="trScanCodebase()">Scan Codebase</button>
|
||||||
|
</div>
|
||||||
|
<div id="scan-result" style="margin-top:12px"></div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="section">
|
||||||
|
<h2>Generate Training Dataset</h2>
|
||||||
|
<p style="font-size:0.8rem;color:var(--text-muted);margin-bottom:12px">
|
||||||
|
Extracts code, architecture Q&A, and module creation examples from the AUTARCH codebase into JSONL training data.
|
||||||
|
</p>
|
||||||
|
<div class="form-row" style="margin-bottom:12px">
|
||||||
|
<div class="form-group">
|
||||||
|
<label>Format</label>
|
||||||
|
<select id="ds-format">
|
||||||
|
<option value="sharegpt">ShareGPT (conversations)</option>
|
||||||
|
<option value="instruction">Alpaca (instruction/output)</option>
|
||||||
|
</select>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div style="margin-bottom:12px">
|
||||||
|
<label><input type="checkbox" id="ds-source" checked> Include source code understanding</label><br>
|
||||||
|
<label><input type="checkbox" id="ds-qa" checked> Include architecture Q&A</label><br>
|
||||||
|
<label><input type="checkbox" id="ds-modules" checked> Include module creation examples</label>
|
||||||
|
</div>
|
||||||
|
<div class="tool-actions">
|
||||||
|
<button id="btn-generate" class="btn btn-primary" onclick="trGenerateDataset()">Generate Dataset</button>
|
||||||
|
</div>
|
||||||
|
<pre class="output-panel" id="gen-result" style="margin-top:12px;min-height:0"></pre>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="section">
|
||||||
|
<h2>Saved Datasets</h2>
|
||||||
|
<div class="tool-actions">
|
||||||
|
<button class="btn btn-small" onclick="trListDatasets()">Refresh</button>
|
||||||
|
</div>
|
||||||
|
<div id="datasets-list" style="margin-top:12px">
|
||||||
|
<p class="empty-state">Click "Refresh" to load datasets.</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- ==================== TRAINING TAB ==================== -->
|
||||||
|
<div class="tab-content" data-tab-group="trainer" data-tab="training">
|
||||||
|
<div class="section">
|
||||||
|
<h2>LoRA Fine-Tuning</h2>
|
||||||
|
<p style="font-size:0.8rem;color:var(--text-muted);margin-bottom:12px">
|
||||||
|
Fine-tune a base model with LoRA adapters on your generated dataset. Requires a HuggingFace model ID or local path.
|
||||||
|
</p>
|
||||||
|
|
||||||
|
<div class="form-row" style="margin-bottom:8px">
|
||||||
|
<div class="form-group" style="flex:2">
|
||||||
|
<label>Base Model (local path)</label>
|
||||||
|
<div class="input-row">
|
||||||
|
<input type="text" id="tr-base-model" placeholder="Click Browse or paste a local model path...">
|
||||||
|
<button class="btn btn-small" onclick="trOpenBrowser()">Browse</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="form-group" style="flex:1">
|
||||||
|
<label>Dataset</label>
|
||||||
|
<select id="tr-dataset">
|
||||||
|
<option value="">Select dataset...</option>
|
||||||
|
</select>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- File Browser Modal -->
|
||||||
|
<div id="model-browser-overlay" style="display:none;position:fixed;inset:0;background:rgba(0,0,0,0.7);z-index:1000;display:none;align-items:center;justify-content:center">
|
||||||
|
<div style="background:var(--bg-secondary);border:1px solid var(--border);border-radius:8px;width:90%;max-width:700px;max-height:80vh;display:flex;flex-direction:column;padding:16px">
|
||||||
|
<div style="display:flex;align-items:center;gap:8px;margin-bottom:12px">
|
||||||
|
<h3 style="margin:0;flex:1">Select Base Model</h3>
|
||||||
|
<button class="btn btn-small" onclick="trCloseBrowser()">Close</button>
|
||||||
|
</div>
|
||||||
|
<div style="display:flex;align-items:center;gap:8px;margin-bottom:8px">
|
||||||
|
<button class="btn btn-small" onclick="trBrowseUp()" title="Parent directory">↑ Up</button>
|
||||||
|
<code id="browser-path" style="flex:1;font-size:0.78rem;color:var(--text-muted);overflow:hidden;text-overflow:ellipsis;white-space:nowrap"></code>
|
||||||
|
</div>
|
||||||
|
<div id="browser-entries" style="flex:1;overflow-y:auto;border:1px solid var(--border);border-radius:4px;font-size:0.82rem"></div>
|
||||||
|
<p style="font-size:0.72rem;color:var(--text-muted);margin:8px 0 0">Select a directory containing config.json + model weights, or navigate to find your model.</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<details style="margin-bottom:12px">
|
||||||
|
<summary style="cursor:pointer;font-size:0.85rem;color:var(--accent)">Advanced Settings</summary>
|
||||||
|
<div class="form-row" style="margin-top:8px">
|
||||||
|
<div class="form-group">
|
||||||
|
<label>LoRA Rank (r)</label>
|
||||||
|
<input type="number" id="tr-lora-r" value="16" min="4" max="128" style="max-width:80px">
|
||||||
|
</div>
|
||||||
|
<div class="form-group">
|
||||||
|
<label>LoRA Alpha</label>
|
||||||
|
<input type="number" id="tr-lora-alpha" value="32" min="8" max="256" style="max-width:80px">
|
||||||
|
</div>
|
||||||
|
<div class="form-group">
|
||||||
|
<label>Dropout</label>
|
||||||
|
<input type="number" id="tr-dropout" value="0.05" step="0.01" min="0" max="0.5" style="max-width:80px">
|
||||||
|
</div>
|
||||||
|
<div class="form-group">
|
||||||
|
<label>Epochs</label>
|
||||||
|
<input type="number" id="tr-epochs" value="3" min="1" max="20" style="max-width:80px">
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="form-row">
|
||||||
|
<div class="form-group">
|
||||||
|
<label>Batch Size</label>
|
||||||
|
<input type="number" id="tr-batch" value="4" min="1" max="32" style="max-width:80px">
|
||||||
|
</div>
|
||||||
|
<div class="form-group">
|
||||||
|
<label>Grad Accum Steps</label>
|
||||||
|
<input type="number" id="tr-grad-accum" value="4" min="1" max="32" style="max-width:80px">
|
||||||
|
</div>
|
||||||
|
<div class="form-group">
|
||||||
|
<label>Learning Rate</label>
|
||||||
|
<input type="text" id="tr-lr" value="2e-4" style="max-width:100px">
|
||||||
|
</div>
|
||||||
|
<div class="form-group">
|
||||||
|
<label>Max Seq Length</label>
|
||||||
|
<input type="number" id="tr-seq-len" value="2048" min="512" max="8192" style="max-width:100px">
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div style="margin-top:4px">
|
||||||
|
<label><input type="checkbox" id="tr-4bit" checked> Use 4-bit quantization (QLoRA)</label><br>
|
||||||
|
<label><input type="checkbox" id="tr-unsloth"> Use Unsloth (faster, if installed)</label>
|
||||||
|
</div>
|
||||||
|
</details>
|
||||||
|
|
||||||
|
<div class="tool-actions">
|
||||||
|
<button id="btn-train-start" class="btn btn-primary" onclick="trStartTraining()">Start Training</button>
|
||||||
|
<button id="btn-train-stop" class="btn btn-danger" onclick="trStopTraining()" style="display:none">Stop Training</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div id="training-status" style="margin-top:12px">
|
||||||
|
<div class="stats-grid" style="grid-template-columns:repeat(auto-fit,minmax(120px,1fr))">
|
||||||
|
<div class="stat-card">
|
||||||
|
<div class="stat-label">Phase</div>
|
||||||
|
<div class="stat-value small" id="tr-phase">Idle</div>
|
||||||
|
</div>
|
||||||
|
<div class="stat-card">
|
||||||
|
<div class="stat-label">Progress</div>
|
||||||
|
<div class="stat-value small" id="tr-progress">0%</div>
|
||||||
|
</div>
|
||||||
|
<div class="stat-card">
|
||||||
|
<div class="stat-label">Status</div>
|
||||||
|
<div class="stat-value small" id="tr-message">Ready</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<h4 style="margin-top:16px">Training Log</h4>
|
||||||
|
<pre class="output-panel scrollable" id="training-log" style="max-height:300px;font-size:0.75rem">Training log will appear here...</pre>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- ==================== CONVERT TAB ==================== -->
|
||||||
|
<div class="tab-content" data-tab-group="trainer" data-tab="convert">
|
||||||
|
<div class="section">
|
||||||
|
<h2>GGUF Conversion</h2>
|
||||||
|
<p style="font-size:0.8rem;color:var(--text-muted);margin-bottom:12px">
|
||||||
|
Merge a trained LoRA adapter with its base model and convert to GGUF format for local inference.
|
||||||
|
</p>
|
||||||
|
|
||||||
|
<h4>Saved LoRA Adapters</h4>
|
||||||
|
<div class="tool-actions">
|
||||||
|
<button class="btn btn-small" onclick="trListAdapters()">Refresh Adapters</button>
|
||||||
|
</div>
|
||||||
|
<div id="adapters-list" style="margin-top:8px">
|
||||||
|
<p class="empty-state">Click "Refresh" to load adapters.</p>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<h4 style="margin-top:16px">Convert to GGUF</h4>
|
||||||
|
<div class="form-row" style="margin-bottom:12px">
|
||||||
|
<div class="form-group" style="flex:2">
|
||||||
|
<label>Adapter Path</label>
|
||||||
|
<input type="text" id="cv-adapter" placeholder="Path to LoRA adapter directory">
|
||||||
|
</div>
|
||||||
|
<div class="form-group">
|
||||||
|
<label>Output Name</label>
|
||||||
|
<input type="text" id="cv-output" placeholder="e.g., Hal_v3" value="Hal_v3">
|
||||||
|
</div>
|
||||||
|
<div class="form-group">
|
||||||
|
<label>Quantization</label>
|
||||||
|
<select id="cv-quant">
|
||||||
|
<option value="Q4_K_M">Q4_K_M (small, fast)</option>
|
||||||
|
<option value="Q5_K_M" selected>Q5_K_M (balanced)</option>
|
||||||
|
<option value="Q6_K">Q6_K (quality)</option>
|
||||||
|
<option value="Q8_0">Q8_0 (high quality)</option>
|
||||||
|
<option value="F16">F16 (unquantized)</option>
|
||||||
|
</select>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="tool-actions">
|
||||||
|
<button id="btn-convert" class="btn btn-primary" onclick="trConvert()">Merge & Convert</button>
|
||||||
|
</div>
|
||||||
|
<pre class="output-panel" id="convert-result" style="margin-top:12px;min-height:0"></pre>
|
||||||
|
|
||||||
|
<h4 style="margin-top:16px">Available GGUF Models</h4>
|
||||||
|
<div class="tool-actions">
|
||||||
|
<button class="btn btn-small" onclick="trListModels()">Refresh Models</button>
|
||||||
|
</div>
|
||||||
|
<div id="models-list" style="margin-top:8px">
|
||||||
|
<p class="empty-state">Click "Refresh" to load models.</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- ==================== EVALUATE TAB ==================== -->
|
||||||
|
<div class="tab-content" data-tab-group="trainer" data-tab="evaluate">
|
||||||
|
<div class="section">
|
||||||
|
<h2>Model Evaluation</h2>
|
||||||
|
<p style="font-size:0.8rem;color:var(--text-muted);margin-bottom:12px">
|
||||||
|
Test a GGUF model with sample prompts to verify it learned the AUTARCH codebase.
|
||||||
|
</p>
|
||||||
|
|
||||||
|
<div class="form-row" style="margin-bottom:12px">
|
||||||
|
<div class="form-group" style="flex:2">
|
||||||
|
<label>Model Path</label>
|
||||||
|
<select id="ev-model">
|
||||||
|
<option value="">Select model...</option>
|
||||||
|
</select>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<h4>Test Prompts</h4>
|
||||||
|
<div id="eval-prompts">
|
||||||
|
<div class="input-row" style="margin-bottom:4px">
|
||||||
|
<input type="text" class="eval-prompt" value="What is AUTARCH?" style="flex:1">
|
||||||
|
</div>
|
||||||
|
<div class="input-row" style="margin-bottom:4px">
|
||||||
|
<input type="text" class="eval-prompt" value="How do I create a new defense module?" style="flex:1">
|
||||||
|
</div>
|
||||||
|
<div class="input-row" style="margin-bottom:4px">
|
||||||
|
<input type="text" class="eval-prompt" value="What module categories does AUTARCH support?" style="flex:1">
|
||||||
|
</div>
|
||||||
|
<div class="input-row" style="margin-bottom:4px">
|
||||||
|
<input type="text" class="eval-prompt" value="Create a module that scans for open ports on localhost." style="flex:1">
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<button class="btn btn-small" onclick="trAddPrompt()" style="margin-top:4px">+ Add Prompt</button>
|
||||||
|
|
||||||
|
<div class="tool-actions" style="margin-top:12px">
|
||||||
|
<button id="btn-evaluate" class="btn btn-primary" onclick="trEvaluate()">Run Evaluation</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div id="eval-results" style="margin-top:12px"></div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<script>
|
||||||
|
/* ── LLM Trainer ── */
|
||||||
|
var _trPollInterval = null;
|
||||||
|
|
||||||
|
function trScanCodebase() {
|
||||||
|
var btn = document.getElementById('btn-scan');
|
||||||
|
setLoading(btn, true);
|
||||||
|
postJSON('/llm-trainer/scan', {}).then(function(data) {
|
||||||
|
setLoading(btn, false);
|
||||||
|
var inv = data.inventory || {};
|
||||||
|
var html = '<table class="data-table" style="font-size:0.82rem"><thead><tr>'
|
||||||
|
+ '<th>Category</th><th>Files</th><th>Lines</th></tr></thead><tbody>';
|
||||||
|
var cats = ['modules','core','routes','templates','configs','other'];
|
||||||
|
cats.forEach(function(cat) {
|
||||||
|
var files = inv[cat] || [];
|
||||||
|
var lines = files.reduce(function(s,f){return s+f.lines},0);
|
||||||
|
html += '<tr><td>' + cat + '</td><td>' + files.length + '</td><td>' + lines.toLocaleString() + '</td></tr>';
|
||||||
|
});
|
||||||
|
html += '<tr style="font-weight:bold"><td>Total</td><td>' + data.total_files
|
||||||
|
+ '</td><td>' + data.total_lines.toLocaleString() + '</td></tr>';
|
||||||
|
html += '</tbody></table>';
|
||||||
|
document.getElementById('scan-result').innerHTML = html;
|
||||||
|
}).catch(function() { setLoading(btn, false); });
|
||||||
|
}
|
||||||
|
|
||||||
|
function trGenerateDataset() {
|
||||||
|
var btn = document.getElementById('btn-generate');
|
||||||
|
setLoading(btn, true);
|
||||||
|
postJSON('/llm-trainer/dataset/generate', {
|
||||||
|
format: document.getElementById('ds-format').value,
|
||||||
|
include_source: document.getElementById('ds-source').checked,
|
||||||
|
include_qa: document.getElementById('ds-qa').checked,
|
||||||
|
include_module_creation: document.getElementById('ds-modules').checked,
|
||||||
|
}).then(function(data) {
|
||||||
|
setLoading(btn, false);
|
||||||
|
if (data.error) {
|
||||||
|
renderOutput('gen-result', 'Error: ' + data.error);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
var lines = [
|
||||||
|
'Dataset Generated!',
|
||||||
|
' File: ' + data.filename,
|
||||||
|
' Samples: ' + data.sample_count,
|
||||||
|
' Format: ' + data.format,
|
||||||
|
' Size: ' + (data.size_bytes / 1024).toFixed(1) + ' KB',
|
||||||
|
'',
|
||||||
|
'Preview (first sample):',
|
||||||
|
];
|
||||||
|
if (data.preview && data.preview.length) {
|
||||||
|
lines.push(JSON.stringify(data.preview[0], null, 2));
|
||||||
|
}
|
||||||
|
renderOutput('gen-result', lines.join('\n'));
|
||||||
|
trListDatasets();
|
||||||
|
trRefreshDatasetSelect();
|
||||||
|
}).catch(function() { setLoading(btn, false); });
|
||||||
|
}
|
||||||
|
|
||||||
|
function trListDatasets() {
|
||||||
|
fetchJSON('/llm-trainer/dataset/list').then(function(data) {
|
||||||
|
var ds = data.datasets || [];
|
||||||
|
var container = document.getElementById('datasets-list');
|
||||||
|
if (!ds.length) {
|
||||||
|
container.innerHTML = '<p class="empty-state">No datasets generated yet.</p>';
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
var html = '<table class="data-table" style="font-size:0.8rem"><thead><tr>'
|
||||||
|
+ '<th>File</th><th>Samples</th><th>Size</th><th>Created</th><th>Actions</th></tr></thead><tbody>';
|
||||||
|
ds.forEach(function(d) {
|
||||||
|
html += '<tr><td>' + escapeHtml(d.filename) + '</td>'
|
||||||
|
+ '<td>' + d.sample_count + '</td>'
|
||||||
|
+ '<td>' + (d.size_bytes/1024).toFixed(1) + ' KB</td>'
|
||||||
|
+ '<td>' + d.created.substring(0,16).replace('T',' ') + '</td>'
|
||||||
|
+ '<td>'
|
||||||
|
+ '<button class="btn btn-small" style="font-size:0.65rem;padding:2px 6px;margin-right:4px" '
|
||||||
|
+ 'onclick="trPreviewDataset(\'' + escapeHtml(d.filename) + '\')">Preview</button>'
|
||||||
|
+ '<button class="btn btn-danger btn-small" style="font-size:0.65rem;padding:2px 6px" '
|
||||||
|
+ 'onclick="trDeleteDataset(\'' + escapeHtml(d.filename) + '\')">Delete</button>'
|
||||||
|
+ '</td></tr>';
|
||||||
|
});
|
||||||
|
html += '</tbody></table>';
|
||||||
|
container.innerHTML = html;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function trPreviewDataset(filename) {
|
||||||
|
postJSON('/llm-trainer/dataset/preview', {filename: filename, limit: 5}).then(function(data) {
|
||||||
|
if (data.error) { alert(data.error); return; }
|
||||||
|
var html = '<h4>Preview: ' + escapeHtml(filename) + '</h4>';
|
||||||
|
(data.samples || []).forEach(function(s, i) {
|
||||||
|
html += '<pre class="output-panel" style="font-size:0.75rem;margin-bottom:4px;max-height:150px;overflow-y:auto">'
|
||||||
|
+ escapeHtml(JSON.stringify(s, null, 2)) + '</pre>';
|
||||||
|
});
|
||||||
|
document.getElementById('datasets-list').innerHTML += html;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function trDeleteDataset(filename) {
|
||||||
|
if (!confirm('Delete ' + filename + '?')) return;
|
||||||
|
postJSON('/llm-trainer/dataset/delete', {filename: filename}).then(function() {
|
||||||
|
trListDatasets();
|
||||||
|
trRefreshDatasetSelect();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function trRefreshDatasetSelect() {
|
||||||
|
fetchJSON('/llm-trainer/dataset/list').then(function(data) {
|
||||||
|
var sel = document.getElementById('tr-dataset');
|
||||||
|
sel.innerHTML = '<option value="">Select dataset...</option>';
|
||||||
|
(data.datasets || []).forEach(function(d) {
|
||||||
|
sel.innerHTML += '<option value="' + escapeHtml(d.path) + '">' + escapeHtml(d.filename)
|
||||||
|
+ ' (' + d.sample_count + ' samples)</option>';
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── Model Browser ── */
|
||||||
|
var _browserCurrentDir = '';
|
||||||
|
var _browserParentDir = '';
|
||||||
|
|
||||||
|
function trOpenBrowser() {
|
||||||
|
document.getElementById('model-browser-overlay').style.display = 'flex';
|
||||||
|
trBrowseDir(''); // starts at models/ directory
|
||||||
|
}
|
||||||
|
|
||||||
|
function trCloseBrowser() {
|
||||||
|
document.getElementById('model-browser-overlay').style.display = 'none';
|
||||||
|
}
|
||||||
|
|
||||||
|
function trBrowseUp() {
|
||||||
|
if (_browserParentDir) trBrowseDir(_browserParentDir);
|
||||||
|
}
|
||||||
|
|
||||||
|
function trBrowseDir(dir) {
|
||||||
|
postJSON('/llm-trainer/browse', {directory: dir}).then(function(data) {
|
||||||
|
if (data.error) {
|
||||||
|
document.getElementById('browser-entries').innerHTML = '<p style="padding:8px;color:var(--danger)">' + escapeHtml(data.error) + '</p>';
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
_browserCurrentDir = data.current_dir || '';
|
||||||
|
_browserParentDir = data.parent_dir || '';
|
||||||
|
document.getElementById('browser-path').textContent = _browserCurrentDir;
|
||||||
|
|
||||||
|
var entries = data.entries || [];
|
||||||
|
if (!entries.length) {
|
||||||
|
document.getElementById('browser-entries').innerHTML = '<p style="padding:8px;color:var(--text-muted)">Empty directory</p>';
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
var html = '<table class="data-table" style="font-size:0.8rem;margin:0"><tbody>';
|
||||||
|
entries.forEach(function(e) {
|
||||||
|
if (e.is_dir) {
|
||||||
|
var icon = e.is_model ? '📦' : '📁';
|
||||||
|
var label = e.is_model
|
||||||
|
? '<span style="color:var(--success);font-weight:bold">' + escapeHtml(e.name) + '</span> <span style="font-size:0.7rem;color:var(--text-muted)">(HF model)</span>'
|
||||||
|
: escapeHtml(e.name);
|
||||||
|
html += '<tr><td style="cursor:pointer" onclick="trBrowseDir(\'' + escapeHtml(e.path) + '\')">' + icon + ' ' + label + '</td>';
|
||||||
|
if (e.is_model) {
|
||||||
|
html += '<td><button class="btn btn-small" style="font-size:0.65rem;padding:2px 8px" onclick="trSelectModel(\'' + escapeHtml(e.path) + '\')">Select</button></td>';
|
||||||
|
} else {
|
||||||
|
html += '<td></td>';
|
||||||
|
}
|
||||||
|
html += '</tr>';
|
||||||
|
} else {
|
||||||
|
var size = e.size_gb !== undefined ? e.size_gb + ' GB' : '';
|
||||||
|
html += '<tr><td>📄 ' + escapeHtml(e.name) + '</td><td style="font-size:0.7rem;color:var(--text-muted)">' + size + '</td></tr>';
|
||||||
|
}
|
||||||
|
});
|
||||||
|
html += '</tbody></table>';
|
||||||
|
document.getElementById('browser-entries').innerHTML = html;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function trSelectModel(path) {
|
||||||
|
document.getElementById('tr-base-model').value = path;
|
||||||
|
trCloseBrowser();
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── Training ── */
|
||||||
|
function trStartTraining() {
|
||||||
|
var baseModel = document.getElementById('tr-base-model').value.trim();
|
||||||
|
var dataset = document.getElementById('tr-dataset').value;
|
||||||
|
if (!baseModel) { alert('Click Browse to select a base model'); return; }
|
||||||
|
if (!dataset) { alert('Please select a dataset'); return; }
|
||||||
|
|
||||||
|
var config = {
|
||||||
|
base_model: baseModel,
|
||||||
|
dataset: dataset,
|
||||||
|
lora_r: parseInt(document.getElementById('tr-lora-r').value),
|
||||||
|
lora_alpha: parseInt(document.getElementById('tr-lora-alpha').value),
|
||||||
|
lora_dropout: parseFloat(document.getElementById('tr-dropout').value),
|
||||||
|
num_epochs: parseInt(document.getElementById('tr-epochs').value),
|
||||||
|
batch_size: parseInt(document.getElementById('tr-batch').value),
|
||||||
|
gradient_accumulation_steps: parseInt(document.getElementById('tr-grad-accum').value),
|
||||||
|
learning_rate: parseFloat(document.getElementById('tr-lr').value),
|
||||||
|
max_seq_length: parseInt(document.getElementById('tr-seq-len').value),
|
||||||
|
use_4bit: document.getElementById('tr-4bit').checked,
|
||||||
|
use_unsloth: document.getElementById('tr-unsloth').checked,
|
||||||
|
};
|
||||||
|
|
||||||
|
var btn = document.getElementById('btn-train-start');
|
||||||
|
setLoading(btn, true);
|
||||||
|
postJSON('/llm-trainer/train/start', config).then(function(data) {
|
||||||
|
setLoading(btn, false);
|
||||||
|
if (data.error) {
|
||||||
|
alert('Training failed: ' + data.error);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
document.getElementById('btn-train-start').style.display = 'none';
|
||||||
|
document.getElementById('btn-train-stop').style.display = '';
|
||||||
|
trStartPolling();
|
||||||
|
}).catch(function() { setLoading(btn, false); });
|
||||||
|
}
|
||||||
|
|
||||||
|
function trStopTraining() {
|
||||||
|
postJSON('/llm-trainer/train/stop', {}).then(function() {
|
||||||
|
trStopPolling();
|
||||||
|
document.getElementById('btn-train-start').style.display = '';
|
||||||
|
document.getElementById('btn-train-stop').style.display = 'none';
|
||||||
|
trUpdateStatus();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function trStartPolling() {
|
||||||
|
if (_trPollInterval) clearInterval(_trPollInterval);
|
||||||
|
_trPollInterval = setInterval(trUpdateStatus, 3000);
|
||||||
|
trUpdateStatus();
|
||||||
|
}
|
||||||
|
|
||||||
|
function trStopPolling() {
|
||||||
|
if (_trPollInterval) { clearInterval(_trPollInterval); _trPollInterval = null; }
|
||||||
|
}
|
||||||
|
|
||||||
|
function trUpdateStatus() {
|
||||||
|
fetchJSON('/llm-trainer/train/status').then(function(data) {
|
||||||
|
document.getElementById('tr-phase').textContent = data.phase || 'idle';
|
||||||
|
document.getElementById('tr-progress').textContent = (data.progress || 0) + '%';
|
||||||
|
document.getElementById('tr-message').textContent = data.message || '';
|
||||||
|
if (data.training_log) {
|
||||||
|
document.getElementById('training-log').textContent = data.training_log;
|
||||||
|
var log = document.getElementById('training-log');
|
||||||
|
log.scrollTop = log.scrollHeight;
|
||||||
|
}
|
||||||
|
if (!data.training_running && _trPollInterval) {
|
||||||
|
trStopPolling();
|
||||||
|
document.getElementById('btn-train-start').style.display = '';
|
||||||
|
document.getElementById('btn-train-stop').style.display = 'none';
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── Conversion ── */
|
||||||
|
function trListAdapters() {
|
||||||
|
fetchJSON('/llm-trainer/adapters').then(function(data) {
|
||||||
|
var adapters = data.adapters || [];
|
||||||
|
var container = document.getElementById('adapters-list');
|
||||||
|
if (!adapters.length) {
|
||||||
|
container.innerHTML = '<p class="empty-state">No LoRA adapters found. Train a model first.</p>';
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
var html = '<table class="data-table" style="font-size:0.8rem"><thead><tr>'
|
||||||
|
+ '<th>Name</th><th>Base Model</th><th>LoRA r</th><th>Action</th></tr></thead><tbody>';
|
||||||
|
adapters.forEach(function(a) {
|
||||||
|
html += '<tr><td>' + escapeHtml(a.name) + '</td>'
|
||||||
|
+ '<td>' + escapeHtml(a.base_model || '—') + '</td>'
|
||||||
|
+ '<td>' + (a.r || '—') + '</td>'
|
||||||
|
+ '<td><button class="btn btn-small" style="font-size:0.65rem;padding:2px 6px" '
|
||||||
|
+ 'onclick="document.getElementById(\'cv-adapter\').value=\'' + escapeHtml(a.path) + '\'">Select</button></td></tr>';
|
||||||
|
});
|
||||||
|
html += '</tbody></table>';
|
||||||
|
container.innerHTML = html;
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function trConvert() {
|
||||||
|
var adapter = document.getElementById('cv-adapter').value.trim();
|
||||||
|
var output = document.getElementById('cv-output').value.trim();
|
||||||
|
var quant = document.getElementById('cv-quant').value;
|
||||||
|
if (!adapter) { alert('Enter or select an adapter path'); return; }
|
||||||
|
if (!output) { alert('Enter an output name'); return; }
|
||||||
|
|
||||||
|
var btn = document.getElementById('btn-convert');
|
||||||
|
setLoading(btn, true);
|
||||||
|
renderOutput('convert-result', 'Merging and converting — this may take a while...');
|
||||||
|
postJSON('/llm-trainer/convert', {
|
||||||
|
adapter_path: adapter,
|
||||||
|
output_name: output,
|
||||||
|
quantization: quant,
|
||||||
|
}).then(function(data) {
|
||||||
|
setLoading(btn, false);
|
||||||
|
if (data.error) {
|
||||||
|
renderOutput('convert-result', 'Error: ' + data.error);
|
||||||
|
} else if (data.partial) {
|
||||||
|
renderOutput('convert-result', data.message);
|
||||||
|
} else {
|
||||||
|
renderOutput('convert-result', 'GGUF model saved!\n Path: ' + data.output_path
|
||||||
|
+ '\n Size: ' + (data.size_bytes / (1024*1024*1024)).toFixed(2) + ' GB');
|
||||||
|
trListModels();
|
||||||
|
}
|
||||||
|
}).catch(function() { setLoading(btn, false); });
|
||||||
|
}
|
||||||
|
|
||||||
|
function trListModels() {
|
||||||
|
fetchJSON('/llm-trainer/models').then(function(data) {
|
||||||
|
var models = data.models || [];
|
||||||
|
var container = document.getElementById('models-list');
|
||||||
|
if (!models.length) {
|
||||||
|
container.innerHTML = '<p class="empty-state">No GGUF models found.</p>';
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
var html = '<table class="data-table" style="font-size:0.8rem"><thead><tr>'
|
||||||
|
+ '<th>Name</th><th>Size</th><th>Modified</th></tr></thead><tbody>';
|
||||||
|
models.forEach(function(m) {
|
||||||
|
html += '<tr><td>' + escapeHtml(m.filename) + '</td>'
|
||||||
|
+ '<td>' + m.size_gb + ' GB</td>'
|
||||||
|
+ '<td>' + m.modified.substring(0,16).replace('T',' ') + '</td></tr>';
|
||||||
|
});
|
||||||
|
html += '</tbody></table>';
|
||||||
|
container.innerHTML = html;
|
||||||
|
|
||||||
|
// Update evaluate model select
|
||||||
|
var sel = document.getElementById('ev-model');
|
||||||
|
sel.innerHTML = '<option value="">Select model...</option>';
|
||||||
|
models.forEach(function(m) {
|
||||||
|
sel.innerHTML += '<option value="' + escapeHtml(m.path) + '">' + escapeHtml(m.filename)
|
||||||
|
+ ' (' + m.size_gb + ' GB)</option>';
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── Evaluate ── */
|
||||||
|
function trAddPrompt() {
|
||||||
|
var div = document.createElement('div');
|
||||||
|
div.className = 'input-row';
|
||||||
|
div.style.marginBottom = '4px';
|
||||||
|
div.innerHTML = '<input type="text" class="eval-prompt" placeholder="Enter test prompt..." style="flex:1">'
|
||||||
|
+ '<button class="btn btn-small" style="font-size:0.65rem;padding:2px 6px;margin-left:4px" '
|
||||||
|
+ 'onclick="this.parentElement.remove()">X</button>';
|
||||||
|
document.getElementById('eval-prompts').appendChild(div);
|
||||||
|
}
|
||||||
|
|
||||||
|
function trEvaluate() {
|
||||||
|
var model = document.getElementById('ev-model').value;
|
||||||
|
if (!model) { alert('Select a model to evaluate'); return; }
|
||||||
|
|
||||||
|
var prompts = [];
|
||||||
|
document.querySelectorAll('.eval-prompt').forEach(function(el) {
|
||||||
|
if (el.value.trim()) prompts.push(el.value.trim());
|
||||||
|
});
|
||||||
|
if (!prompts.length) { alert('Add at least one test prompt'); return; }
|
||||||
|
|
||||||
|
var btn = document.getElementById('btn-evaluate');
|
||||||
|
setLoading(btn, true);
|
||||||
|
document.getElementById('eval-results').innerHTML = '<p style="color:var(--text-muted)">Running evaluation — loading model and generating responses...</p>';
|
||||||
|
|
||||||
|
postJSON('/llm-trainer/evaluate', {model_path: model, prompts: prompts}).then(function(data) {
|
||||||
|
setLoading(btn, false);
|
||||||
|
if (data.error) {
|
||||||
|
document.getElementById('eval-results').innerHTML = '<p style="color:var(--danger)">Error: ' + escapeHtml(data.error) + '</p>';
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
var html = '';
|
||||||
|
(data.results || []).forEach(function(r, i) {
|
||||||
|
html += '<div style="margin-bottom:12px;border:1px solid var(--border);border-radius:6px;padding:12px">'
|
||||||
|
+ '<div style="font-weight:bold;margin-bottom:6px;color:var(--accent)">Prompt: ' + escapeHtml(r.prompt) + '</div>'
|
||||||
|
+ '<pre class="output-panel scrollable" style="max-height:200px;font-size:0.78rem;margin:0">'
|
||||||
|
+ escapeHtml(r.response) + '</pre>'
|
||||||
|
+ '<div style="font-size:0.7rem;color:var(--text-muted);margin-top:4px">Response length: ' + r.length + ' chars</div>'
|
||||||
|
+ '</div>';
|
||||||
|
});
|
||||||
|
document.getElementById('eval-results').innerHTML = html;
|
||||||
|
}).catch(function() { setLoading(btn, false); });
|
||||||
|
}
|
||||||
|
|
||||||
|
/* ── Init ── */
|
||||||
|
document.addEventListener('DOMContentLoaded', function() {
|
||||||
|
trRefreshDatasetSelect();
|
||||||
|
trListModels();
|
||||||
|
});
|
||||||
|
</script>
|
||||||
|
{% endblock %}
|
||||||
184
web/templates/system_deps.html
Normal file
184
web/templates/system_deps.html
Normal file
@ -0,0 +1,184 @@
|
|||||||
|
{% extends "base.html" %}
|
||||||
|
{% block title %}Dependencies - AUTARCH{% endblock %}
|
||||||
|
|
||||||
|
{% block content %}
|
||||||
|
<div class="page-header" style="display:flex;align-items:center;gap:1rem;flex-wrap:wrap">
|
||||||
|
<div>
|
||||||
|
<h1>System Dependencies</h1>
|
||||||
|
<p style="margin:0;font-size:0.85rem;color:var(--text-secondary)">
|
||||||
|
Check and install Python packages required by AUTARCH modules.
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- System Info -->
|
||||||
|
<div class="section" id="sys-info-section" style="display:none">
|
||||||
|
<h2>System Info</h2>
|
||||||
|
<div class="stats-grid" style="grid-template-columns:repeat(auto-fit,minmax(160px,1fr))">
|
||||||
|
<div class="stat-card">
|
||||||
|
<div class="stat-label">Python</div>
|
||||||
|
<div class="stat-value small" id="sys-python">--</div>
|
||||||
|
</div>
|
||||||
|
<div class="stat-card">
|
||||||
|
<div class="stat-label">Platform</div>
|
||||||
|
<div class="stat-value small" id="sys-platform">--</div>
|
||||||
|
</div>
|
||||||
|
<div class="stat-card">
|
||||||
|
<div class="stat-label">CUDA GPU</div>
|
||||||
|
<div class="stat-value small" id="sys-gpu">--</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Dependency Groups -->
|
||||||
|
<div class="section">
|
||||||
|
<div class="tool-actions">
|
||||||
|
<button id="btn-check-all" class="btn btn-primary" onclick="depsCheckAll()">Check All Dependencies</button>
|
||||||
|
<span id="deps-status" style="font-size:0.8rem;color:var(--text-muted);margin-left:12px"></span>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div id="deps-groups" style="margin-top:16px">
|
||||||
|
<p class="empty-state">Click "Check All Dependencies" to scan installed packages.</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<!-- Quick Install -->
|
||||||
|
<div class="section">
|
||||||
|
<h2>Install Packages</h2>
|
||||||
|
<p style="font-size:0.8rem;color:var(--text-muted);margin-bottom:12px">
|
||||||
|
Install individual packages or preset groups.
|
||||||
|
</p>
|
||||||
|
<div class="form-row" style="margin-bottom:12px">
|
||||||
|
<div class="form-group" style="flex:2">
|
||||||
|
<div class="input-row">
|
||||||
|
<input type="text" id="install-pkg" placeholder="Package name (e.g., scapy, torch, peft)">
|
||||||
|
<button class="btn btn-primary btn-small" onclick="depsInstallOne()">Install</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div style="display:flex;gap:8px;flex-wrap:wrap">
|
||||||
|
<button class="btn btn-small" onclick="depsInstallGroup('core')">Install Core</button>
|
||||||
|
<button class="btn btn-small" onclick="depsInstallGroup('llm')">Install LLM</button>
|
||||||
|
<button class="btn btn-small" onclick="depsInstallGroup('training')">Install Training</button>
|
||||||
|
<button class="btn btn-small" onclick="depsInstallGroup('network')">Install Network</button>
|
||||||
|
<button class="btn btn-small" onclick="depsInstallGroup('hardware')">Install Hardware</button>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<pre class="output-panel" id="install-result" style="margin-top:12px;min-height:0;display:none"></pre>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<script>
|
||||||
|
var _depsGroupPkgs = {
|
||||||
|
core: ['flask', 'jinja2', 'requests', 'cryptography'],
|
||||||
|
llm: ['llama-cpp-python', 'transformers', 'anthropic'],
|
||||||
|
training: ['torch', 'peft', 'datasets', 'trl', 'accelerate', 'bitsandbytes'],
|
||||||
|
network: ['scapy', 'pyshark', 'miniupnpc', 'msgpack', 'paramiko'],
|
||||||
|
hardware: ['pyserial', 'esptool', 'adb-shell'],
|
||||||
|
};
|
||||||
|
|
||||||
|
var _depsGroupLabels = {
|
||||||
|
core: 'Core Framework',
|
||||||
|
llm: 'LLM Inference',
|
||||||
|
training: 'LLM Training (LoRA)',
|
||||||
|
network: 'Network & Packet Capture',
|
||||||
|
hardware: 'Hardware & Serial',
|
||||||
|
};
|
||||||
|
|
||||||
|
function depsCheckAll() {
|
||||||
|
var btn = document.getElementById('btn-check-all');
|
||||||
|
setLoading(btn, true);
|
||||||
|
document.getElementById('deps-status').textContent = 'Scanning...';
|
||||||
|
postJSON('/settings/deps/check', {}).then(function(data) {
|
||||||
|
setLoading(btn, false);
|
||||||
|
document.getElementById('deps-status').textContent = '';
|
||||||
|
|
||||||
|
// System info
|
||||||
|
if (data.python) {
|
||||||
|
document.getElementById('sys-python').textContent = data.python.version || '--';
|
||||||
|
document.getElementById('sys-platform').textContent = data.python.platform || '--';
|
||||||
|
document.getElementById('sys-info-section').style.display = '';
|
||||||
|
}
|
||||||
|
if (data.gpu) {
|
||||||
|
document.getElementById('sys-gpu').textContent = data.gpu.cuda_available
|
||||||
|
? (data.gpu.device || 'Available') + ' (CUDA ' + (data.gpu.cuda_version || '') + ')'
|
||||||
|
: 'Not available';
|
||||||
|
document.getElementById('sys-gpu').style.color = data.gpu.cuda_available ? 'var(--success)' : 'var(--text-muted)';
|
||||||
|
}
|
||||||
|
|
||||||
|
// Render groups
|
||||||
|
var html = '';
|
||||||
|
var groupOrder = ['core', 'llm', 'training', 'network', 'hardware'];
|
||||||
|
groupOrder.forEach(function(group) {
|
||||||
|
var pkgs = data[group];
|
||||||
|
if (!pkgs) return;
|
||||||
|
var installed = 0, total = 0;
|
||||||
|
for (var k in pkgs) { total++; if (pkgs[k].installed) installed++; }
|
||||||
|
|
||||||
|
var statusColor = installed === total ? 'var(--success)' : installed > 0 ? 'var(--warning)' : 'var(--danger)';
|
||||||
|
html += '<div style="margin-bottom:16px">';
|
||||||
|
html += '<h3 style="margin-bottom:6px">' + (_depsGroupLabels[group] || group)
|
||||||
|
+ ' <span style="font-size:0.75rem;font-weight:normal;color:' + statusColor + '">'
|
||||||
|
+ installed + '/' + total + ' installed</span></h3>';
|
||||||
|
html += '<table class="data-table" style="font-size:0.82rem"><thead><tr>'
|
||||||
|
+ '<th>Package</th><th>Status</th><th>Version</th><th>Action</th></tr></thead><tbody>';
|
||||||
|
for (var name in pkgs) {
|
||||||
|
var info = pkgs[name];
|
||||||
|
html += '<tr><td>' + escapeHtml(name) + '</td>'
|
||||||
|
+ '<td><span class="status-dot ' + (info.installed ? 'active' : 'inactive') + '"></span>'
|
||||||
|
+ (info.installed ? 'Installed' : 'Missing') + '</td>'
|
||||||
|
+ '<td>' + (info.version || '—') + '</td>'
|
||||||
|
+ '<td>' + (info.installed ? '' : '<button class="btn btn-small" style="font-size:0.65rem;padding:2px 6px" '
|
||||||
|
+ 'onclick="depsInstallPkg(\'' + escapeHtml(name) + '\',this)">Install</button>') + '</td></tr>';
|
||||||
|
}
|
||||||
|
html += '</tbody></table></div>';
|
||||||
|
});
|
||||||
|
|
||||||
|
document.getElementById('deps-groups').innerHTML = html;
|
||||||
|
}).catch(function() {
|
||||||
|
setLoading(btn, false);
|
||||||
|
document.getElementById('deps-status').textContent = 'Error checking dependencies';
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function depsInstallPkg(pkg, btn) {
|
||||||
|
if (btn) setLoading(btn, true);
|
||||||
|
var out = document.getElementById('install-result');
|
||||||
|
out.style.display = '';
|
||||||
|
renderOutput('install-result', 'Installing ' + pkg + '...');
|
||||||
|
postJSON('/settings/deps/install', {packages: [pkg]}).then(function(data) {
|
||||||
|
if (btn) setLoading(btn, false);
|
||||||
|
var results = data.results || [];
|
||||||
|
var lines = results.map(function(r) {
|
||||||
|
return r.package + ': ' + (r.success ? 'OK' : 'FAILED — ' + r.output);
|
||||||
|
});
|
||||||
|
renderOutput('install-result', lines.join('\n'));
|
||||||
|
// Refresh deps list
|
||||||
|
depsCheckAll();
|
||||||
|
}).catch(function() { if (btn) setLoading(btn, false); });
|
||||||
|
}
|
||||||
|
|
||||||
|
function depsInstallOne() {
|
||||||
|
var pkg = document.getElementById('install-pkg').value.trim();
|
||||||
|
if (!pkg) return;
|
||||||
|
depsInstallPkg(pkg, null);
|
||||||
|
document.getElementById('install-pkg').value = '';
|
||||||
|
}
|
||||||
|
|
||||||
|
function depsInstallGroup(group) {
|
||||||
|
var pkgs = _depsGroupPkgs[group];
|
||||||
|
if (!pkgs || !pkgs.length) return;
|
||||||
|
var out = document.getElementById('install-result');
|
||||||
|
out.style.display = '';
|
||||||
|
renderOutput('install-result', 'Installing ' + (_depsGroupLabels[group] || group) + ' packages: ' + pkgs.join(', ') + '...');
|
||||||
|
postJSON('/settings/deps/install', {packages: pkgs}).then(function(data) {
|
||||||
|
var results = data.results || [];
|
||||||
|
var lines = results.map(function(r) {
|
||||||
|
return r.package + ': ' + (r.success ? 'OK' : 'FAILED — ' + r.output);
|
||||||
|
});
|
||||||
|
renderOutput('install-result', lines.join('\n'));
|
||||||
|
depsCheckAll();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
</script>
|
||||||
|
{% endblock %}
|
||||||
Loading…
x
Reference in New Issue
Block a user