Files
hackingtool/tests/test_ai_command.py
T
Hardik Zinzuvadiya 2482cc9fd0 feat: AI operator console — 215 curated tools, AI layer, /find discovery (v2.0.0 rework)
Squashed rework of hackingtool from a tool launcher into an AI-guided operator
console for authorized security testing. 93 commits collapsed into this one;
the pre-rework tree is tagged v2.0.0.

CATALOG & ENGINE
- Data-driven YAML catalog (21 categories, 215 live tools + 59 archived) with a
  registry/overlay loader and a fixed 66-tag taxonomy (63 in use). Adding a tool
  is one YAML entry, not edits across the codebase.
- Engine honesty: real exit codes, truthful install success/failure, reuse-first
  skip, EOF-safe prompts, command audit logging.
- Safe installs: sha256-required safe-fetch (killed `curl | bash` in feroxbuster,
  Caido and Sliver), list-form subprocess only, no forced sudo.

AI LAYER (bring-your-own-key or local model; degrades offline, never fabricates)
- AI1 intent -> tools; AI2 tool+goal -> command, curated-first with a grounded
  fallback; AI3 findings summary and engagement report; AI4 per-finding impact
  and remediation. Prompt-injection hardened per OWASP LLM01.
- /goal plans an objective and runs it one step at a time, showing every command
  before it runs, with a plan.json + run.log audit trail.

/find TOOL DISCOVERY (this branch's headline feature)
- Suggests real GitHub projects when the catalog has no tool for a need.
  Deterministic: zero model calls, structured API fields only, suggest-only —
  it never clones, installs or runs anything.
- A charter filter refuses destructive/DoS/jamming/mass-targeting/evasion asks
  before any network I/O, while a defensive-intent guard keeps blue-team and
  DFIR phrasing ("detect a SYN flood in a pcap") from being false-refused.
- Query rewriting proved to be the dominant quality lever (the first design
  measured 29% precision with no results on 5 of 8 needs): a curated 41-row
  intent table maps plain English to canonical jargon plus a GitHub topic, and
  a two-arm search unions topic coverage with jargon precision.
- Explainable additive ranking: log-flattened stars, license/age/language,
  trusted-author bonus derived from owners we already ship, docs-repo demotion
  by name, staleness as a soft demotion rather than a filter (a hard cutoff
  would delete THC-Hydan and John the Ripper), and a relevance term weighting
  curated topics above free-text description.
- Optional no-scope GitHub token purely as a rate-limit lever (10 -> 30 req/min);
  it reaches only an Authorization header, never a cache key, log or output.
- `[a]` saves a pick to ~/.hackingtool/found.yaml as a structurally inert entry,
  and the loader strips executable keys from user catalogs at read time so a
  hand-edited file cannot become a runnable command.

CONSOLE & PACKAGING
- REPL with a / command palette, @ tool mentions, tag filters, history and
  completion; background tmux panes; settings and first-run scaffolding.
- src-layout package with catalog and pipelines as package data, console entry
  point, Docker image, signed releases with SBOM and build provenance.
- Health docs (SECURITY, CONTRIBUTING, CHANGELOG, CODE_OF_CONDUCT), a CI gate
  (ruff + pytest + catalog/taxonomy conformance) and a pre-push hook.
- README rewritten with a section index, the tool catalog split into
  docs/TOOLS.md and a step-by-step docs/HOW-TO-USE.md.

278 tests passing; scripts/check.sh green.
2026-07-27 00:18:34 +05:30

46 lines
1.7 KiB
Python

"""AI2 build_command: curated-first, AI-gap fill, anti-fabrication binary guard."""
import hackingtool.ai_command as ai_command
USAGE = [
("crack MD5 with a wordlist", "hashcat -m 0 -a 0 hash.txt rockyou.txt"),
("show already-cracked results", "hashcat -m 0 hash.txt --show"),
]
def test_curated_match_returns_exact_command():
src, cmd = ai_command.build_command("Hashcat", USAGE, "crack an md5 with a wordlist")
assert src == "curated"
assert cmd == "hashcat -m 0 -a 0 hash.txt rockyou.txt"
def test_foreign_binary_dropped():
# AI reply naming another tool is fabrication -> dropped.
assert ai_command._parse_command("nmap -sV target", USAGE) is None
def test_no_command_sentinel():
assert ai_command._parse_command("NO-COMMAND", USAGE) is None
assert ai_command._parse_command(None, USAGE) is None
def test_code_fence_stripped_and_known_binary_kept():
assert ai_command._parse_command("```sh\nhashcat -m 100 hash.txt\n```", USAGE) == \
"hashcat -m 100 hash.txt"
def test_empty_usage_or_goal_returns_none():
assert ai_command.build_command("Hashcat", [], "anything") is None
assert ai_command.build_command("Hashcat", USAGE, " ") is None
def test_ai_leg_used_only_when_no_curated_match(monkeypatch):
monkeypatch.setattr(ai_command, "ask", lambda p: "hashcat -m 1800 shadow.txt")
src, cmd = ai_command.build_command("Hashcat", USAGE, "crack a sha512crypt shadow entry")
assert src == "ai"
assert cmd == "hashcat -m 1800 shadow.txt"
def test_ai_leg_none_when_unreachable(monkeypatch):
monkeypatch.setattr(ai_command, "ask", lambda p: None)
assert ai_command.build_command("Hashcat", USAGE, "some novel uncurated goal xyz") is None