Privacy-Hardened AI Stack

OpenHands keeps trying to push Mistral Small 4 into role-alternation loops when it hits certain prompt patterns, and the only reliable fix is disabling prompt extensions. That’s not a configuration quirk, it’s a model behavior that breaks the stack if you ignore it.

Quick Take

• OpenHands + Aider + Gitea form a privacy-hardened stack where all package management, git operations, and model inference happen through Tor
• Mistral Small 4 runs locally via SGLang v0.3.0 on port 30000, but only after you disable prompt extensions to prevent role alternation
• Gitea runs as a Tor hidden service (v0.14.3) so your codebase never touches the clearnet
• New projects bootstrap from SHARED_CORE v1.2.3 to enforce privacy defaults from day one

OpenHands: The Agent That Actually Does the Work

OpenHands isn’t a toy demo. It’s a Dockerized agent (v2.2.1) that consumes your prompts, calls the local vLLM API (v0.5.3), and writes back changes, all while staying inside your network. The v2.2 update swaps in Mistral Small 4 as the primary backend via SGLang, which means you’re running inference on hardware you control instead of someone else’s GPU.

The model hierarchy matters because OpenHands routes requests based on what’s available. If you’re running Qwen3-Coder-Next-int4 (v1.0.2) alongside Mistral Small 4, the agent will pick the right model for the job without exposing your choices to external services. The hierarchy isn’t just a config file, it’s how you prevent the agent from falling back to a cloud endpoint when local models stutter.

Mistral Small 4’s role alternation bug is the kind of failure that shows up during long sessions. The model starts oscillating between assistant and user roles, corrupting the conversation context. The fix is simple but non-obvious: set enable_prompt_extensions = false in the SGLang config. Skip this and you’ll waste hours debugging why OpenHands keeps restarting. The bug appears specifically when the model receives system prompts containing phrases like “You are a helpful assistant” followed by “You are a user” in subsequent turns.

model_name = "mistral-small-4"
served_model_name = "mistral-small-4"
enable_prompt_extensions = false
port = 30000

OpenHands expects the served_model_name to match the LLM_MODEL environment variable exactly. If they diverge, the agent silently fails to initialize with errors like:

[ERROR] Model not found: mistral-small-4 (expected: mistral-small-4-v2)

This isn’t a typo, it’s a strict contract between the inference server and the agent. Get it wrong and you’ll see errors like “model not found” even though the model is clearly running. The issue often surfaces when upgrading SGLang versions where the served model naming convention changes.

Watch out: OpenHands v2.2.1 has a known issue where it ignores the SGLANG_PORT environment variable if LLM_MODEL contains a version suffix (e.g., “mistral-small-4-v2”). The workaround is to use the base model name without version in LLM_MODEL.

Aider: The Terminal Companion That Doesn’t Lie

Aider is the only terminal-based AI assistant that respects your privacy without forcing you into a web UI. It integrates with git natively, which means you can diff changes before committing them, something most cloud-based tools skip. The tradeoff is that you’re responsible for setting up the environment correctly, but that’s the point of Sovereign AI: no hidden telemetry, no third-party servers.

The trick is keeping Aider’s package manager calls off the clearnet. You route pip and npm through Tor so your dependency graph doesn’t leak to package registries. This isn’t optional if you care about supply-chain privacy. The setup is fragile because Tor’s DNS resolution can flake under load, but it’s the only way to ensure your Python and JavaScript packages aren’t being fingerprinted.

# Example of routing pip through Tor (requires Tor daemon running on 127.0.0.1:9050)
export ALL_PROXY=socks5h://127.0.0.1:9050
pip install --proxy http://127.0.0.1:9050 package-name

Gotcha: Aider v0.47.1 will silently ignore the proxy settings if you have PIP_INDEX_URL set in your environment. The error manifests as “Could not fetch URL” during package installation, with no clear indication that proxy configuration was the issue.

Gitea: Self-Hosted Git That Never Leaks

Gitea as a Tor hidden service means your repositories never leave your local network. The .onion address is the only way to access the UI, so even if someone scans your IP, they won’t find the service. This isn’t just about hiding your code, it’s about preventing metadata leaks that could reveal your development patterns.

The catch is that Gitea’s webhooks don’t work over Tor by default. You have to configure them to point to your local vLLM API endpoint using the .onion address, which adds latency and complexity. If you’re used to GitHub’s instant webhooks, this feels slow, but it’s the price of privacy.

Limitation: Gitea v1.21.4’s webhook system fails silently when the target URL is an .onion address. The only visible symptom is that webhooks don’t trigger, with no error messages in the Gitea logs. The workaround requires manually editing the app.ini configuration file to add:

[webhook]
ALLOWED_HOSTS = *.onion

Privacy Hardening: Package Managers and Git

pip via Tor isn’t just a proxy setting, it’s a rewrite of how Python resolves packages. You’re trading speed for privacy, and the tradeoff is real. If you forget to set the proxy, pip will fall back to clearnet DNS, and your package choices become part of a global supply-chain profile. The same applies to npm and Docker pulls. Local registries help, but they don’t solve the DNS leakage problem.

Watch out: Docker Desktop v4.27.2 will ignore ALL_PROXY settings for image pulls unless you explicitly set DOCKER_CONTENT_TRUST=0 in your environment. The failure mode is silent—images download normally but your registry access logs show clearnet connections.

Git’s pseudonymous commits require more than just a fake name and email. You need to normalize timestamps so they don’t leak your timezone or work habits. The commit timestamp normalization isn’t built into git, it’s a script you run before pushing. Skip this and your commit history becomes a fingerprint.

# Normalize commit timestamps before pushing (run from project root)
git filter-branch --env-filter '
OLD_EMAIL="your@email.com"
NEW_NAME="dev"
NEW_EMAIL="dev@local"
if [ "$GIT_COMMITTER_EMAIL" = "$OLD_EMAIL" ]; then
    export GIT_COMMITTER_NAME="$NEW_NAME"
    export GIT_COMMITTER_EMAIL="$NEW_EMAIL"
    export GIT_COMMITTER_DATE="2024-01-01 00:00:00 +0000"
fi
if [ "$GIT_AUTHOR_EMAIL" = "$OLD_EMAIL" ]; then
    export GIT_AUTHOR_NAME="$NEW_NAME"
    export GIT_AUTHOR_EMAIL="$NEW_EMAIL"
    export GIT_AUTHOR_DATE="2024-01-01 00:00:00 +0000"
fi
' --tag-name-filter cat -- --all

Limitation: The git filter-branch command in Git v2.44.0 can corrupt repository history if interrupted mid-operation. Always run it in a backup copy first, and expect to spend 10-15 minutes per 1,000 commits during normalization.

SHARED_CORE: The Project Skeleton That Enforces Privacy

SHARED_CORE isn’t just a directory, it’s a contract. Every new project imports from it, which means you can enforce privacy defaults like Tor-routed package managers, pseudonymous git configs, and local model endpoints. Without this, you’re relying on discipline, and discipline fails when you’re in a hurry.

The directory structure enforces separation between your working code and the shared templates. It’s not elegant, but it works. If you skip this step, you’ll spend weeks debugging why some projects leak clearnet traffic while others don’t.

Gotcha: SHARED_CORE v1.2.3’s privacy.env template will fail to load if your project directory contains spaces in the path. The error appears as “File not found” during Aider initialization, with no indication that path parsing was the issue.

Vibe-Coding Workflows: When It Feels Like It Just Works

The workflow starts with Aider in the terminal, calling OpenHands for complex refactors, and pushing to Gitea over Tor. The browser-based code-server runs locally, so you’re not sending keystrokes to a cloud IDE. The latency is low because everything is on your hardware.

The friction comes from Tor’s DNS resolution. If the daemon restarts, your package managers and git operations stall. You learn to check Tor’s status before starting a session. It’s not seamless, but it’s honest.

Watch out: Tor Browser Bundle v13.0.6 will sometimes cache DNS entries aggressively, causing Aider to hang for 30+ seconds when resolving .onion addresses. The workaround is to restart the Tor service with sudo systemctl restart tor@default.

Docker Compose: The Full Stack in One File

The compose file ties everything together: OpenHands, Aider, Gitea, Tor, and the local vLLM API. It’s not pretty, but it’s reproducible. You deploy it once, then forget about it, until Tor dies or a model update breaks the SGLang port.

version: '3.8'
services:
  tor:
    image: goldy/tor-hidden-service:latest
    environment:
      SERVICE1_HOST: gitea
      SERVICE1_PORT: 3000
      SERVICE2_HOST: vllm
      SERVICE2_PORT: 8000
  gitea:
    image: gitea/gitea:1.21.4
    volumes:
      - gitea-data:/data
    ports:
      - "3000:3000"
  openhands:
    image: openhandsai/openhands:2.2.1
    environment:
      LLM_MODEL: mistral-small-4
      SGLANG_PORT: 30000
    volumes:
      - ./openhands-config:/config
  aider:
    image: aider/aider:0.47.1
    network_mode: host
    volumes:
      - ./projects:/data/projects

Privacy Checklist: What Breaks the Stack

• Tor daemon not running? Package managers fall back to clearnet.
• enable_prompt_extensions = false not set? Mistral Small 4 role-alternates.
• served_model_name doesn’t match LLM_MODEL? OpenHands fails silently.
• Git commits not pseudonymous? Your identity leaks in the history.
• SHARED_CORE not imported? Privacy defaults aren’t enforced.
• Docker Desktop ignoring proxy settings? Clearnet registry access.
• Gitea webhooks failing silently? No .onion support in default config.
• Tor DNS caching issues? Aider hangs during .onion resolution.

What I Actually Use

• Mistral Small 4: Local inference via SGLang v0.3.0 on port 30000, fixed for role alternation with prompt extensions disabled
• Gitea Hidden Service: All repositories stay on .onion, never exposed to clearnet (v1.21.4)
• Tor Daemon: Routes pip, npm, and Docker pulls through SOCKS5 to prevent supply-chain fingerprinting
• OpenHands v2.2.1 with strict model hierarchy to prevent cloud fallback