rsspipe/docs/006_ai_rules_and_autonomous_triage.md

# RSSPipe AI Rules & Autonomous Triage Specification (`006_ai_rules_and_autonomous_triage.md`)

This document defines the architecture, data structures, security guardrails, and execution models for integrating AI capabilities into RSSPipe's event-driven rules engine.

---

## 1. Core Principles

1. **AI as Standard Rule Actions**: AI is not a separate silo; it is integrated directly as actions within the rule pipeline (`Trigger ➜ Conditions ➜ Actions`).
2. **Dual AI Modes**:
   - **Mode A: Variant Enrichment (`ai_enrich`)**: Extracts structured summaries, key takeaways, sentiment, or translations into `entry_variants` (`variant_key: "ai_summary"`).
   - **Mode B: Autonomous Triage Agent (`ai_agent_action`)**: Evaluates content and proposes actions restricted strictly to a user-defined allowed action pool (e.g., auto-archiving, starring, tagging).
3. **Flexible Prompt Hierarchy**:
   - **Global Prompt Library**: Saved reusable templates configured in `Settings > AI & Integrations`.
   - **Rule-Level Custom Prompts**: Individual rules can override or write feed-specific prompts (e.g. "Summarize for Kubernetes engineers" vs "Summarize for general news").
4. **Security & Cost Guardrails**:
   - Authentication-enforced settings and testing endpoints.
   - Rate-limited and token-capped test pings (`max_tokens: 15`).
   - Strict action validator discarding any hallucinated or unallowed agent actions.

---

## 2. Architecture & Data Flow

```mermaid
flowchart TD
    subgraph TriggerLayer ["1. Trigger & Evaluation"]
        Event[Event Ingested / Retroactive Apply] --> Evaluator{Conditions Match?}
    end

    subgraph ActionExecution ["2. Action Routing & AI Dispatch"]
        Evaluator -- Yes --> ActionRouter[Action Router]
        ActionRouter --> StandardActions[Standard Actions: Star, Read, Scrape, Set Tag]
        ActionRouter --> EnrichAction["AI Variant Enrichment (ai_enrich)"]
        ActionRouter --> AgentAction["AI Autonomous Triage (ai_agent_action)"]
    end

    subgraph AIClient ["3. LLM Gateway (internal/ai)"]
        Settings[(system_settings Table)] --> Client[OpenAI-Compatible Client]
        EnrichAction --> Client
        AgentAction --> Client
        Client --> RemoteLLM["LLM Provider (OpenAI, Gemini, DeepSeek, Ollama)"]
    end

    subgraph PersistenceLayer ["4. Persistence & Chaining"]
        EnrichAction --> VariantSave["Save to entry_variants (e.g. ai_summary)"]
        VariantSave --> ChainedEvent["Publish EventVariantUpdated"]
        AgentAction --> ActionValidator["Strict Action Pool Validator"]
        ActionValidator -- Allowed Only --> DispatchPool["Worker Pool Dispatch"]
    end
```

---

## 3. Configuration & Settings Schema

### 3.1 Database: `system_settings` Table
```sql
CREATE TABLE IF NOT EXISTS system_settings (
    key TEXT PRIMARY KEY,
    value TEXT NOT NULL,
    updated_at DATETIME NOT NULL
);
```

### 3.2 Stored Settings Keys
| Key | Type | Description | Example |
|---|---|---|---|
| `ai_enabled` | `bool` | Master toggle for AI processing | `true` |
| `ai_provider` | `string` | Provider preset identifier | `openai`, `deepseek`, `gemini`, `ollama`, `custom` |
| `ai_base_url` | `string` | OpenAI-compatible endpoint URL | `https://api.openai.com/v1`, `http://localhost:11434/v1` |
| `ai_api_key` | `string` | Provider API Key (masked in GET responses) | `sk-...` |
| `ai_model` | `string` | Model identifier | `gpt-4o-mini`, `deepseek-chat`, `gemini-2.0-flash`, `llama3.2` |
| `ai_temperature` | `float` | Sampling temperature | `0.3` |
| `ai_max_tokens` | `int` | Maximum response tokens | `1000` |
| `ai_prompts` | `json` | Array of saved prompt template objects | `[{"id": "tech_brief", "name": "Tech Brief", "prompt": "..."}]` |

---

## 4. AI Action Specifications

### 4.1 Action `ai_enrich` (Variant Enrichment)
* **Goal**: Generate text/markdown and save as an `EntryVariant`.
* **Parameters**:
  ```json
  {
    "type": "ai_enrich",
    "params": {
      "variant_key": "ai_summary",
      "content_type": "text",
      "prompt": "You are an expert technical editor. Summarize the following article in 3 concise bullet points followed by 1 key takeaway.\n\nTitle: {{title}}\nContent:\n{{content}}"
    }
  }
  ```
* **Variables Available for Prompt Interpolation**:
  - `{{title}}`: Article headline
  - `{{content}}`: Article body (uses full-text / clean RSS content)
  - `{{feed_title}}`: Source feed name
  - `{{author}}`: Article author
  - `{{url}}`: Original article URL
  - `{{published_at}}`: Publication date

### 4.2 Action `ai_agent_action` (Autonomous Triage)
* **Goal**: Analyze the article against a policy and execute allowed actions automatically.
* **Parameters**:
  ```json
  {
    "type": "ai_agent_action",
    "params": {
      "prompt": "Evaluate this engineering post. If it contains a critical zero-day or high-impact CVE, tag as 'security-urgent' and star it. If it is promotional marketing fluff, mark as read.",
      "allowed_actions": [
        "mark_read",
        "mark_starred",
        "set_variant:tags:security-urgent",
        "set_variant:tags:newsletter"
      ]
    }
  }
  ```
* **Enforced Output Schema**:
  ```json
  {
    "reasoning": "Post discusses a critical remote execution CVE in Kubernetes ingress controllers.",
    "actions": [
      { "type": "mark_starred" },
      { "type": "set_variant", "params": { "variant_key": "tags", "value": "security-urgent" } }
    ]
  }
  ```
* **Security Validation**: The backend strictly matches proposed actions against `allowed_actions`. Any unallowed action is logged as a rejection and discarded.

---

## 5. Security & DDoS Protection

1. **Authentication Requirement**: All AI configuration and testing endpoints (`/api/settings/ai`, `/api/settings/ai/test`) require valid JWT bearer tokens with administrative privileges.
2. **Test Endpoint Token Cap & Rate Limiting**:
   - `POST /api/settings/ai/test` forces `max_tokens: 15` and a lightweight prompt ("Reply with OK in 2 words") to prevent token exhaustion.
   - Enforced rate limit: Maximum 5 connection tests per minute per IP/user session.
3. **API Key Masking**: The `GET /api/settings/ai` endpoint never returns full API keys in plaintext (returns `sk-••••••••abcd`).