feat: implement AI integration with backend configuration and rule-based enrichment actions
This commit is contained in:
@@ -132,3 +132,13 @@ This document records planned features, UX enhancements, and architectural exten
|
||||
- Retention triggers (e.g. delete unstarred articles older than 90 days, or limit feeds to latest 500 posts).
|
||||
- Background garbage collection cleaning associated `entry_variants` and on-disk audio blobs (`./data/blobs/{entry_id}/`).
|
||||
|
||||
---
|
||||
|
||||
## 14. Outbound Webhook Dispatcher (P4)
|
||||
|
||||
- **Goal**: Allow rule-triggered notifications and payload delivery to external platforms.
|
||||
- **Scope**:
|
||||
- Outbound webhook action in rule engine supporting Telegram, Discord, Slack, and generic JSON HTTP POST.
|
||||
- Template engine with interpolation of article variables (`title`, `feed_title`, `url`, `published_at`, `ai_summary`, `tags`).
|
||||
- Retry mechanism with exponential backoff on delivery failure.
|
||||
|
||||
|
||||
@@ -0,0 +1,140 @@
|
||||
# RSSPipe AI Rules & Autonomous Triage Specification (`006_ai_rules_and_autonomous_triage.md`)
|
||||
|
||||
This document defines the architecture, data structures, security guardrails, and execution models for integrating AI capabilities into RSSPipe's event-driven rules engine.
|
||||
|
||||
---
|
||||
|
||||
## 1. Core Principles
|
||||
|
||||
1. **AI as Standard Rule Actions**: AI is not a separate silo; it is integrated directly as actions within the rule pipeline (`Trigger ➜ Conditions ➜ Actions`).
|
||||
2. **Dual AI Modes**:
|
||||
- **Mode A: Variant Enrichment (`ai_enrich`)**: Extracts structured summaries, key takeaways, sentiment, or translations into `entry_variants` (`variant_key: "ai_summary"`).
|
||||
- **Mode B: Autonomous Triage Agent (`ai_agent_action`)**: Evaluates content and proposes actions restricted strictly to a user-defined allowed action pool (e.g., auto-archiving, starring, tagging).
|
||||
3. **Flexible Prompt Hierarchy**:
|
||||
- **Global Prompt Library**: Saved reusable templates configured in `Settings > AI & Integrations`.
|
||||
- **Rule-Level Custom Prompts**: Individual rules can override or write feed-specific prompts (e.g. "Summarize for Kubernetes engineers" vs "Summarize for general news").
|
||||
4. **Security & Cost Guardrails**:
|
||||
- Authentication-enforced settings and testing endpoints.
|
||||
- Rate-limited and token-capped test pings (`max_tokens: 15`).
|
||||
- Strict action validator discarding any hallucinated or unallowed agent actions.
|
||||
|
||||
---
|
||||
|
||||
## 2. Architecture & Data Flow
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
subgraph TriggerLayer ["1. Trigger & Evaluation"]
|
||||
Event[Event Ingested / Retroactive Apply] --> Evaluator{Conditions Match?}
|
||||
end
|
||||
|
||||
subgraph ActionExecution ["2. Action Routing & AI Dispatch"]
|
||||
Evaluator -- Yes --> ActionRouter[Action Router]
|
||||
ActionRouter --> StandardActions[Standard Actions: Star, Read, Scrape, Set Tag]
|
||||
ActionRouter --> EnrichAction["AI Variant Enrichment (ai_enrich)"]
|
||||
ActionRouter --> AgentAction["AI Autonomous Triage (ai_agent_action)"]
|
||||
end
|
||||
|
||||
subgraph AIClient ["3. LLM Gateway (internal/ai)"]
|
||||
Settings[(system_settings Table)] --> Client[OpenAI-Compatible Client]
|
||||
EnrichAction --> Client
|
||||
AgentAction --> Client
|
||||
Client --> RemoteLLM["LLM Provider (OpenAI, Gemini, DeepSeek, Ollama)"]
|
||||
end
|
||||
|
||||
subgraph PersistenceLayer ["4. Persistence & Chaining"]
|
||||
EnrichAction --> VariantSave["Save to entry_variants (e.g. ai_summary)"]
|
||||
VariantSave --> ChainedEvent["Publish EventVariantUpdated"]
|
||||
AgentAction --> ActionValidator["Strict Action Pool Validator"]
|
||||
ActionValidator -- Allowed Only --> DispatchPool["Worker Pool Dispatch"]
|
||||
end
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## 3. Configuration & Settings Schema
|
||||
|
||||
### 3.1 Database: `system_settings` Table
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS system_settings (
|
||||
key TEXT PRIMARY KEY,
|
||||
value TEXT NOT NULL,
|
||||
updated_at DATETIME NOT NULL
|
||||
);
|
||||
```
|
||||
|
||||
### 3.2 Stored Settings Keys
|
||||
| Key | Type | Description | Example |
|
||||
|---|---|---|---|
|
||||
| `ai_enabled` | `bool` | Master toggle for AI processing | `true` |
|
||||
| `ai_provider` | `string` | Provider preset identifier | `openai`, `deepseek`, `gemini`, `ollama`, `custom` |
|
||||
| `ai_base_url` | `string` | OpenAI-compatible endpoint URL | `https://api.openai.com/v1`, `http://localhost:11434/v1` |
|
||||
| `ai_api_key` | `string` | Provider API Key (masked in GET responses) | `sk-...` |
|
||||
| `ai_model` | `string` | Model identifier | `gpt-4o-mini`, `deepseek-chat`, `gemini-2.0-flash`, `llama3.2` |
|
||||
| `ai_temperature` | `float` | Sampling temperature | `0.3` |
|
||||
| `ai_max_tokens` | `int` | Maximum response tokens | `1000` |
|
||||
| `ai_prompts` | `json` | Array of saved prompt template objects | `[{"id": "tech_brief", "name": "Tech Brief", "prompt": "..."}]` |
|
||||
|
||||
---
|
||||
|
||||
## 4. AI Action Specifications
|
||||
|
||||
### 4.1 Action `ai_enrich` (Variant Enrichment)
|
||||
* **Goal**: Generate text/markdown and save as an `EntryVariant`.
|
||||
* **Parameters**:
|
||||
```json
|
||||
{
|
||||
"type": "ai_enrich",
|
||||
"params": {
|
||||
"variant_key": "ai_summary",
|
||||
"content_type": "text",
|
||||
"prompt": "You are an expert technical editor. Summarize the following article in 3 concise bullet points followed by 1 key takeaway.\n\nTitle: {{title}}\nContent:\n{{content}}"
|
||||
}
|
||||
}
|
||||
```
|
||||
* **Variables Available for Prompt Interpolation**:
|
||||
- `{{title}}`: Article headline
|
||||
- `{{content}}`: Article body (uses full-text / clean RSS content)
|
||||
- `{{feed_title}}`: Source feed name
|
||||
- `{{author}}`: Article author
|
||||
- `{{url}}`: Original article URL
|
||||
- `{{published_at}}`: Publication date
|
||||
|
||||
### 4.2 Action `ai_agent_action` (Autonomous Triage)
|
||||
* **Goal**: Analyze the article against a policy and execute allowed actions automatically.
|
||||
* **Parameters**:
|
||||
```json
|
||||
{
|
||||
"type": "ai_agent_action",
|
||||
"params": {
|
||||
"prompt": "Evaluate this engineering post. If it contains a critical zero-day or high-impact CVE, tag as 'security-urgent' and star it. If it is promotional marketing fluff, mark as read.",
|
||||
"allowed_actions": [
|
||||
"mark_read",
|
||||
"mark_starred",
|
||||
"set_variant:tags:security-urgent",
|
||||
"set_variant:tags:newsletter"
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
* **Enforced Output Schema**:
|
||||
```json
|
||||
{
|
||||
"reasoning": "Post discusses a critical remote execution CVE in Kubernetes ingress controllers.",
|
||||
"actions": [
|
||||
{ "type": "mark_starred" },
|
||||
{ "type": "set_variant", "params": { "variant_key": "tags", "value": "security-urgent" } }
|
||||
]
|
||||
}
|
||||
```
|
||||
* **Security Validation**: The backend strictly matches proposed actions against `allowed_actions`. Any unallowed action is logged as a rejection and discarded.
|
||||
|
||||
---
|
||||
|
||||
## 5. Security & DDoS Protection
|
||||
|
||||
1. **Authentication Requirement**: All AI configuration and testing endpoints (`/api/settings/ai`, `/api/settings/ai/test`) require valid JWT bearer tokens with administrative privileges.
|
||||
2. **Test Endpoint Token Cap & Rate Limiting**:
|
||||
- `POST /api/settings/ai/test` forces `max_tokens: 15` and a lightweight prompt ("Reply with OK in 2 words") to prevent token exhaustion.
|
||||
- Enforced rate limit: Maximum 5 connection tests per minute per IP/user session.
|
||||
3. **API Key Masking**: The `GET /api/settings/ai` endpoint never returns full API keys in plaintext (returns `sk-••••••••abcd`).
|
||||
Reference in New Issue
Block a user