AI, RAG & Enterprise AI Brain

10. AI / RAG Pipeline

AI Service Architecture

External API calls (all rate-limited and cached):

• OpenAI: $0.0001/1K tokens (embedding), $0.002/1K (GPT)
• Anthropic: $0.008/1K tokens (Claude Sonnet)
• Cohere: $0.001/1K tokens (Rerank API)

AI Pod Configuration (GPU vs CPU)

AI endpoints can run on CPU (small models) or GPU (large models):

CPU Tier (standard nodes):

• Embedding generation (text-embedding-3-small via API)
• Chunking (pure CPU, ~100K chunks/second)
• Reranking (heuristic, no model)
• LLM via API (network call, no local GPU needed)

GPU Tier (optional, for local models):

• Local embedding model (e.g., all-MiniLM-L6-v2)
• Local LLM (e.g., Llama 4 8B via Ollama)
• Only cost-effective at very high volume (>1M tokens/day)

At 100k users → Use API-based AI (OpenAI/Anthropic/Cohere) At 10M users → Hybrid AI (Local GPU cluster + API routing) — see Enterprise AI Brain section

---

Enterprise AI Brain & Digital Workforce

Enterprise Only: This section covers Stackhouse's enterprise AI pivot — transforming from an AI-enhanced database to an AI Orchestrator for large organizations.

1. Enterprise AI Pivot

As Stackhouse scales into an Enterprise context, simple RAG and vector queries are not enough. Large organizations require autonomous problem-solving capabilities deeply integrated into their proprietary data silos.

The Enterprise AI Brain shifts Stackhouse from an AI-enhanced database to an AI Orchestrator.

2. The Native 47 SaaS Connectors

Stackhouse natively embeds 47 enterprise connectors via the stackhouse_connectors module. These eliminate the need to pipe data through external ETLs (like Fivetran or Airbyte) before it reaches the AI.

Connector Architecture:

• Universal REST Module: Dynamic OAuth handling via oauth_vault.rs
• Bidirectional: Connectors don't just pull for RAG; they also push actions (e.g., "Create Jira Ticket", "Update Salesforce Lead")
• Cost Scaling: At 10M users, hosting a centralized Fivetran cluster for thousands of tenants becomes prohibitive. Stackhouse executes lightweight native polling/webhooks per tenant directly into the unified data model.

Example Enterprise Connectors:

• CRM: Salesforce, HubSpot, Pipedrive
• Support: Zendesk, Intercom, Freshdesk
• DevOps: Jira, GitHub, GitLab, Linear
• Communication: Slack, Teams, Discord
• Storage: Google Drive, Dropbox, Box, SharePoint
• Finance: Stripe, QuickBooks, Xero
• Analytics: Mixpanel, Amplitude, Segment

3. Autonomous Agents (Digital Workforce)

Stackhouse provides specialized domain experts (agents) that can be dispatched by users or workflows.

• Agent Registry: A compiled list of specialized agents (Support, Marketing, DevOps, Legal, Finance, etc.)
• Tasks & Execution: Built into the core runtime (agent_registry.rs), tasks are spawned as async Rust tasks interacting with the Connectors and the Brain
• Human-in-the-loop: POST /v1/agent/task/:id/approve allows agents to pause execution before performing destructive actions (e.g., dropping SaaS data, launching a mass email campaign) until a human approves

Sample Agent Types:

4. Integration with PostgreSQL & Qdrant

While the RAG pipeline leverages Qdrant for semantic search and PostgreSQL for structured data, the Enterprise AI Brain acts as the overarching intelligence layer.

Example Workflow:

1. User asks: "Analyze churn rate based on recent Stripe cancellations and matching Zendesk tickets."
2. StackhouseBrain: Understands the intent
3. Execution:
   • Dispatches the Data Analyst Agent
   • Agent triggers the Stripe connector (pulls churn events)
   • Agent triggers the Zendesk connector (pulls tickets)
   • Joins data in an ephemeral SQLite / LSM table
   • Computes analysis via local Llama 3 or GPT-4
   • Returns result to user

5. Enterprise AI Architecture at 10M Scale

6. AI Cost Control at Enterprise Scale

Scale Economics:

• At 100K users: API-only AI (~$500-2,000/month)
• At 10M users: Hybrid AI (Local GPU cluster + API) (~$14,000/month GPU + API costs)
• Break-even: ~5M tokens/day justifies local GPU investment

---