Architecture Reference

graph TB
    subgraph "External Services"
        Ollama[Ollama<br/>LLM & Embeddings]
        OpenAI[OpenAI API<br/>LLM & Embeddings]
        Gemini[Google Gemini<br/>LLM & Embeddings]
    end
 
    subgraph "Communication Platforms"
        Slack[Slack]
        Email[Email/SMTP]
        UI[Web UI]
    end
 
    subgraph "Otto Core System"
        subgraph "API Layer"
            FastAPI[FastAPI Backend<br/>Port 8000]
        end
 
        subgraph "Agent Layer"
            ParentGraph[Parent Agent<br/>LangGraph StateGraph]
            SubagentGraph[Subagent<br/>LangGraph StateGraph]
            Preprocess[Preprocess Intent<br/>Skill + Tool Matching]
            AgentNode[Agent Node<br/>LLM Invocation]
            ActionNode[Action Node<br/>Tool Execution]
            Router[Router<br/>Conditional Edge]
        end
 
        subgraph "Worker Layer"
            ARQWorker[ARQ Worker Process]
            InteractiveQueue[arq:interactive<br/>Task Queue]
            SubagentQueue[arq:subagent<br/>Subagent Queue]
            CronJobs[Cron Jobs<br/>Scheduled Tasks]
        end
 
        subgraph "Tool Ecosystem"
            StaticTools[Static Tools<br/>13 built-in]
            MCPTools[MCP Dynamic Tools<br/>Runtime loaded]
            MCPManager[MCP Manager<br/>Multi-Server Client]
        end
 
        subgraph "Memory & Skills"
            VectorStore[Vector Memory Store<br/>SQLite + Embeddings]
            SkillsMgr[Skills Manager<br/>Matching + Injection]
            Compactor[Conversation Compactor<br/>LLM Summarization]
        end
 
        subgraph "Communication Layer"
            CommManager[Communication Manager]
            SlackProvider[Slack Provider]
            EmailProvider[Email Provider]
            UIProvider[UI Provider]
        end
 
        subgraph "Data Layer"
            Redis[(Redis<br/>Queues, Checkpoints, Cache)]
            SQLite[(SQLite<br/>Tasks, Users, Logs)]
            MemoryDB[(Memory DB<br/>Vector Store)]
            FileStore[File Storage<br/>Local / GCS / Drive]
        end
    end
 
    FastAPI --> InteractiveQueue
    InteractiveQueue --> ARQWorker
    SubagentQueue --> ARQWorker
    ARQWorker --> ParentGraph
    ARQWorker --> SubagentGraph
 
    ParentGraph --> Preprocess
    Preprocess --> AgentNode
    AgentNode --> Router
    Router --> ActionNode
    ActionNode --> AgentNode
 
    AgentNode -->|LLM Calls| OpenAI
    AgentNode -->|LLM Calls| Gemini
    AgentNode -->|LLM Calls| Ollama
 
    ActionNode --> StaticTools
    ActionNode --> MCPTools
    MCPManager --> MCPTools
 
    AgentNode --> VectorStore
    Preprocess --> SkillsMgr
    AgentNode --> Compactor
 
    CommManager --> SlackProvider
    CommManager --> EmailProvider
    CommManager --> UIProvider
    SlackProvider --> Slack
    EmailProvider --> Email
    UIProvider --> UI
 
    ARQWorker --> Redis
    ARQWorker --> SQLite
    VectorStore --> MemoryDB
    ActionNode --> FileStore
 
    classDef api fill:#4A90E2,stroke:#2E5C8A,stroke-width:2px,color:#fff
    classDef agent fill:#7B68EE,stroke:#4B0082,stroke-width:2px,color:#fff
    classDef worker fill:#FF8C00,stroke:#CC7000,stroke-width:2px,color:#fff
    classDef tool fill:#50C878,stroke:#2F7C4F,stroke-width:2px,color:#fff
    classDef data fill:#FFA07A,stroke:#CD5C5C,stroke-width:2px,color:#fff
    classDef external fill:#DDA0DD,stroke:#9370DB,stroke-width:2px,color:#fff
    classDef comm fill:#FF6B6B,stroke:#C92A2A,stroke-width:2px,color:#fff
 
    class FastAPI api
    class ParentGraph,SubagentGraph,Preprocess,AgentNode,ActionNode,Router agent
    class ARQWorker,InteractiveQueue,SubagentQueue,CronJobs worker
    class StaticTools,MCPTools,MCPManager tool
    class Redis,SQLite,MemoryDB,FileStore data
    class Ollama,OpenAI,Gemini,Slack,Email,UI external
    class CommManager,SlackProvider,EmailProvider,UIProvider comm

Entry
  |
  v
[preprocess_intent]  --->  [agent]  <---->  [action]
                              |                  |
                              |   (conditional)  |
                              v                  |
                             END <---------------+

Entry
  |
  v
[agent]  <---->  [action]
   |                |
   v                |
  END <-------------+

1. User submits task (API POST /chat, Slack @mention, or webhook)
     |
2. API creates task record (status: pending)
   Enqueues `process_agent_task` to arq:interactive
     |
3. ARQ worker picks up job
   Lazy-initializes heavy components (LLM, MCP, embeddings) if needed
     |
4. run_agent_in_background():
   - Generate checkpoint thread_id ("task-{task_id}")
   - Save checkpoint_thread_id to DB
   - Set task status: running
   - Create initial HumanMessage from user input
     |
5. Graph execution:
   |
   |-- [preprocess_intent_node]
   |     Match skills (always)
   |     Extract steps + match tools (if ENABLE_INTENT_PREPROCESSING=true)
   |
   |-- [call_model] <-----------+
   |     Assemble system prompt  |
   |     Inject context blocks:  |
   |       - Project context     |
   |       - Skills instructions |
   |       - Team info           |
   |       - Vector memories     |
   |       - Persona (SOUL.md)   |
   |     Compact messages if >12 |
   |     Invoke LLM with tools   |
   |                             |
   |-- [should_call_tool_or_pause]
   |     |                       |
   |     |-- tool_calls? -----> [tool_node_with_logging]
   |     |                       Execute tools
   |     |                       Truncate large results
   |     |                       Detect file attachments
   |     |                       Handle ask_human interrupt
   |     |                       Auto-park on subagent dispatch
   |     |                       Return tool results --------+
   |     |
   |     |-- no tool_calls + content --> complete_task_with_result()
   |     |     Save TaskOutput
   |     |     Set status: completed, progress: 100
   |     |     Auto-save memory
   |     |     Send completion notification (with attachments)
   |     |     Clean up checkpoint
   |     |     --> END
   |     |
   |     |-- hard iteration limit --> force complete
   |
6. Interrupt handling (if applicable):
   |
   |-- ask_human: status → waiting_input
   |     Human responds via API or Slack
   |     resume_agent_task enqueued
   |     Graph resumes from Redis checkpoint
   |
   |-- subagent dispatch: status → waiting_subagents
         Subagent(s) run in parallel on arq:interactive
         Each has own graph, MCP session, iteration budget
         On completion: _check_and_resume_parent()
         All siblings terminal? Collect results, resume parent

Agent completes task or calls ask_human
     |
     v
CommunicationManager.send_notification()
     |
     |--> SlackProvider  --> Slack API  --> User's Slack DM / channel
     |--> EmailProvider  --> SMTP       --> User's email
     |--> UIProvider     --> DB         --> Frontend polls /users/{id}/notifications

User uploads file via POST /upload-file
     |
     v
Validate extension (allowlist)
Check storage quota (STORAGE_QUOTA_GB)
     |
     v
Save to user sandbox: {LOCAL_STORAGE_PATH}/{user_id}/{filename}
     |
     v
Agent accesses via read_file / list_directory tools (sandboxed)

Task Created
  |
  v
run_agent_in_background()
  |-- thread_id = "task-{task_id}"
  |-- Save checkpoint_thread_id to task record
  |-- agent.astream() with AsyncRedisSaver checkpointer
  |
  |-- Every graph node execution: full AgentState serialized to Redis
  |
  |-- [GraphInterrupt: ask_human or subagent park]
  |     |
  |     v
  |   Task paused
  |   Status: waiting_input or waiting_subagents
  |   last_tool_call_id saved to DB
  |   Checkpoint preserved in Redis
  |     |
  |     v  (human responds or subagents complete)
  |   resume_agent_in_background()
  |     |-- Read checkpoint_thread_id from DB
  |     |-- Read last_tool_call_id from DB
  |     |-- Construct ToolMessage(s) with response
  |     |-- agent.astream() with same thread_id
  |     |-- LangGraph restores full state from Redis
  |     |-- ReAct loop continues
  |
  v
Task completed
  |-- cleanup_task_checkpoint(): delete all Redis keys for thread
  |-- Clear checkpoint_thread_id from DB

Parent agent calls dispatch_subagent(task_description, context, agent_type, max_iterations)
  |
  v
New task record created (task_type: "subtask", parent_task_id: parent)
  |
  v
process_subagent_task_v2 enqueued to arq:interactive
  |
  v
Subtask ID appended to _pending_dispatch_ids (module-level list)
  |
  v
After all tools execute, tool_node_with_logging calls take_pending_dispatches()
  |
  |--> Fast path: All subtasks already terminal? Collect results inline. Parent continues.
  |
  |--> Slow path (common): Update parent status to waiting_subagents.
       Save dispatch tool_call_ids as JSON array in last_tool_call_id.
       Raise GraphInterrupt. Parent paused.

                              +--------------------+
                              |   USER REQUEST     |
                              |  "Create a gamma   |
                              |   presentation"    |
                              +---------+----------+
                                        |
                                        v
                         +-----------------------------+
                         | PREPROCESS_INTENT_NODE      |
                         |                             |
                         |  1. Extract Work Steps      |
                         |     (LLM structured output) |
                         |     "research AI topic"     |
                         |     "create presentation"   |
                         |                             |
                         |  2. Semantic Tool Matching   |
                         |     (if ENABLE_INTENT_      |
                         |      PREPROCESSING=true)    |
                         |     Embed steps + tools     |
                         |     Cosine similarity       |
                         |     Top-k per step          |
                         |                             |
                         |  3. Skill Matching (always) |
                         |     a. Keyword boost:       |
                         |        tag in query? +0.1   |
                         |     b. Semantic matching:   |
                         |        embed description    |
                         |        cosine similarity    |
                         |     c. Threshold: 0.35      |
                         +---------+-------------------+
                                   |
                                   v
                         +-----------------------------+
                         | AGENT NODE (call_model)     |
                         |                             |
                         | 4. Skill Tool Resolution    |
                         |    For each matched skill:  |
                         |      allowed_tools list     |
                         |      Exact match first      |
                         |      Then partial match     |
                         |      Case insensitive       |
                         |                             |
                         | 5. Union Logic              |
                         |    Skills-required tools    |
                         |    (guaranteed) + semantic  |
                         |    matches (deduplicated)   |
                         |                             |
                         | 6. Prompt Injection         |
                         |    Skill instructions       |
                         |    appended to system prompt|
                         |                             |
                         | 7. LLM Invocation           |
                         |    bind_tools(final set)    |
                         +-----------------------------+

Skills-Required Tools (guaranteed inclusion)
  + Semantic Matched Tools (added if not duplicate)
  + Static Tools (always included)
  = Final available tools bound to LLM

---
name: Otto
role: Marketing Assistant
specialization: Digital Marketing and Content Creation
tone: Professional but friendly
---
 
Additional instructions and personality details as free-form markdown.