> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/nearai/ironclaw/llms.txt
> Use this file to discover all available pages before exploring further.

# Workspace & Memory

> Persistent memory system with hybrid search and filesystem-like API

## Overview

The workspace is IronClaw's persistent memory system. It provides a filesystem-like API for storing notes, logs, and context, backed by PostgreSQL with full-text and semantic search.

## Filesystem Metaphor

The workspace looks and feels like a file tree:

```text theme={null}
workspace/
├── README.md              <- Root index
├── MEMORY.md              <- Curated long-term memory
├── HEARTBEAT.md           <- Periodic checklist
├── IDENTITY.md            <- Agent personality
├── SOUL.md                <- Core values
├── AGENTS.md              <- Operational instructions
├── USER.md                <- User context
├── context/               <- Additional context
│   ├── vision.md
│   └── priorities.md
├── daily/                 <- Daily logs
│   ├── 2026-03-01.md
│   ├── 2026-03-02.md
│   └── 2026-03-03.md
└── projects/              <- Arbitrary structure
    └── alpha/
        ├── README.md
        └── notes.md
```

<Info>
  Paths are **virtual**. There's no actual filesystem - everything is stored in PostgreSQL with path-based indexing.
</Info>

## Core Operations

<CodeGroup>
  ```rust Read theme={null}
  let doc = workspace.read("MEMORY.md").await?;
  println!("{}", doc.content);
  ```

  ```rust Write theme={null}
  workspace.write(
      "projects/alpha/notes.md",
      "# Project Alpha\n\nDeployment notes..."
  ).await?;
  ```

  ```rust Append theme={null}
  workspace.append(
      "MEMORY.md",
      "\n- User prefers concise responses"
  ).await?;
  ```

  ```rust List Directory theme={null}
  let entries = workspace.list("projects/").await?;
  for entry in entries {
      if entry.is_directory {
          println!("📁 {}/", entry.name());
      } else {
          println!("📄 {}", entry.name());
      }
  }
  ```

  ```rust Delete theme={null}
  workspace.delete("projects/old/notes.md").await?;
  ```

  ```rust Search theme={null}
  let results = workspace.search(
      "deployment issues",
      limit: 5
  ).await?;
  for result in results {
      println!("{}: {}", result.path, result.content);
  }
  ```
</CodeGroup>

## Identity Files

Core files that shape agent behavior.

<AccordionGroup>
  <Accordion title="MEMORY.md - Long-term curated memory">
    **Purpose**: Important facts, decisions, and preferences worth remembering across sessions.

    **Usage**:

    * Agent appends new learnings during conversations
    * User curates periodically (remove stale, consolidate duplicates)
    * Loaded into system prompt for every session
    * Keep concise - this affects token usage

    **Example**:

    ```markdown theme={null}
    # Memory

    ## User Preferences
    - Prefers concise, direct responses
    - Works in PST timezone
    - Uses Python and Rust primarily

    ## Important Context
    - Working on Project Alpha (internal tool)
    - Deploy schedule: Tuesdays/Thursdays
    - Team lead: Sarah Chen

    ## Decisions
    - 2026-03-01: Switched from SQLite to PostgreSQL for production
    - 2026-02-15: Agreed to use Trunk for asset bundling
    ```
  </Accordion>

  <Accordion title="IDENTITY.md - Agent personality">
    **Purpose**: Define agent's name, vibe, and personality.

    **Usage**:

    * Loaded into system prompt
    * Agent evolves this over time
    * User can edit directly

    **Example**:

    ```markdown theme={null}
    # Identity

    - **Name:** Atlas
    - **Vibe:** Calm, thoughtful, competent
    - **Emoji:** 🗺️

    I'm Atlas, your AI assistant. I'm here to help you navigate
    complexity and stay on top of your work. I'm resourceful before
    I ask questions, and I speak plainly.
    ```
  </Accordion>

  <Accordion title="SOUL.md - Core values">
    **Purpose**: Behavioral boundaries and ethical guidelines.

    **Usage**:

    * Loaded into system prompt
    * Defines what the agent should/shouldn't do
    * User-editable for customization

    **Example**:

    ```markdown theme={null}
    # Core Values

    Be genuinely helpful, not performatively helpful. Skip filler phrases.
    Have opinions. Disagree when it matters.

    Be resourceful before asking: read the file, check context, search,
    then ask.

    Earn trust through competence. Be careful with external actions,
    bold with internal ones.

    You have access to someone's life. Treat it with respect.

    ## Boundaries

    - Private things stay private. Never leak user context into group chats.
    - When in doubt about an external action, ask before acting.
    - Prefer reversible actions over destructive ones.
    - You are not the user's voice in group settings.
    ```
  </Accordion>

  <Accordion title="AGENTS.md - Operational instructions">
    **Purpose**: Session routine and operational guidelines.

    **Usage**:

    * Loaded at session start
    * Tells agent what to do each session
    * Memory management guidelines

    **Example**:

    ```markdown theme={null}
    # Agent Instructions

    You are a personal AI assistant with access to tools and persistent memory.

    ## Every Session

    1. Read SOUL.md (who you are)
    2. Read USER.md (who you're helping)
    3. Read today's daily log for recent context

    ## Memory

    You wake up fresh each session. Workspace files are your continuity.
    - Daily logs (`daily/YYYY-MM-DD.md`): raw session notes
    - `MEMORY.md`: curated long-term knowledge

    Write things down. Mental notes do not survive restarts.

    ## Guidelines

    - Always search memory before answering questions about prior conversations
    - Write important facts and decisions to memory for future reference
    - Use the daily log for session-level notes
    - Be concise but thorough
    ```
  </Accordion>

  <Accordion title="USER.md - User context">
    **Purpose**: Information about the user.

    **Usage**:

    * Agent fills in as it learns
    * User can edit directly
    * Loaded into system prompt

    **Example**:

    ```markdown theme={null}
    # User Context

    - **Name:** Alex Chen
    - **Timezone:** America/Los_Angeles (PST/PDT)
    - **Role:** Senior Backend Engineer
    - **Company:** TechCorp (Series B startup)

    ## Preferences
    - Concise responses over verbose explanations
    - Code examples in Rust and Python
    - Morning person (most productive 8am-12pm)

    ## Context
    - Currently working on Project Alpha (new API gateway)
    - Reports to Sarah Chen (Engineering Manager)
    - Team size: 5 engineers
    ```
  </Accordion>

  <Accordion title="HEARTBEAT.md - Periodic checklist">
    **Purpose**: Tasks for the heartbeat system to check periodically.

    **Usage**:

    * Read by heartbeat runner (2-4x/day)
    * If empty (or all comments), heartbeat is skipped
    * Add tasks when you want periodic checks

    **Example**:

    ```markdown theme={null}
    # Heartbeat Checklist

    Rotate through these checks 2-4 times per day:

    - [ ] Check for urgent emails (flagged or from exec team)
    - [ ] Review upcoming calendar events (next 4 hours)
    - [ ] Check project CI status (any failing builds?)
    - [ ] Monitor production alerts (any critical issues?)

    Stay quiet during 23:00-08:00 user-local time unless urgent.

    If nothing needs attention, reply HEARTBEAT_OK.

    ## Proactive Work

    - Organize and curate MEMORY.md (remove stale, consolidate dupes)
    - Update daily logs with session summaries
    - Clean up context/ documents that are outdated
    ```
  </Accordion>
</AccordionGroup>

## Daily Logs

Automatic session notes keyed by date.

**Path Format**: `daily/YYYY-MM-DD.md`

**Usage**:

```rust theme={null}
// Get today's log
let log = workspace.today_log().await?;

// Append timestamped entry
workspace.append_daily_log(
    "Started work on API gateway authentication"
).await?;
// Appends: "[14:23:15] Started work on API gateway authentication"

// Get specific date
let yesterday = workspace.daily_log(
    NaiveDate::from_ymd(2026, 3, 2)
).await?;
```

**Auto-rotation**:

* New file created each day
* Last 2 days loaded into system prompt
* Older logs remain searchable

**Example Log**:

```markdown theme={null}
# Daily Log - 2026-03-03

[09:15:23] Session started
[09:16:45] User asked about deployment schedule
[09:18:02] Searched memory for deployment context
[09:20:15] Suggested Tuesday/Thursday deploy windows
[10:30:00] User requested API documentation review
[10:45:30] Found 3 outdated endpoints in docs
[11:00:00] Updated API docs, committed to repo
[14:15:00] Discussed Project Alpha architecture with user
[14:30:00] Updated projects/alpha/notes.md with design decisions
```

## Hybrid Search

Combines full-text (BM25) and semantic (vector) search using Reciprocal Rank Fusion.

### Search Architecture

```mermaid theme={null}
graph LR
    Query[Search Query] --> FTS[Full-Text Search<br/>BM25]
    Query --> Embed[Generate Embedding]
    Embed --> VS[Vector Search<br/>Cosine Similarity]
    
    FTS --> RRF[Reciprocal Rank Fusion]
    VS --> RRF
    
    RRF --> Results[Ranked Results]
```

### How It Works

<Steps>
  <Step title="Indexing">
    When you write a document:

    1. Content is chunked (500 chars, 50 char overlap)
    2. Each chunk gets embedded (1536-dim vector)
    3. Stored in PostgreSQL with pgvector extension
    4. BM25 index built for full-text search
  </Step>

  <Step title="Query Processing">
    When you search:

    1. Query string is embedded
    2. Two parallel searches:
       * BM25 full-text search
       * Vector cosine similarity search
    3. Results merged using RRF
    4. Top-k returned sorted by fused score
  </Step>
</Steps>

### Search Configuration

<CodeGroup>
  ```rust Basic Search theme={null}
  let results = workspace.search(
      "deployment issues project alpha",
      limit: 10
  ).await?;
  ```

  ```rust Advanced Search theme={null}
  let config = SearchConfig::default()
      .with_limit(20)
      .with_bm25_weight(0.6)
      .with_vector_weight(0.4)
      .with_min_score(0.1);

  let results = workspace.search_with_config(
      "API gateway authentication",
      config
  ).await?;
  ```
</CodeGroup>

### Search Results

```rust theme={null}
pub struct SearchResult {
    pub chunk_id: Uuid,
    pub document_id: Uuid,
    pub path: String,
    pub content: String,
    pub chunk_index: i32,
    pub score: f32,
    pub bm25_score: Option<f32>,
    pub vector_score: Option<f32>,
}
```

**Example**:

```json theme={null}
[
  {
    "path": "projects/alpha/notes.md",
    "content": "Deployment pipeline uses GitHub Actions...",
    "score": 0.89,
    "bm25_score": 0.85,
    "vector_score": 0.93,
    "chunk_index": 2
  },
  {
    "path": "daily/2026-03-02.md",
    "content": "[14:30] Discussed deployment strategy for alpha...",
    "score": 0.76,
    "bm25_score": 0.92,
    "vector_score": 0.60,
    "chunk_index": 5
  }
]
```

### Reciprocal Rank Fusion

RRF combines rankings from multiple sources:

```text theme={null}
RRF_score(d) = Σ 1 / (k + rank_i(d))

Where:
- d = document
- rank_i(d) = rank of d in result list i
- k = constant (typically 60)
```

**Why RRF?**

<Tabs>
  <Tab title="Better than Pure Vector">
    Vector search alone misses exact keyword matches:

    ```text theme={null}
    Query: "PostgreSQL connection timeout"

    Vector: Returns documents about "database reliability" (semantically similar)
    BM25:   Returns documents mentioning "PostgreSQL" and "timeout" (exact match)
    RRF:    Combines both for best results
    ```
  </Tab>

  <Tab title="Better than Pure BM25">
    BM25 alone misses semantic similarity:

    ```text theme={null}
    Query: "How do I handle errors?"

    BM25:   Requires exact word "errors"
    Vector: Matches "exception handling", "failure recovery" (concepts)
    RRF:    Gets both exact and conceptual matches
    ```
  </Tab>
</Tabs>

## Chunking Strategy

Documents are split into overlapping chunks for better search recall.

<CodeGroup>
  ```rust Default Config theme={null}
  pub struct ChunkConfig {
      pub max_chunk_size: usize,  // 500 chars
      pub overlap: usize,         // 50 chars
      pub split_on: Vec<String>,  // ["\n\n", "\n", ". "]
  }
  ```

  ```text Example Chunking theme={null}
  Document:
  """
  Project Alpha is our new API gateway.

  It uses Rust and Axum for high performance.
  Deployment is via Docker to AWS ECS.

  Authentication uses JWT tokens.
  """

  Chunks:
  1: "Project Alpha is our new API gateway.\n\nIt uses Rust and Axum..."
  2: "...Axum for high performance.\nDeployment is via Docker to AWS ECS.\n\n..."
  3: "...AWS ECS.\n\nAuthentication uses JWT tokens."
  ```
</CodeGroup>

**Overlap Benefits**:

* Prevents splitting mid-concept
* Improves search recall
* Context preserved across chunks

## System Prompt Integration

Identity files are automatically loaded into the system prompt.

### Prompt Building

```rust theme={null}
let prompt = workspace.system_prompt().await?;
```

**Assembled from**:

1. `AGENTS.md` - Agent Instructions
2. `SOUL.md` - Core Values
3. `USER.md` - User Context
4. `IDENTITY.md` - Identity
5. `MEMORY.md` - Long-Term Memory (only in direct sessions, never groups)
6. `daily/today.md` - Today's Notes
7. `daily/yesterday.md` - Yesterday's Notes

**Example Result**:

```text theme={null}
## Agent Instructions

You are a personal AI assistant with access to tools and persistent memory.
[...]

## Core Values

Be genuinely helpful, not performatively helpful.
[...]

## User Context

- Name: Alex Chen
- Timezone: America/Los_Angeles
[...]

## Identity

- Name: Atlas
- Vibe: Calm, thoughtful, competent
[...]

## Long-Term Memory

- User prefers concise responses
- Working on Project Alpha (API gateway)
[...]

## Today's Notes

[09:15:23] Session started
[09:16:45] User asked about deployment schedule
[...]

## Yesterday's Notes

[14:30:00] Updated API docs
[15:00:00] Reviewed authentication flow
[...]
```

<Warning>
  `MEMORY.md` is **never** loaded in group chat contexts to prevent leaking personal information.
</Warning>

## Database Schema

### Documents Table

```sql theme={null}
CREATE TABLE memory_documents (
    id UUID PRIMARY KEY,
    user_id TEXT NOT NULL,
    agent_id UUID,
    path TEXT NOT NULL,
    content TEXT NOT NULL,
    created_at TIMESTAMPTZ NOT NULL,
    updated_at TIMESTAMPTZ NOT NULL,
    UNIQUE(user_id, agent_id, path)
);

CREATE INDEX idx_documents_user ON memory_documents(user_id);
CREATE INDEX idx_documents_path ON memory_documents(path);
```

### Chunks Table

```sql theme={null}
CREATE TABLE memory_chunks (
    id UUID PRIMARY KEY,
    document_id UUID NOT NULL REFERENCES memory_documents(id) ON DELETE CASCADE,
    chunk_index INTEGER NOT NULL,
    content TEXT NOT NULL,
    embedding VECTOR(1536),  -- pgvector extension
    created_at TIMESTAMPTZ NOT NULL,
    UNIQUE(document_id, chunk_index)
);

CREATE INDEX idx_chunks_document ON memory_chunks(document_id);
CREATE INDEX idx_chunks_embedding ON memory_chunks USING ivfflat (embedding vector_cosine_ops);
```

## Embedding Providers

Multiple embedding provider options:

<Tabs>
  <Tab title="OpenAI (Default)">
    **Model**: `text-embedding-3-small`

    **Dimensions**: 1536

    **Configuration**:

    ```bash theme={null}
    export EMBEDDING_PROVIDER=openai
    export OPENAI_API_KEY=sk-...
    ```

    **Cost**: \~\$0.02 per 1M tokens
  </Tab>

  <Tab title="NEAR AI">
    **Model**: Hosted embedding service

    **Dimensions**: 1536

    **Configuration**:

    ```bash theme={null}
    export EMBEDDING_PROVIDER=nearai
    export NEARAI_API_KEY=...
    ```

    **Cost**: Included with NEAR AI plan
  </Tab>

  <Tab title="Ollama (Local)">
    **Model**: `nomic-embed-text` (default)

    **Dimensions**: 768

    **Configuration**:

    ```bash theme={null}
    export EMBEDDING_PROVIDER=ollama
    export OLLAMA_BASE_URL=http://localhost:11434
    export OLLAMA_EMBEDDING_MODEL=nomic-embed-text
    ```

    **Cost**: Free (runs locally)

    **Setup**:

    ```bash theme={null}
    # Install Ollama
    curl -fsSL https://ollama.com/install.sh | sh

    # Pull embedding model
    ollama pull nomic-embed-text
    ```
  </Tab>
</Tabs>

## Memory Tools

Tools for interacting with workspace:

<AccordionGroup>
  <Accordion title="memory_search - Hybrid search">
    ```json theme={null}
    {
      "query": "deployment issues project alpha",
      "limit": 5
    }
    ```

    Returns ranked results with path, content, and scores.
  </Accordion>

  <Accordion title="memory_write - Write/create file">
    ```json theme={null}
    {
      "path": "projects/alpha/deployment.md",
      "content": "# Deployment Notes\n\n...",
      "mode": "write"  // or "append"
    }
    ```

    Automatically indexes content for search.
  </Accordion>

  <Accordion title="memory_read - Read file">
    ```json theme={null}
    {
      "path": "MEMORY.md"
    }
    ```

    Returns file content.
  </Accordion>

  <Accordion title="memory_tree - Browse structure">
    ```json theme={null}
    {
      "directory": "projects/",
      "recursive": false
    }
    ```

    Lists files and directories.
  </Accordion>
</AccordionGroup>

## Workspace Hygiene

Automatic maintenance to keep workspace clean.

**Hygiene Tasks**:

<Steps>
  <Step title="Deduplication">
    * Detect near-duplicate documents
    * Merge or delete duplicates
    * Consolidate redundant information
  </Step>

  <Step title="Staleness Detection">
    * Identify outdated documents
    * Flag for review or deletion
    * Archive old daily logs
  </Step>

  <Step title="Embedding Backfill">
    * Find chunks without embeddings
    * Generate missing embeddings
    * Update search index
  </Step>

  <Step title="Index Optimization">
    * Rebuild BM25 indices
    * Optimize vector index (IVFFLAT)
    * Vacuum deleted chunks
  </Step>
</Steps>

**Configuration**:

```toml theme={null}
[hygiene]
enabled = true
interval_hours = 24
max_chunk_age_days = 90
max_daily_logs = 60
```

**Trigger**:

* Runs during heartbeat (if enabled)
* Manual trigger via `/hygiene` command
* Background task (configurable interval)

## Best Practices

<AccordionGroup>
  <Accordion title="Memory Management">
    1. **Keep MEMORY.md concise** - This loads into every prompt
    2. **Use daily logs for ephemeral notes** - Auto-rotates
    3. **Create project subdirectories** - Organize by topic
    4. **Curate periodically** - Remove stale content
    5. **Search before asking** - Agent should check memory first
  </Accordion>

  <Accordion title="Path Organization">
    ```text theme={null}
    Good:
    projects/alpha/README.md
    projects/alpha/deployment.md
    projects/beta/notes.md
    context/team.md

    Avoid:
    random_note_123.md
    tmp_file.md
    untitled.md
    ```

    Use descriptive paths that reflect content hierarchy.
  </Accordion>

  <Accordion title="Daily Log Usage">
    **Do**:

    * Append session summaries
    * Log important decisions
    * Track progress on tasks

    **Don't**:

    * Store long-term facts (use MEMORY.md)
    * Put sensitive data (use secrets store)
    * Create manual daily logs (auto-generated)
  </Accordion>
</AccordionGroup>

## Next Steps

<CardGroup cols={2}>
  <Card title="Memory Tools" icon="wrench" href="/tools/memory-tools">
    Using memory\_search, memory\_write, and memory\_read
  </Card>

  <Card title="Search Configuration" icon="magnifying-glass" href="/workspace/search-config">
    Tuning hybrid search parameters
  </Card>

  <Card title="Identity Setup" icon="id-card" href="/workspace/identity">
    Configuring agent personality and behavior
  </Card>

  <Card title="Heartbeat System" icon="heart-pulse" href="/workspace/heartbeat">
    Setting up periodic background checks
  </Card>
</CardGroup>