Understanding ElizaOS memory system, including storage, retrieval, and semantic search capabilities with actual implementation details

Memory System

ElizaOS provides a sophisticated memory system for persistent storage and retrieval of conversations, facts, and documents with built-in support for embeddings and semantic search. The system is designed to handle different types of memories with appropriate metadata and efficient querying.

Memory Types

Eliza categorizes memories into distinct types for optimal storage and retrieval:

enum MemoryType {
  DOCUMENT = "document", // Complete documents or large text
  FRAGMENT = "fragment", // Chunks of documents for embedding
  MESSAGE = "message", // Conversational messages
  DESCRIPTION = "description", // Descriptive information
  CUSTOM = "custom", // Application-specific memories
}

Memory Structure

Core Memory Interface

interface Memory {
  id?: UUID;
  entityId: UUID; // Associated user/entity
  agentId?: UUID; // Associated agent
  createdAt?: number; // Timestamp in milliseconds
  content: Content; // Memory content with text
  embedding?: number[]; // Vector embedding for semantic search
  roomId: UUID; // Conversation room/channel
  worldId?: UUID; // Optional world context
  unique?: boolean; // Prevent duplicates
  similarity?: number; // Similarity score (when searching)
  metadata?: MemoryMetadata; // Type-specific metadata
}

Memory Content

Content can contain text and additional structured data:

interface Content {
  text: string;
  // Additional fields based on content type
  [key: string]: any;
}

Memory Metadata

Each memory type has specific metadata requirements:

// Message metadata
interface MessageMetadata extends BaseMetadata {
  type: MemoryType.MESSAGE;
}

// Document metadata
interface DocumentMetadata extends BaseMetadata {
  type: MemoryType.DOCUMENT;
}

// Fragment metadata (for document chunks)
interface FragmentMetadata extends BaseMetadata {
  type: MemoryType.FRAGMENT;
  documentId: UUID;
  position: number;
}

Database Tables

ElizaOS uses multiple specialized tables for different types of data. All memory operations use the same createMemory, getMemories, and searchMemories methods, but with different table names:

Core Memory Tables

Table Name	Purpose	Common Usage
`messages`	Conversation messages	Chat history, user inputs, agent responses
`facts`	Persistent facts and knowledge	User preferences, learned information
`documents`	Full documents	PDFs, text files, knowledge bases
`fragments`	Document chunks for search	Embedded document pieces for RAG
`memories`	General memories	Custom memory types

System Tables

Table Name	Purpose	Common Usage
`agents`	Agent configurations	Agent metadata, settings
`entities`	User/entity profiles	User information, entity relationships
`rooms`	Conversation contexts	Chat rooms, channels, sessions
`relationships`	Entity relationships	Connections between users/entities
`components`	Entity components	Modular entity data
`tasks`	Scheduled tasks	Reminders, scheduled actions
`cache`	Temporary data	Performance optimization
`logs`	System logs	Debugging, audit trail

Integration Tables

Table Name	Purpose	Common Usage
`channels`	External channel mappings	Discord, Telegram channel info
`channel_participants`	Channel membership	Who's in which channel
`participants`	Room participants	Active conversation members
`message_servers`	Message server configs	Server-specific settings
`server_agents`	Agent-server mappings	Multi-agent deployments
`embeddings`	Vector embeddings	Semantic search indexes

Table Usage Examples

// Store a chat message
await runtime.createMemory({
  content: { text: "Hello, how are you?" },
  entityId: userId,
  roomId: roomId,
  metadata: { type: MemoryType.MESSAGE }
}, "messages");

// Store a fact
await runtime.createMemory({
  content: { text: "User prefers email notifications" },
  entityId: userId,
  metadata: { type: MemoryType.DESCRIPTION }
}, "facts");

// Store a task
await runtime.createMemory({
  content: { text: "Send weekly report" },
  entityId: agentId,
  metadata: { 
    type: MemoryType.CUSTOM,
    scheduledFor: Date.now() + 7 * 24 * 60 * 60 * 1000
  }
}, "tasks");

// Query specific tables
const userFacts = await runtime.getMemories({
  tableName: "facts",
  entityId: userId,
  count: 50
});

const recentMessages = await runtime.getMemories({
  tableName: "messages",
  roomId: roomId,
  count: 10
});

Creating Memories

Basic Memory Creation

// Create a message memory
const messageMemory = await runtime.createMemory(
  {
    content: { text: "Hello, how can I help you today?" },
    entityId: userId,
    roomId: roomId,
    metadata: {
      type: MemoryType.MESSAGE,
      timestamp: Date.now(),
    },
  },
  "messages"
);

// Create a document memory
const documentMemory = await runtime.createMemory(
  {
    content: {
      text: documentText,
      title: "User Guide",
      source: "docs/user-guide.md",
    },
    entityId: agentId,
    roomId: roomId,
    metadata: {
      type: MemoryType.DOCUMENT,
      source: "filesystem",
      tags: ["documentation", "guide"],
    },
  },
  "documents"
);

Memory with Embeddings

Embeddings are automatically generated for semantic search:

// Memory with automatic embedding generation
const memory = await runtime.addEmbeddingToMemory({
  content: { text: "The weather today is sunny and warm." },
  entityId: userId,
  roomId: roomId,
});

// Custom embedding
const customMemory = {
  content: { text: "Custom content" },
  embedding: await generateCustomEmbedding(text),
  entityId: userId,
  roomId: roomId,
};
await runtime.createMemory(customMemory, "memories");

Retrieving Memories

Basic Retrieval

// Get recent messages
const recentMessages = await runtime.getMemories({
  roomId: roomId,
  tableName: "messages",
  count: 50,
  unique: false,
});

// Get memories by time range
const timeRangeMemories = await runtime.getMemories({
  roomId: roomId,
  tableName: "messages",
  start: Date.now() - 86400000, // Last 24 hours
  end: Date.now(),
});

// Get unique memories only
const uniqueMemories = await runtime.getMemories({
  roomId: roomId,
  tableName: "facts",
  unique: true,
  count: 100,
});

Memory by ID

// Single memory
const memory = await runtime.getMemoryById(memoryId);

// Multiple memories
const memories = await runtime.getMemoriesByIds([id1, id2, id3], "messages");

Semantic Search

Eliza supports vector-based semantic search using embeddings:

Basic Semantic Search

// Search for similar memories
const searchResults = await runtime.searchMemories({
  query: "weather forecast",
  tableName: "messages",
  roomId: roomId,
  count: 10,
  match_threshold: 0.7,
});

// Results include similarity scores
searchResults.forEach((memory) => {
  console.log(`Text: ${memory.content.text}`);
  console.log(`Similarity: ${memory.similarity}`);
});

Advanced Search with Embeddings

// Generate embedding for search query
const queryEmbedding = await runtime.useModel(ModelType.TEXT_EMBEDDING, {
  text: "What's the temperature outside?",
});

// Search with pre-computed embedding
const results = await runtime.searchMemories({
  embedding: queryEmbedding,
  tableName: "messages",
  roomId: roomId,
  match_threshold: 0.8,
  count: 5,
});

Memory Management

Updating Memories

// Update memory content and regenerate embedding
const updated = await runtime.updateMemory({
  id: memoryId,
  content: { text: "Updated content" },
  metadata: {
    type: MemoryType.MESSAGE,
    edited: true,
    editedAt: Date.now(),
  },
});

Deleting Memories

// Delete single memory
await runtime.deleteMemory(memoryId);

// Delete multiple memories
await runtime.deleteManyMemories([id1, id2, id3]);

// Delete all memories in a room
await runtime.deleteAllMemories(roomId, "messages");

Memory Deduplication

The unique flag prevents duplicate memories:

// Store unique facts only
await runtime.createMemory(
  {
    content: { text: "The user's favorite color is blue" },
    entityId: userId,
    roomId: roomId,
    unique: true,
  },
  "facts"
);

// Second attempt with same content will be ignored
await runtime.createMemory(
  {
    content: { text: "The user's favorite color is blue" },
    entityId: userId,
    roomId: roomId,
    unique: true,
  },
  "facts"
); // No duplicate created

Document Processing

For large documents, Eliza supports fragmentation:

// Create document with fragments
async function processDocument(documentText: string, title: string) {
  // Create main document
  const documentId = await runtime.createMemory(
    {
      content: { text: title, fullText: documentText },
      metadata: {
        type: MemoryType.DOCUMENT,
        timestamp: Date.now(),
      },
    },
    "documents"
  );

  // Create fragments for embedding
  const chunks = splitIntoChunks(documentText, 1000);

  for (let i = 0; i < chunks.length; i++) {
    await runtime.createMemory(
      {
        content: { text: chunks[i] },
        metadata: {
          type: MemoryType.FRAGMENT,
          documentId: documentId,
          position: i,
        },
      },
      "fragments"
    );
  }
}

Caching

Eliza includes caching for frequently accessed data:

// Store in cache
await runtime.setCache("user_preferences", {
  theme: "dark",
  language: "en",
});

// Retrieve from cache
const preferences = await runtime.getCache("user_preferences");

// Delete from cache
await runtime.deleteCache("user_preferences");

Memory Tables

Eliza uses different tables for different memory types:

memories: General purpose memories
messages: Conversation messages
facts: Agent facts and knowledge
documents: Full documents
fragments: Document chunks for search

Best Practices

1. Choose Appropriate Memory Types

// Use specific tables for better organization
const message = { content: { text: "Hello" }, ... };
await runtime.createMemory(message, 'messages');

const fact = { content: { text: "User is 25 years old" }, ... };
await runtime.createMemory(fact, 'facts', true); // unique

2. Optimize Embeddings

// Batch processing for multiple memories
const memories = texts.map((text) => ({
  content: { text },
  entityId: userId,
  roomId: roomId,
}));

// Add embeddings in batch
const memoriesWithEmbeddings = await Promise.all(
  memories.map((m) => runtime.addEmbeddingToMemory(m))
);

3. Use Metadata Effectively

const memory = {
  content: { text: "Meeting scheduled for tomorrow" },
  metadata: {
    type: MemoryType.MESSAGE,
    source: "calendar",
    sourceId: eventId,
    tags: ["meeting", "schedule"],
    timestamp: Date.now(),
  },
};

4. Implement Search Strategies

// Combine semantic and metadata filtering
async function searchWithContext(query: string, tags: string[]) {
  const embedding = await runtime.useModel(ModelType.TEXT_EMBEDDING, {
    text: query,
  });

  const results = await runtime.searchMemories({
    embedding,
    tableName: "messages",
    roomId: roomId,
    match_threshold: 0.75,
  });

  // Filter by tags
  return results.filter((memory) => tags.some((tag) => memory.metadata?.tags?.includes(tag)));
}

Example: Conversation Memory

Here's a complete example of managing conversation memory:

class ConversationManager {
  constructor(private runtime: IAgentRuntime) {}

  async saveMessage(userId: UUID, text: string, roomId: UUID) {
    // Create message memory with embedding
    const memory = await this.runtime.addEmbeddingToMemory({
      content: {
        text,
        role: "user",
        timestamp: Date.now(),
      },
      entityId: userId,
      roomId: roomId,
      metadata: {
        type: MemoryType.MESSAGE,
        timestamp: Date.now(),
      },
    });

    await this.runtime.createMemory(memory, "messages");
    return memory.id;
  }

  async getConversationContext(roomId: UUID, query: string) {
    // Get recent messages
    const recentMessages = await this.runtime.getMemories({
      roomId,
      tableName: "messages",
      count: 10,
    });

    // Search for relevant context
    const relevantMessages = await this.runtime.searchMemories({
      query,
      tableName: "messages",
      roomId,
      count: 5,
      match_threshold: 0.7,
    });

    // Combine and deduplicate
    const allMessages = [...recentMessages, ...relevantMessages];
    const unique = Array.from(new Map(allMessages.map((m) => [m.id, m])).values());

    return unique.sort((a, b) => (a.createdAt || 0) - (b.createdAt || 0));
  }
}

State Management - How state is composed using memories
Knowledge Management - Managing agent knowledge base
Database Architecture - Underlying database structure

Memory System

On this page