Cognitive Core for AI Agents

Explore the Virtuals Protocol Cognitive Core, which combines LLMs, RAG, fine-tuning, vector databases, and persistent memory for personalized AI agents.

The Cognitive Core is the central intelligence layer for a VIRTUAL agent. It uses large language models (LLMs), retrieval, and persistent memory to execute tasks and deliver a distinct AI agent personality.

Large language models for AI agents

VIRTUAL agents use open-source LLMs. The Cognitive Core combines retrieval-augmented generation and model fine-tuning to build each agent’s personality and domain intelligence.

• AI agent personality development Retrieval-augmented generation (RAG) develops an agent’s backstory, lore, traits, and characteristics. RAG combines language generation with knowledge-base retrieval, creating more relevant and lifelike AI agent interactions.

• AI agent central intelligence

  VIRTUAL agents with substantial datasets use direct fine-tuning of open-source models. Fine-tuning improves accurate, domain-specific responses. Instruction fine-tuning further aligns an AI agent’s responses and actions with defined rules or objectives. Smaller datasets are stored in a vector database and retrieved through RAG. This gives the AI agent efficient access to specialized information.

AI training data preprocessing

AI agent training data can include text, video, and audio from textbooks, forums, and wikis. The Cognitive Core primarily uses text-based LLMs. Video and audio training data require transcription before model training.

• Data cleaning: Removes noise and null values to maintain data integrity and quality.

• Data transformation: Standardizes datasets for reliable model training.

Persistent AI agent memory and conversation context

VIRTUAL agents use a persistent memory system for personalized, context-aware interactions. The system retains conversation context while supporting efficient long-term memory processing.

1. User and conversation identification: The system identifies users and their conversations for accurate recall.

2. Long conversation storage: The system stores and processes extended conversations efficiently.

Unique user identifier

Each VIRTUAL agent user receives a unique identifier. This maintains user-specific context and conversation continuity.

A Sample Database Table

A sample database table is formed as below.

CREATE TABLE Messages (
    message_id VARCHAR(32) NOT NULL PRIMARY KEY,
    conversation_id VARCHAR(32) NOT NULL,
    user_id VARCHAR(32) NOT NULL,
    prompt TEXT NOT NULL,
    timestamp DATETIME NOT NULL,
    response TEXT, 
    FOREIGN KEY (conversation_id) REFERENCES Conversations(conversation_id)
);

Vector database and memory retrieval

Embedding techniques vectorize messages into numerical formats for efficient storage and retrieval.

When the getPrompt('identifier', 'context', 'params') function runs, the system retrieves the user’s messages from the vector database. The LLM uses this retrieved conversation history to generate personalized, contextually relevant responses without requiring added context from the dApp.

Learn more about contributing to Cognitive Core.