Engram – A distributed memory system for AI agents, with extensible architecture

Development Guide - Engram - Distributed Memory for AI Agents Keyboard shortcuts Press ← or → to navigate between chapters Press S or / to search in the book Press ? to show this help Press Esc to hide this help Auto Light Rust Coal Navy Ayu Engram - Distributed Memory for AI Agents Engram - Rust Implementation A distributed memory system for AI agents, implemented in Rust with extensible architecture and BDD testing. Features Core Architecture Extensible Entity System : Trait-based entity architecture supporting dynamic model loading Git-based Storage : Content-addressable storage with multi-agent synchronization Modular CLI : Command-line interface with subcommands for all operations BDD Testing : Gherkin-style behavior-driven development testing Configuration System : Extensible configuration with validation Plugin Support : Dynamic plugin loading for custom extensions Entity Types Tasks : Work items with status tracking, priority, and hierarchy Context : Background information with relevance scoring and source tracking Reasoning : Decision chains with confidence levels and evidence tracking Knowledge : Information storage with usage metrics and categorization Sessions : Agent sessions with SPACE framework and DORA metrics Compliance : Requirements tracking with violations and remediation Rules : System rules with execution history and conditions Standards : Team standards with versioning and relationships ADRs : Architectural Decision Records with alternatives and outcomes Workflows : State machines with transitions and permission schemes Relationships : Entity relationships with graph operations and path finding Commit Validation : Pre-commit hook system with task relationship requirements Relationship Management The relationship system provides powerful graph-based operations for modeling connections between entities: Relationship Types DependsOn : Entity dependencies and prerequisites Contains : Hierarchical containment relationships References : Cross-references and citations Fulfills : Implementation and completion relationships Implements : Technical implementation relationships Supersedes : Version and replacement relationships AssociatedWith : General associations Influences : Impact and influence relationships Custom : User-defined relationship types Relationship Features Bidirectional/Unidirectional : Configurable relationship directions Weighted Connections : Relationship strength (Weak, Medium, Strong, Critical) Graph Traversal : BFS, DFS, and Dijkstra pathfinding algorithms Relationship Constraints : Validation rules and cardinality limits Analytics : Connection statistics and graph metrics Storage Features Content-addressable : SHA-256 hashing for integrity verification Multi-agent : Collaboration with conflict resolution strategies Git Integration : Full Git history, branching, and merging Performance : Efficient querying and indexing CLI Commands # Setup engram setup workspace engram setup agent --name alice --type coder # Task Management engram task create --title "Implement auth" --priority high engram task list --agent alice engram task update --status done # Context Management engram context create --title "API docs" --source "documentation" engram context list # Reasoning Chains engram reasoning create --title "Authentication approach" --task-id # Knowledge Management engram knowledge create --title "OAuth2 flows" --type pattern # Session Management engram session start --agent alice --auto-detect engram session status --id --metrics # Compliance & Standards engram compliance create --title "Security requirements" --category security engram standard create --title "Coding standards" --category coding # ADRs engram adr create --title "Database choice" --number 001 # Workflows engram workflow create --title "Development pipeline" ### Skills Integration Engram ships with specialized skills for common workflows: ```bash # Use skills from ./engram/skills/ cat ./engram/skills/meta/use-engram-memory.md # Core memory skill cat ./engram/skills/meta/delegate-to-agents.md # Agent delegation cat ./engram/skills/workflow/plan-feature.md # Feature planning cat ./engram/skills/compliance/check-compliance.md # Compliance checking Prompts Library Engram includes a comprehensive prompt library for agent orchestration: # Agent prompts (170+) ls ./engram/prompts/agents/ # Pipeline templates (100+) ls ./engram/prompts/ai/pipelines/ # Compliance prompts (250+) ls ./engram/prompts/compliance_and_certification/prompts/audit_checkpoints/ Engram-Adapted Prompts Core agents and pipelines are adapted for engram integration: 01-the-one.yaml - Orchestrator with engram task creation 03-the-architect.yaml - Architecture with engram context storage 05-the-deconstructor.yaml - Task breakdown with engram subtasks 01-greenfield-feature-launch.yaml - Engram workflow orchestration All adapted prompts include: task_id parameter for engram tracking engram reasoning create for progress storage engram context create for result storage engram relationship create for entity linking JSON response with engram entity IDs Using Skills and Prompts # Check available skills ls ./engram/skills/ # Use a skill cat ./engram/skills/meta/use-engram-memory.md # Use an adapted agent prompt cat ./engram/prompts/agents/01-the-one.yaml # Use a pipeline template cat ./engram/prompts/ai/pipelines/01-greenfield-feature-launch.yaml Documentation Using Engram - Core workflow protocol Prompt Engineering Guide - Prompt construction patterns Relationship Management engram relationship create --source-id task1 --source-type task --target-id task2 --target-type task --relationship-type depends-on --agent alice engram relationship list --agent alice engram relationship get `` engram relationship find-path --source-id task1 --target-id task3 --algorithm dijkstra engram relationship connected --entity-id task1 --relationship-type depends-on engram relationship stats --agent alice engram relationship delete `` Commit Validation and Hooks engram validation commit --message "feat: implement user authentication [TASK-123]" engram validation commit --message "test commit" --dry-run engram validation hook install engram validation hook uninstall engram validation hook status engram validation check Synchronization engram sync --agents "alice,bob" --strategy intelligent_merge Perkeep Backup and Restore engram perkeep backup --description "Full backup" engram perkeep backup --entity-type task --include-relationships engram perkeep list --detailed engram perkeep restore --blobref "sha256-..." engram perkeep health engram perkeep config --server "http://localhost:3179" Perkeep Integration Engram supports backing up and restoring entities using Perkeep , a personal data store for content-addressable storage. Configuration Set environment variables to configure Perkeep: export PERKEEP_SERVER="http://localhost:3179" # Default: http://localhost:3179 export PERKEEP_AUTH_TOKEN="your-token" # Optional: for authenticated servers Backup Commands # Backup all entity types engram perkeep backup # Backup specific entity type engram perkeep backup --entity-type task # Backup with description engram perkeep backup --description "Weekly backup" # Include relationships in backup (default: true) engram perkeep backup --include-relationships Restore Commands # Restore from most recent backup engram perkeep restore # Restore from specific backup blobref engram perkeep restore --blobref "sha256-abc123..." # Dry run (preview what would be restored) engram perkeep restore --dry-run # Restore with agent override engram perkeep restore --agent default Management Commands # List available backups engram perkeep list engram perkeep list --detailed # Check server health engram perkeep health # Configure settings engram perkeep config --server "http://localhost:3179" How It Works Content-Addressable Storage : Entities are serialized to JSON and uploaded as blobs to Perkeep Backup Metadata : A schema object tracks all entity blobrefs, timestamps, and counts Entity Types : task, context, reasoning, knowledge, session, and relationship entities Restore Process : Fetches backup metadata, retrieves all entity blobs, and stores them Development Building cargo build --release Testing # Run BDD tests cargo test --test bdd # Run unit tests cargo test # Run with specific feature cargo test --features plugins Examples See the examples/ directory for usage examples: cargo run --example basic_usage Configuration Engram supports YAML/TOML configuration with the following structure: app: log_level: info default_agent: default git: author_name: Your Name author_email: [email protected] workspace: agents: coder: type: implementation description: "Handles code changes" storage: storage_type: git base_path: .engram sync_strategy: intelligent_merge features: plugins: true analytics: true experimental: false Architecture Extensible Models The Rust implementation uses trait-based architecture for extensibility: #![allow(unused)] fn main() { pub trait Entity: Serialize + for<'de> Deserialize<'de> + Send + Sync { fn entity_type() -> &'static str; fn id(&self) -> &str; fn validate(&self) -> Result<(), String>; // ... other methods } } Plugin System Custom entity types can be added dynamically: #![allow(unused)] fn main() { // Register new entity type registry.register::(); // Create from generic representation let entity = registry.create(generic_entity)?; } Commit Validation and Pre-commit Hooks The validation system enforces disciplined development practices by ensuring all commits follow established patterns and reference proper tasks with required relationships. Key Features: Task Reference Validation : Commits must reference existing tasks in multiple formats Relationship Requirements : Tasks must have reasoning and context relationships File Scope Validation : Changed files must match planned task scope Exemption Support : Configurable exemptions for chore, docs, fixup commits Performance Optimized : Caching and parallel validation support Supported Task ID Formats: [TASK-123] - Brackets format [task:auth-impl-001] - Colon format Refs: #456 - Reference format Validation Rules: # Validate a commit engram validation commit --message "feat: implement authentication [TASK-123]" # Check hook status engram validation hook status # Install validation hook engram validation hook install Configuration: Configuration via .engram/validation.yaml : enabled: true require_task_reference: true require_reasoning_relationship: true require_context_relationship: true task_id_patterns: - pattern: '\[([A-Z]+-\d+)\]' name: "Brackets format" example: "[TASK-123]" exemptions: - message_pattern: '^(chore|docs):' skip_specific: ["require_task_reference"] performance: cache_ttl_seconds: 300 enable_parallel_validation: true Plugin System Storage Layer The storage layer supports multiple backends through the Storage trait: #![allow(unused)] fn main() { pub trait Storage: Send + Sync { fn store(&mut self, entity: &dyn Entity) -> Result<(), EngramError>; fn get(&self, id: &str, entity_type: &str) -> Result>, EngramError>; // ... other methods } } BDD Testing Behavior-driven development testing with Gherkin syntax: Feature: Task Management Scenario: Create a new task Given I have a workspace And I am logged in as agent "test-agent" When I create a new task "Implement login feature" Then the task should be created successfully Performance Memory Efficient : Content-addressable storage prevents duplication Fast Queries : Indexed storage with agent-based filtering Scalable : Git handles large repositories efficiently Concurrent Safe : Thread-safe storage operations Enterprise Features Multi-tenant : Isolated workspaces with shared memory Audit Trails : Complete Git history for compliance Conflict Resolution : Multiple strategies for team collaboration Analytics : SPACE framework and DORA metrics Security : Content integrity verification with SHA-256 Migration from Go The Rust implementation is designed as a drop-in replacement for the Go version: Same CLI commands and arguments Compatible data formats (JSON/YAML) Same Git storage structure Enhanced performance and features Contributing The Rust implementation welcomes contributions for: New entity types Storage backends CLI commands BDD test scenarios Plugin examples License AGPL-3.0-or-later OR Commercial - dual-licensed for open source and commercial use