Telemem

TeleMem: Building Long-Term and Multimodal Memory for Agentic AI

If you find this project helpful, please give us a ⭐️ on GitHub for the latest update.

_🤝 Contributions welcome! Feel free to open an issue or submit a pull request._

---

English | 简体中文

📄 Awesome-Agent-Memory →

TeleMem is an agent memory management layer that can be used as a high-performance drop-in replacement for [Mem0](https://mem0.ai/) with one line of code (import telemem as mem0) , deeply optimized for complex scenarios involving multi-turn dialogues, character modeling, long-term information storage, and semantic retrieval.

Through its unique context-aware enhancement mechanism, TeleMem provides conversational AI with core infrastructure offering higher accuracy, faster performance, and stronger character memory capabilities.

Building upon this foundation, TeleMem implements video understanding, multimodal reasoning, and visual question answering capabilities. Through a complete pipeline of video frame extraction, caption generation, and vector database construction, AI Agents can effortlessly store, retrieve, and reason over video content just like handling text memories.

The ultimate goal of the TeleMem project is to _use an agent's hindsight to improve its foresight_.

TeleMem, where memory lives on and intelligence grows strong.

Why TeleMem?

• 🎭 Character memory done right — the only open-source memory layer that automatically builds isolated, per-character memory profiles, built for role-play, companion AI, NPCs, and multi-persona assistants.
• 🎬 Memory for video, not just text — a full video → frames → captions → vector DB pipeline with ReAct-style multi-step video QA.
• 🏠 Fully local by default — runs end-to-end on your hardware (Qwen + FAISS); no cloud service, no paid tier, no data leaving your machine.
• 🔌 mem0-compatible API — add() / search() accept the same arguments and return the same {"results": [...]} shapes, so existing Mem0 code keeps working.

---

📢 Latest Updates

• **[2026-08-15] 🎉 TeleMem v1.10.0 adds first-class DeepSeek Harness support: an opt-in Cordis/MCP patch starts TeleMem with uvx, registers all 8 memory tools under mcp__telemem__*, and securely forwards provider configuration. See the MCP server docs.**
• [2026-08-06] 🎉 TeleMem [v1.9.0](https://github.com/TeleAI-UAGI/telemem/releases/tag/v1.9.0) is on the latest MCP standard: migrated to the official [MCP Python SDK v2](https://github.com/modelcontextprotocol/python-sdk) (spec 2026-07-28) — all 8 tools now declare titles, behavior annotations, and structured output, while staying compatible with older MCP clients. See the [MCP server docs](docs/MCP.md).
• [2026-07-11] 🎉 TeleMem [v1.8.0](https://github.com/TeleAI-UAGI/telemem/releases/tag/v1.8.0) — the "claims = contracts" release: character-memory extraction fix, `infer=False`/`prompt`/`memory_type` now fully honored, offline contract test suite, telemetry disabled by default, and a [multi-NPC demo](examples/multi_npc.py)!
• [2026-06-12] 🎉 TeleMem [v1.7.1](https://github.com/TeleAI-UAGI/telemem/releases/tag/v1.7.1) is live on the [official MCP registry](https://registry.modelcontextprotocol.io) — run the memory server with zero install: `uvx telemem`! Also new: [evaluation principles](https://teleai-uagi.github.io/telemem/evaluation/) and a LongMemEval harness with built-in baselines.
• [2026-06-12] 🎉 TeleMem is now on PyPI: `pip install telemem`! [v1.6.0](https://github.com/TeleAI-UAGI/telemem/releases/tag/v1.6.0) adds Ollama/DeepSeek/Kimi configs, LangChain & LlamaIndex examples, and a [documentation site](https://teleai-uagi.github.io/telemem/).
• [2026-06-12] 🎉 TeleMem [v1.5.0](https://github.com/TeleAI-UAGI/telemem/releases/tag/v1.5.0) has been released: true mem0 drop-in API, lightweight core install, and CI!
• [2026-06-11] 🎉 TeleMem [v1.4.0](https://github.com/TeleAI-UAGI/telemem/releases/tag/v1.4.0) has been released with [MCP support](docs/MCP.md)!
• [2026-01-28] 🎉 TeleMem [v1.3.0](https://github.com/TeleAI-UAGI/telemem/releases/tag/v1.3.0) has been released!
• [2026-01-22] 🎉 TeleMem [Tech Report](https://arxiv.org/abs/2601.06037) has been updated to its 4th version!
• [2026-01-13] 🎉 TeleMem [Tech Report](https://arxiv.org/abs/2601.06037) has been released on arXiv!
• [2026-01-09] 🎉 TeleMem [v1.2.0](https://github.com/TeleAI-UAGI/telemem/releases/tag/v1.2.0) has been released!
• [2025-12-31] 🎉 TeleMem [v1.1.0](https://github.com/TeleAI-UAGI/telemem/releases/tag/v1.1.0) has been released!
• [2025-12-05] 🎉 TeleMem [v1.0.0](https://github.com/TeleAI-UAGI/telemem/releases/tag/v1.0.0) has been released!

---

🔥 Research Highlights

• Significantly improved memory accuracy: Achieved 86.33% accuracy on the ZH-4O Chinese multi-character long-dialogue benchmark, 19% higher than Mem0.
• Doubled speed performance: Millisecond-level semantic retrieval enabled by efficient buffering and batch writing.
• Greatly reduced token cost: Optimized token usage delivers the same performance with significantly lower LLM overhead.
• Precise character memory preservation: Automatically builds independent memory profiles for each character, eliminating confusion.
• Automated Video Processing Pipeline: From raw video → frame extraction → caption generation → vector database, fully automated
• ReAct-Style Video QA: Multi-step reasoning + tool calling for precise video content understanding

---

📌 Table of Contents

• Project Introduction
• TeleMem vs Mem0: Core Advantages
• Experimental Results
• Quick Start
• Project Structure
• Core Functions
• Multimodal Extensions
• MCP Server
• Framework Integrations
• Data Storage Explanation
• Telemetry & Privacy
• Development and Contribution
• Acknowledgements
• Citation

---

Project Introduction

TeleMem enables conversational AI to maintain stable, natural, and continuous worldviews and character settings during long-term interactions through a deeply optimized pipeline of character-aware summarization → semantic clustering deduplication → efficient storage → precise retrieval.

flowchart LR

A["Dialogue messages"] --> B["Character-aware summarization (global + per-character)"] B --> C["Embedding + similar-memory retrieval"] C --> D["Write buffer (batch flush)"] D --> E["LLM semantic clustering & fusion"] E --> F[("FAISS index + JSON metadata")] Q["Query"] --> S["Vector search

• rerank"] F --> S S --> R["results"]

Features

• Automatic memory extraction: Extracts and structures key facts from dialogues.
• Semantic clustering & deduplication: Uses LLMs to semantically merge similar memories, reducing conflicts and improving consistency.
• Character-profiled memory management: Builds independent memory archives for each character in a dialogue, ensuring precise isolation and personalized management.
• Efficient asynchronous writing: Employs a buffer + batch-flush mechanism for high-performance, stable persistence.
• Precise semantic retrieval: Combines FAISS + JSON dual storage for fast recall and human-readable auditability.

Applicable Scenarios

• Multi-character virtual agent systems
• Long-memory AI assistants (e.g., customer service, companionship, creative co-pilots)
• Complex narrative/world-building in virtual environments
• Dialogue scenarios with strong contextual dependencies
• Video content QA and reasoning
• Multimodal agent memory management
• Long video understanding and information retrieval

---

TeleMem vs Mem0: Core Advantages

TeleMem deeply refactors Mem0 to address characterization, long-term memory, and high performance. Key differences:

---

Experimental Results

Dataset

We evaluate the ZH-4O Chinese long-character dialogue dataset constructed in the paper MOOM: Maintenance, Organization and Optimization of Memory in Ultra-Long Role-Playing Dialogues:

• Average dialogue length: 600 turns per conversation
• Scenarios: daily interactions, plot progression, evolving character relationships

Memory capability was assessed via QA benchmarks, e.g.:

{

"question": "What is Zhao Qi's nickname for Bai Yulan? A Xiaobai B Xiaoyu C Lanlan D Yuyu", "answer": "A" }, { "question": "What is the relationship between Zhao Qi and Bai Yulan? A Classmates B Teacher and student C Enemies D Neighbors", "answer": "B" }

Experimental Configuration

• LLM: Qwen3-8B (thinking mode disabled)
• Embedding model: [Qwen3-Embedding-8B](https://huggingface.co/Qwen/Qwen3-E