Engram docs

docs/engram/api-reference/index.md

@@ -0,0 +1,68 @@
+---
+title: API reference
+sidebar_position: 4
+description: "Engram REST API overview: authentication, base URL, and interactive API reference."
+---
+
+The Engram API is a REST API for storing, searching, and managing memories.
+
+## Base URL
+
+Your project's API URL is available in the [Weaviate Cloud console](https://console.weaviate.cloud). It follows the format:
+
+```
+https://your-project.engram.weaviate.cloud
+```
+
+## Authentication
+
+Include your API key in the `Authorization` header:
+
+```
+Authorization: Bearer eng_your_api_key
+```
+
+API keys are scoped to a project. All requests authenticated with a key operate within that project's scope. You can create and manage API keys in the Weaviate Cloud console.
+
+## Interactive API reference
+
+The full API reference is generated from the OpenAPI spec and includes request/response schemas, parameter details, and example payloads.
+
+**[Open the interactive API reference](/engram/api-reference/rest)**
+
+## Endpoints
+
+| Method | Path | Description |
+|--------|------|-------------|
+| POST | `/v1/memories` | Store a new memory (async) |
+| GET | `/v1/memories/{id}` | Get a memory by ID |
+| DELETE | `/v1/memories/{id}` | Delete a memory |
+| POST | `/v1/memories/search` | Search memories |
+| GET | `/v1/runs/{run_id}` | Get pipeline run status |
+
+## Error format
+
+All error responses use a consistent format:
+
+```json
+{
+  "status": 400,
+  "message": "error description"
+}
+```
+
+### HTTP status codes
+
+| Code | Description |
+|------|-------------|
+| 400 | Bad request — invalid input or missing required fields |
+| 401 | Unauthorized — missing or invalid API key |
+| 403 | Forbidden — insufficient permissions |
+| 404 | Not found — resource does not exist |
+| 500 | Internal server error |
+
+## Questions and feedback
+
+import DocsFeedback from '/_includes/docs-feedback.mdx';
+
+<DocsFeedback/>

docs/engram/concepts.md

@@ -0,0 +1,131 @@
+---
+title: Concepts
+sidebar_position: 2
+description: "Core concepts in Engram: memories, topics, groups, scoping, pipelines, retrieval types, and runs."
+---
+
+This page explains the core concepts behind Engram's memory system.
+
+## Memories
+
+A memory is a discrete piece of information stored in Engram. Each memory has:
+
+- **Content** — The text of the memory (e.g. "The user prefers dark mode").
+- **Topic** — The category it belongs to (e.g. `user_facts`, `preferences`).
+- **Group** — The memory group that defines how it was processed.
+- **Scope** — The project, user, and conversation it belongs to.
+- **Tags** — Optional string labels for additional classification.
+
+Memories are automatically embedded as vectors, making them searchable by meaning.
+
+## Topics
+
+Topics are named categories within a group. They tell Engram what kinds of information to extract and how to scope it.
+
+Each topic has:
+
+| Property | Description |
+|----------|-------------|
+| `name` | Unique identifier within the group (e.g. `user_facts`) |
+| `description` | Natural language description used in LLM prompts during extraction (e.g. "What food the user likes to eat") |
+| `scoping` | Whether the topic requires a `user_id` and/or `conversation_id` |
+| `is_bounded` | Whether the topic has size constraints |
+
+The topic `description` is important — it's what the extraction pipeline uses to decide how to categorize information. For example, a travel agent might have separate topics with descriptions like "The places the user would like to visit" and "What food the user likes to eat" so the pipeline can route extracted facts to the right topic.
+
+When you create a project, Engram sets up a default group with a default topic automatically.
+
+## Groups
+
+A group bundles a pipeline definition with one or more topics. Each project can have multiple named groups, but most use cases only need the `default` group.
+
+Groups provide:
+
+- A stable UUID identifier for the pipeline configuration
+- Topic definitions that control what gets extracted
+- Pipeline steps that define the processing flow
+- Topic name isolation — different groups can have topics with the same name without collision (e.g. two agents can each have a `user_preferences` topic in separate groups)
+
+## Scoping
+
+Engram uses a multi-level scoping system to isolate memories:
+
+- **Project** — Always required. Every memory belongs to a project, identified by the API key.
+- **User** — Required for user-scoped topics. Memories are strictly isolated between users.
+- **Conversation** — Required when storing to conversation-scoped topics. Optional when searching (see below).
+
+Which scopes are required depends on the topic configuration:
+
+### User-scoped topics
+
+User-scoped topics store memories that belong to an individual user, such as preferences or personal details. Memories are strictly isolated between users — a query for one `user_id` never returns another user's memories. Both storing and searching require the `user_id`.
+
+### Project-wide topics
+
+Topics that are not user-scoped are shared across the entire project. These are useful for procedural memory — things an agent learns about how to perform a task, regardless of which user it is working with. No `user_id` is needed for storing or searching.
+
+### Conversation-scoped topics
+
+Conversation-scoped topics associate memories with a specific conversation. When **storing**, you must provide the `conversation_id`. When **searching**, the `conversation_id` is optional:
+
+- **With `conversation_id`** — Returns only memories from that conversation (e.g. to get a summary of a specific chat).
+- **Without `conversation_id`** — Returns memories across all conversations (e.g. to find everything a user has discussed).
+
+Conversation-scoped topics are typically also user-scoped (e.g. conversation summaries are private to a user).
+
+### Multiple topics in one request
+
+A single request can interact with multiple topics. When it does, the required scope parameters are the union of each topic's requirements. For example, if one topic requires `user_id` and another requires `conversation_id`, the request must include both.
+
+## Pipelines
+
+When you send content to Engram, it runs through an asynchronous pipeline that extracts, transforms, and commits memories. Pipelines are defined as a directed acyclic graph (DAG) of steps.
+
+### Input types
+
+Engram accepts three types of input content:
+
+| Type | Description | Use case |
+|------|-------------|----------|
+| `string` | Raw text | Free-form notes, agent observations |
+| `pre_extracted` | Already-structured content | When you've done your own extraction |
+| `conversation` | Multi-turn messages with roles | Chat transcripts, agent conversations |
+
+### Pipeline steps
+
+Each pipeline processes content through a sequence of steps:
+
+1. **Extract** — Pulls structured memories from the input content. The extraction method depends on the input type (`ExtractFromString`, `ExtractFromConversation`, or `ExtractFromPreExtracted`).
+2. **Transform** — Refines extracted memories using existing context. Steps like `TransformWithContext` and `TransformOperations` deduplicate, merge, and resolve conflicts with existing memories.
+3. **Commit** — Finalizes the operations (create, update, delete) and persists them to storage.
+
+## Retrieval types
+
+Engram supports three search strategies:
+
+| Type | Description | Best for |
+|------|-------------|----------|
+| `vector` | Pure semantic search using embeddings | Finding conceptually related memories |
+| `bm25` | Full-text keyword search | Exact term matching |
+| `hybrid` | Combination of vector and BM25 | General-purpose search (recommended) |
+
+You specify the retrieval type in the `retrieval_config` when searching.
+
+## Runs
+
+Each call to store memories creates a **run** — a trackable unit of pipeline execution. Runs have four possible states:
+
+| Status | Meaning |
+|--------|---------|
+| `running` | Pipeline is actively processing |
+| `in_buffer` | Queued and waiting to start |
+| `completed` | All operations committed successfully |
+| `failed` | An error occurred during processing |
+
+When a run completes, its `committed_operations` field shows exactly which memories were created, updated, or deleted.
+
+## Questions and feedback
+
+import DocsFeedback from '/_includes/docs-feedback.mdx';
+
+<DocsFeedback/>

docs/engram/guides/check-run-status.md

@@ -0,0 +1,83 @@
+---
+title: Check run status
+sidebar_position: 4
+description: "How to poll pipeline run status and interpret committed operations in Engram."
+---
+
+When you store memories, Engram processes them asynchronously through a pipeline. Each request returns a `run_id` that you can use to track progress.
+
+## Poll a run
+
+```bash
+curl $ENGRAM_API_URL/v1/runs/{run_id} \
+  -H "Authorization: Bearer $ENGRAM_API_KEY"
+```
+
+### Response
+
+```json
+{
+  "run_id": "run-uuid",
+  "status": "completed",
+  "group_id": "group-uuid",
+  "starting_step": 0,
+  "input_type": "string",
+  "error": null,
+  "committed_operations": {
+    "created": [
+      {
+        "memory_id": "memory-uuid-1",
+        "committed_at": "2025-01-01T00:00:01Z"
+      }
+    ],
+    "updated": [],
+    "deleted": []
+  },
+  "created_at": "2025-01-01T00:00:00Z",
+  "updated_at": "2025-01-01T00:00:01Z"
+}
+```
+
+## Run statuses
+
+| Status | Meaning |
+|--------|---------|
+| `running` | Pipeline is actively processing the content |
+| `in_buffer` | Run is queued and waiting to start |
+| `completed` | All operations have been committed successfully |
+| `failed` | An error occurred during processing |
+
+## Committed operations
+
+When a run completes, the `committed_operations` field tells you exactly what changed:
+
+- **`created`** — New memories that were added to storage.
+- **`updated`** — Existing memories that were modified (e.g. merged or refined).
+- **`deleted`** — Memories that were removed (e.g. superseded by an update).
+
+Each entry includes the `memory_id` and a `committed_at` timestamp.
+
+## Handling failures
+
+If a run fails, the `error` field contains a description of what went wrong.
+
+```json
+{
+  "run_id": "run-uuid",
+  "status": "failed",
+  "error": "extraction failed: invalid input format",
+  "committed_operations": null,
+  "created_at": "2025-01-01T00:00:00Z",
+  "updated_at": "2025-01-01T00:00:01Z"
+}
+```
+
+:::tip
+For production systems, implement a polling loop that checks the run status at regular intervals (e.g. every 1-2 seconds) until the status is `completed` or `failed`.
+:::
+
+## Questions and feedback
+
+import DocsFeedback from '/_includes/docs-feedback.mdx';
+
+<DocsFeedback/>

docs/engram/guides/index.md

@@ -0,0 +1,20 @@
+---
+title: Guides
+sidebar_position: 3
+description: "Step-by-step guides for storing, searching, and managing memories in Engram."
+---
+
+These guides cover common Engram operations with detailed examples.
+
+## Available guides
+
+- **[Store memories](store-memories.md)** — Send string, pre-extracted, or conversation content to Engram.
+- **[Search memories](search-memories.md)** — Query memories using vector, BM25, or hybrid search with filtering and scoping.
+- **[Manage memories](manage-memories.md)** — Get and delete individual memories by ID.
+- **[Check run status](check-run-status.md)** — Poll pipeline runs and interpret committed operations.
+
+## Questions and feedback
+
+import DocsFeedback from '/_includes/docs-feedback.mdx';
+
+<DocsFeedback/>

docs/engram/guides/manage-memories.md

@@ -0,0 +1,64 @@
+---
+title: Manage memories
+sidebar_position: 3
+description: "How to get and delete individual memories in Engram by ID."
+---
+
+You can retrieve and delete individual memories using their ID.
+
+## Get a memory
+
+Retrieve a single memory by its ID.
+
+```bash
+curl $ENGRAM_API_URL/v1/memories/{id}?user_id={user-uuid}&topic={topic-name}&group={group-name} \
+  -H "Authorization: Bearer $ENGRAM_API_KEY"
+```
+
+### Query parameters
+
+| Parameter | Type | Description |
+|-----------|------|-------------|
+| `user_id` | string | User scope (required if the topic is user-scoped) |
+| `conversation_id` | string | Conversation scope (required if the topic is conversation-scoped) |
+| `topic` | string | The topic the memory belongs to |
+| `group` | string | The memory group name |
+
+### Response
+
+```json
+{
+  "id": "memory-uuid",
+  "project_id": "project-uuid",
+  "user_id": "user-uuid",
+  "conversation_id": null,
+  "content": "The user prefers dark mode",
+  "topic": "user_facts",
+  "group": "default",
+  "tags": ["preference", "ui"],
+  "created_at": "2025-01-01T00:00:00Z",
+  "updated_at": "2025-01-01T00:00:00Z",
+  "score": null
+}
+```
+
+## Delete a memory
+
+Remove a memory permanently by its ID.
+
+```bash
+curl -X DELETE $ENGRAM_API_URL/v1/memories/{id}?user_id={user-uuid}&topic={topic-name}&group={group-name} \
+  -H "Authorization: Bearer $ENGRAM_API_KEY"
+```
+
+The query parameters are the same as for the get request. You must provide the correct scoping parameters to identify the memory.
+
+:::warning
+Deleting a memory is permanent and cannot be undone.
+:::
+
+## Questions and feedback
+
+import DocsFeedback from '/_includes/docs-feedback.mdx';
+
+<DocsFeedback/>