# Welcome AIOS is the AI Agent Operating System, which embeds large language model (LLM) into the operating system and facilitates the development and deployment of LLM-based AI Agents. AIOS is designed to address problems (e.g., scheduling, context switch, memory management, storage management, tool management, Agent SDK management, etc.) during the development and deployment of LLM-based agents, towards a better AIOS-Agent ecosystem for agent developers and agent users. AIOS includes the AIOS Kernel and the AIOS SDK (Cerebrum). AIOS supports both Web UI and Terminal UI.

AIOS provides the AIOS kernel as an abstraction on top of the OS kernel. The kernel provides the core functions driven by AIOS system calls, to serve LLM-based agents. On top of the kernel layer, the AIOS SDK facilitates the development and deployment of agents. ### Different deployment modes of AIOS Here are some key notations that are required to know before introducing the different modes of AIOS. * **AHM (Agent Hub Machine)**: Central server that hosts the agent marketplace/repository where users can publish, download, and share agents. Acts as the distribution center for all agent-related resources. * **AUM (Agent UI Machine)**: Client machine that provides user interface for interacting with agents. Can be any device from mobile phones to desktops that supports agent visualization and control. * **ADM (Agent Development Machine)**: Development environment where agent developers write, debug and test their agents. Requires proper development tools and libraries. * **ARM (Agent Running Machine)**: Execution environment where agents actually run and perform tasks. Needs adequate computational resources for agent operations. The following parts introduce different modes of deploying AIOS. #### Mode 1 (Local Kernel Mode)

* Features: * For agent users: They can download agents from agent hub from Machine B and run agents on Machine A. * For agent developers: They can develop and test agents in Machine A and can upload agents to agent hub on Machine B. #### **Mode 2 (Remote Kernel Mode)**

* Features: * Remote use of agents: Agent users / developers can use agents on Machine B, which is different from the development and running machine (Machine A) * Benefit users who would like to use agents on resource-restricted machine (e.g., mobile device or edge device) #### **Mode 2.5 (Remote Kernel Dev Mode)**

* Features: * Remote development of agents: Agent developers can develop their agents on Machine B while running and testing their agents in Machine A. Benefit developers who would like to develop agents on resource-restricted machine (e.g., mobile device or edge device) * Critical technique: * Packaging and agent transmission on different machines for distributed agent development and testing #### Mode 3 (Personal Remote Kernel Mode)

* Features: * Each user/developer can have their personal AIOS with long-term persistent data as long as they have registered account in the AIOS ecosystem * Their personal data can be synced to different machines with the same account * Critical techniques: * User account registration and verification mechanism * Persistent personal data storage for each user's AIOS * Synchronization for different AIOS instances on different devices within the same account * Data privacy mechanism #### Mode 4 (Personal Remote Virtual Kernel Mode)

* Features: * Different user/developer’s personal AIOS kernels can co-exist in the same physical machine through virtualization * Critical techniques: * Virtualization of different AIOS kernel instances in the same machine * Scheduling and resource allocation mechanism for different virtual machines located in the same machine # Installation ### Prerequisites * Python: 3.10-3.11 * Git ### Installation #### Installation from source **Step 1: Install AIOS Kernel** Git clone AIOS kernel ```bash git clone https://github.com/agiresearch/AIOS.git ``` Create venv environment ```bash python3.x -m venv venv # Only support for Python 3.10 and 3.11 source venv/bin/activate ``` or create conda environment ```bash conda create -n venv python=3.x # Only support for Python 3.10 and 3.11 conda activate venv ``` > We strongly recommend using [uv](https://github.com/astral-sh/uv) for faster and more reliable package installation. > > To install uv: > > ``` > pip install uv > ``` **For GPU environments:** ```bash uv pip install -r requirements-cuda.txt ``` **For CPU-only environments:** ```bash uv pip install -r requirements.txt ``` Alternatively, if you prefer using pip: **For GPU environments:** ```bash pip install -r requirements-cuda.txt ``` **For CPU-only environments:** ```bash pip install -r requirements.txt ``` **Step 2: Install AIOS SDK (Cerebrum)** 1. Clone the Cerebrum repository: ```bash git clone https://github.com/agiresearch/Cerebrum.git ``` 2. Install using uv (recommended): ```bash cd Cerebrum && uv pip install -e . ``` Or using pip: ```bash cd Cerebrum && pip install -e . ``` {% hint style="success" %} **Note**: The machine where the AIOS kernel (AIOS) is installed must also have the AIOS SDK (Cerebrum) installed. Installing AIOS kernel will install the AIOS SDK automatically by default. If you are using the Local Kernel mode, i.e., you are running AIOS and agents on the same machine, then simply install both AIOS and Cerebrum on that machine. If you are using Remote Kernel mode, i.e., running AIOS on Machine 1 and running agents on Machine 2 and the agents remotely interact with the kernel, then you need to install both AIOS kernel and AIOS SDK on Machine 1, and install the AIOS SDK alone on Machine 2. {% endhint %} Then you can follow the instructions in the [Quickstart](/aios-docs/getting-started/quickstart) to launch AIOS. # Quickstart Be sure to complete the [installation instructions](/aios-docs/getting-started/installation) before continuing with this guide. Before starting AIOS, you need to make sure you have installed the LLM backends that you would like to run. Here are the LLM providers for supported backends for AIOS. | Provider 🏢 | Model Name 🤖 | Open Source 🔓 | Model String ⌨️ | Backend ⚙️ | Required API Key | | ----------- | --------------------------------------------- | -------------- | ------------------------------------- | ----------- | ------------------- | | Anthropic | Claude 3.5 Sonnet | ❌ | claude-3-5-sonnet-20241022 | anthropic | ANTHROPIC\_API\_KEY | | Anthropic | Claude 3.5 Haiku | ❌ | claude-3-5-haiku-20241022 | anthropic | ANTHROPIC\_API\_KEY | | Anthropic | Claude 3 Opus | ❌ | claude-3-opus-20240229 | anthropic | ANTHROPIC\_API\_KEY | | Anthropic | Claude 3 Sonnet | ❌ | claude-3-sonnet-20240229 | anthropic | ANTHROPIC\_API\_KEY | | Anthropic | Claude 3 Haiku | ❌ | claude-3-haiku-20240307 | anthropic | ANTHROPIC\_API\_KEY | | Deepseek | Deepseek-reasoner (R1) | ❌ | deepseek-reasoner | deepseek | DEEPSEEK\_API\_KEY | | Deepseek | Deepseek-chat (V3) | ❌ | deepseek-chat | deepseek | DEEPSEEK\_API\_KEY | | OpenAI | GPT-4 | ❌ | gpt-4 | openai | OPENAI\_API\_KEY | | OpenAI | GPT-4 Turbo | ❌ | gpt-4-turbo | openai | OPENAI\_API\_KEY | | OpenAI | GPT-4o | ❌ | gpt-4o | openai | OPENAI\_API\_KEY | | OpenAI | GPT-4o mini | ❌ | gpt-4o-mini | openai | OPENAI\_API\_KEY | | OpenAI | GPT-3.5 Turbo | ❌ | gpt-3.5-turbo | openai | OPENAI\_API\_KEY | | Google | Gemini 1.5 Flash | ❌ | gemini-1.5-flash | google | GEMINI\_API\_KEY | | Google | Gemini 1.5 Flash-8B | ❌ | gemini-1.5-flash-8b | google | GEMINI\_API\_KEY | | Google | Gemini 1.5 Pro | ❌ | gemini-1.5-pro | google | GEMINI\_API\_KEY | | Google | Gemini 1.0 Pro | ❌ | gemini-1.0-pro | google | GEMINI\_API\_KEY | | Groq | Llama 3.2 90B Vision | ✅ | llama-3.2-90b-vision-preview | groq | GROQ\_API\_KEY | | Groq | Llama 3.2 11B Vision | ✅ | llama-3.2-11b-vision-preview | groq | GROQ\_API\_KEY | | Groq | Llama 3.1 70B | ✅ | llama-3.1-70b-versatile | groq | GROQ\_API\_KEY | | Groq | Llama Guard 3 8B | ✅ | llama-guard-3-8b | groq | GROQ\_API\_KEY | | Groq | Llama 3 70B | ✅ | llama3-70b-8192 | groq | GROQ\_API\_KEY | | Groq | Llama 3 8B | ✅ | llama3-8b-8192 | groq | GROQ\_API\_KEY | | Groq | Mixtral 8x7B | ✅ | mixtral-8x7b-32768 | groq | GROQ\_API\_KEY | | Groq | Gemma 7B | ✅ | gemma-7b-it | groq | GROQ\_API\_KEY | | Groq | Gemma 2B | ✅ | gemma2-9b-it | groq | GROQ\_API\_KEY | | Groq | Llama3 Groq 70B | ✅ | llama3-groq-70b-8192-tool-use-preview | groq | GROQ\_API\_KEY | | Groq | Llama3 Groq 8B | ✅ | llama3-groq-8b-8192-tool-use-preview | groq | GROQ\_API\_KEY | | ollama | [All Models](https://ollama.com/search) | ✅ | model-name | ollama | - | | vLLM | [All Models](https://docs.vllm.ai/en/latest/) | ✅ | model-name | vllm | - | | HuggingFace | [All Models](https://huggingface.co/models/) | ✅ | model-name | huggingface | HF\_HOME | ### Configuration **Set up configuration file directly (Recommended)** You need API keys for services like OpenAI, Anthropic, Groq and HuggingFace. The simplest way to configure them is to edit the `aios/config/config.yaml`. > \[!TIP] It is important to mention that, we strongly recommend using the `aios/config/config.yaml` file to set up your API keys. This method is straightforward and helps avoid potential sychronization issues with environment variables. A simple example to set up your API keys in `aios/config/config.yaml` is shown below: ```yaml api_keys: openai: "your-openai-key" gemini: "your-gemini-key" groq: "your-groq-key" anthropic: "your-anthropic-key" huggingface: auth_token: "token to authorize specific models for use" home: "path to store downloaded model weights" ``` To obtain these API keys: 1. Deepseek API: 2. OpenAI API: 3. Google Gemini API: 4. Groq API: 5. HuggingFace Token: 6. Anthropic API: **Configure LLM Models** You can configure which LLM models to use in the same `aios/config/config.yaml` file. Here's an example configuration: ```yaml llms: models: # Ollama Models - name: "qwen2.5:7b" backend: "ollama" hostname: "http://localhost:11434" # Make sure to run ollama server # vLLM Models - name: "meta-llama/Llama-3.1-8B-Instruct" backend: "vllm" hostname: "http://localhost:8091/v1" # Make sure to run vllm server ``` **Using Ollama Models:** 1. First, download ollama from 2. Start the ollama server in a separate terminal: ```bash ollama serve ``` 3. Pull your desired models from : ```bash ollama pull qwen2.5:7b # example model ``` > Ollama supports both CPU-only and GPU environments. For more details about ollama usage, visit [ollama documentation](https://github.com/ollama/ollama) **Using vLLM Models:** 1. Install vLLM following their [installation guide](https://docs.vllm.ai/en/latest/getting_started/installation.html) 2. Start the vLLM server in a separate terminal: ```bash vllm serve meta-llama/Llama-3.1-8B-Instruct --port 8091 ``` > vLLM currently only supports Linux and GPU-enabled environments. If you don't have a compatible environment, please choose other backend options. To enable the tool calling feature of vllm, refer to **Using HuggingFace Models:** You can configure HuggingFace models with specific GPU memory allocation: ```yaml - name: "meta-llama/Llama-3.1-8B-Instruct" backend: "huggingface" max_gpu_memory: {0: "24GB", 1: "24GB"} # GPU memory allocation eval_device: "cuda:0" # Device for model evaluation ``` ### Launch AIOS After you setup the required keys, you can run the following command to launch the AIOS kernel. ```bash bash runtime/launch_kernel.sh ``` And then you can start a client to interact with the AIOS kernel using Terminal or WebUI. # Use Terminal ### Run agents and tools in the unix-like terminal List agents from agenthub ```bash list-agenthub-agents ``` List agents from local ```bash list-local-agents ``` Download agents ```bash download-agent \ --agent_author \ --agent_name \ --agent_version \ --agenthub_url ``` Upload agent ```bash upload-agents \ --agent_path \ --agenthub_url ``` List tools from toolhub ```bash list-toolhub-tools ``` List tools from local ```bash list-local-tools ``` Download tool ```bash download-tool \ --tool_author \ --tool_name \ --tool_version \ --toolhub_url ``` Upload tool ```bash upload-tool \ --tool_path \ --toolhub_url ``` ### Interact with AIOS terminal UI Before running the AIOS terminal, please make sure you have read the instructions in [AIOS](https://github.com/agiresearch/AIOS) to set up the environment and launch the AIOS kernel. And you need to create a folder to put the files you want to manage, the default folder is `./root` After that, you can run the following command to start the AIOS terminal. ``` python scripts/run_terminal.py ``` When you successfully enter the system, the interface is as follows:

Then you can start interacting with the AIOS terminal by typing natural language commands. For instructions of how to use commands, refer to . # Use WebUI ## Chat Interface Guide > 🚀 The WebUI demo is now fully functional and live at ### Quick Start 1. Pick a model * Choose between Gemini or GPT * Look for the model selector at the top 2. Add your API key * Click "Configure API Key" when prompted * Enter your key in the popup window 3. View an agent * Click an agent from the left sidebar * Each agent has special skills 4. Start chatting! * @ an agent to select the agent * Type your message * Press Enter to send and chat with the selected agent ### Tips & Tricks 💡 **Quick Commands** * Use `@agent_name` to talk to specific agent * Example: `@math_agent what is 2+2?` 🔄 **Managing Chats** * Click "New Chat" for fresh conversation * All chats save automatically * Search old chats in right sidebar ### Common Issues ⚠️ **No API Key?** * Get Gemini key from Google AI Studio * Get GPT key from OpenAI

# Environment Variables Configuration The configuration file can be found at the following relative path in your installation directory of AIOS: ```arduino aios/config/config.yaml ``` AIOS supports several API integrations that require configuration. You can use the following commands: * `aios env list`: Show current environment variables, or show available API keys if no variables are set * `aios env set`: Show current environment variables, or show available API keys if no variables are set * `aios refresh`: Refresh AIOS configuration. * Reloads the configuration from `aios/config/config.yaml`. * Reinitializes all components without restarting the server. * The server must be running. When no environment variables are set, the following API keys will be shown: * `OPENAI_API_KEY`: OpenAI API key for accessing OpenAI services * `GEMINI_API_KEY`: Google Gemini API key for accessing Google's Gemini services * `DEEPSEEK_API_KEY`: Deepseek API key for accessing Deepseek services * `ANTHROPIC_API_KEY`: Anthropic API key for accessing Anthropic Claude services * `GROQ_API_KEY`: Groq API key for accessing Groq services * `HF_AUTH_TOKEN`: HuggingFace authentication token for accessing models * `HF_HOME`: Optional path to store HuggingFace models To obtain these API keys: 1. OpenAI API: Visit 2. Google Gemini API: Visit 3. Deepseek API: Visit 4. Anthropic Claude API: Visit 5. Groq API: Visit 6. HuggingFace Token: Visit **API Keys**: ```yaml openai: "your-openai-key" gemini: "your-gemini-key" deepseek: "your-deepseek-key" groq: "your-groq-key" anthropic: "your-anthropic-key" huggingface: auth_token: "your-huggingface-token" cache: "optional-path" ``` **Model Settings**: {% hint style="info" %} It is required to follow the following parameters to set up different backends as below {% endhint %}

Backend Type	Required Parameters
openai	`name`, `backend`
anthropic	`name`, `backend`
google	`name`, `backend`
ollama	`name`, `backend`, `host_name`
vLLM	`name`, `backend`, `host_name`
huggingface	`name`, `backend`, `max_gpu_memory`, `eval_device`

The example of how to set up different models are as below ```yaml # LLM Configuration llms: models: # OpenAI backend # - name: "gpt-4o-mini" # backend: "openai" # Google Models # - name: "gemini-1.5-flash" # backend: "google" # Anthropic Models # - name: "claude-3-opus" # backend: "anthropic" # Ollama backend # - name: "qwen2.5:7b" # backend: "ollama" # hostname: "http://localhost:11434" # Make sure to run ollama server # HuggingFace backend # - name: "meta-llama/Llama-3.1-8B-Instruct" # backend: "huggingface" # max_gpu_memory: {0: "48GB"} # GPU memory allocation # eval_device: "cuda:0" # Device for model evaluation # vLLM Models # To use vllm as backend, you need to install vllm and run the vllm server https://docs.vllm.ai/en/latest/serving/openai_compatible_server.html # An example command to run the vllm server is: # vllm serve meta-llama/Llama-3.2-3B-Instruct --port 8091 # - name: "meta-llama/Llama-3.1-8B-Instruct" # backend: "vllm" # hostname: "http://localhost:8091" log_mode: "console" use_context_manager: false ``` **Memory Settings**: ```yaml log_mode: "console" # can be "console" or "file" ``` **Storage Settings**: ```yaml root_dir: "root" use_vector_db: true ``` **Scheduler Settings**: ```yaml log_mode: "console" # can be "console" or "file" ``` # Overview Before diving deep into the kernel functions, this page provides an overview of the kernel design to illustrate how the different modules in the AIOS kernel interact with each other. The AIOS Kernel comprises several key modules, each playing a specialized role to facilitate and manage agent requests effectively.

Details of how agents leverage SDK to interact with AIOS kernel and how AIOS kernel leverages syscalls in different modules to achieve functionalties

### **LLM Core(s)** The [LLM core](#llm-core-s) wraps the deployed instances of LLM from different providers in a unified manner, treating each LLM as a core similar to CPU cores. This abstraction allows multiple LLMs to be integrated via a single interface, enabling flexible and efficient usage of different LLM models. ### **Context Manager** The [Context Manager](#context-manager) is responsible for handling context management (such as context switch) during the processing of each LLM Core. In the LLM agent scenario, "context" refers to the data that is actively fed into each LLM Core during processing. ### **Memory Manager** The [Memory Manager](#memory-manager) handles data storage in RAM that agents need during runtime, distinguishing it from persistent storage. "Memory" here refers to transient data necessary for ongoing operations, such as temporary variables or recently processed data. It supports memory operations required by agents to store and retrieve information quickly during active tasks. ### **Storage Manager** The [Storage Manager](#storage-manager) is designed for handling persistent storage operations, where "storage" refers to data saved on disk that agents may require for future tasks or for logging purposes. ### Tool Manager The [tool manager](/aios-docs/aios-kernel/tools) loads and manages various tools available within the AIOS-Agent SDK. This module manages tool call conflicts and prioritizes tool operations to ensure the stable execution of tool calls. #### Key Differences among Context, Memory, and Storage {% hint style="success" %} **Context**: Data that is fed into the LLM core and is under processing by the LLM core. **Memory**: Transient data stored in RAM, which agents require temporarily during active tasks or sessions. Memory is cleared after tasks complete. **Storage**: Persistent data saved on disk, intended for long-term use or logging, accessible by agents even after a session ends. {% endhint %} # LLM Core(s) AIOS wraps different LLMs as LLM Cores to support the unified interface for addressing requests to both cloud LLM apis and locally-hosted LLMs. AIOS supports three main categories of backends: 1. LiteLLM compatible backends (cloud and local) 2. vLLM backends (local) 3. Huggingface backends (local) Each backend type handles different input scenarios, including standard text generation, tool calling, and JSON-formatted responses. This document explains how each backend processes these inputs and the implementation details. | Feature | LiteLLM Compatible | vLLM | Huggingface | | -------------- | ---------------------------------- | ---------------------------------- | ---------------------------------------- | | Standard Input | Uses completion function | Uses OpenAI client | Uses generate method | | Tool Calls | Native support via tools parameter | Native support via tools parameter | Uses message merging and custom decoding | | JSON Responses | Uses format="json" | Uses format="json" | Uses message merging | Different backend details are as below * [Litellm compatible backends](/aios-docs/aios-kernel/llm-cores/litellm-compatible-backend) * [vLLM backends](/aios-docs/aios-kernel/llm-cores/vllm-backend) * [Huggingface local backends](/aios-docs/aios-kernel/llm-cores/hugging-face-backend) # LiteLLM Compatible Backend LiteLLM is a unified interface for various LLM providers, offering a consistent API across different model providers. Supported model providers can be found at . LiteLLM compatible backends are initialized by using string identifiers for models in the format `{provider}/{model}`, such as: * `openai/gpt-4o-mini` * `anthropic/claude-3.5-sonnet` * `gemini/gemini-1.5-flash` In the code, these models are initialized and stored as string identifiers in the `llms` array: ```python # During initialization if config.backend == "google": config.backend = "gemini" # Convert backend name for compatibility prefix = f"{config.backend}/" if not config.name.startswith(prefix): self.llms.append(prefix + config.name) ``` **Standard Text Input** For standard text generation, LiteLLM backends process requests using the `completion()` function: ```python completion_kwargs = { "messages": messages, "temperature": temperature, "max_tokens": max_tokens } completed_response = completion(model=model, **completion_kwargs) return completed_response.choices[0].message.content, True ``` The system passes the messages array, temperature, and max\_tokens parameters to the completion function. **Tool Calls** When processing requests with tools, the tool definitions are added to the completion parameters: ```python if tools: tools = slash_to_double_underscore(tools) # Prevent invalid tool string completion_kwargs["tools"] = tools completed_response = completion(model=model, **completion_kwargs) completed_response = decode_litellm_tool_calls(completed_response) return completed_response, True ``` {% hint style="warning" %} Note that in some providers such as OpenAI and Anthropic, tool names can not contain "/" character when it is passed. Therefore, it is required to use [`slash_to_double_underscore`](https://github.com/agiresearch/AIOS/blob/main/aios/llm_core/utils.py) function to convert tools {% endhint %} The [`decode_litellm_tool_calls()`](https://github.com/agiresearch/AIOS/blob/main/aios/llm_core/utils.py) function processes the raw response to extract and format the tool calls. **JSON-Formatted Responses** For JSON-formatted responses, the adapter adds the format parameter: ```python if message_return_type == "json": completion_kwargs["format"] = "json" if response_format: completion_kwargs["response_format"] = response_format ``` # vLLM Backend [Source code](https://github.com/agiresearch/AIOS/blob/main/aios/llm_core/adapter.py) vLLM backends are handled using the OpenAI client class due to compatibility issues with LiteLLM. These backends are initialized as OpenAI client instances with a custom base URL: ```python case "vllm": self.llms.append(OpenAI( base_url=config.hostname, api_key="sk-1234" # Dummy API key )) case "sglang": self.llms.append(OpenAI( base_url=config.hostname, api_key="sk-1234" # Dummy API key )) ``` {% hint style="warning" %} It is important to note that a dummy API key ("sk-1234") is required to set up the OpenAI hosted client since these backends typically don't require authentication when run locally. {% endhint %} **Standard Text Input** For standard text input, the OpenAI client is used directly: ```python completed_response = model.chat.completions.create( model=model_name, **completion_kwargs ) return completed_response.choices[0].message.content, True ``` **Tool Calls** When processing tool calls with OpenAI client-based backends: ```python # Add tools to completion parameters if tools: completion_kwargs["tools"] = tools completed_response = model.chat.completions.create( model=model_name, **completion_kwargs ) if tools: completed_response = decode_litellm_tool_calls(completed_response) return completed_response, True ``` The response is processed using the same [`decode_litellm_tool_calls`](https://github.com/agiresearch/AIOS/blob/main/aios/llm_core/utils.py) function as for LiteLLM backends. **JSON-Formatted Responses** JSON formatting uses the same approach as standard OpenAI clients: ```python if message_return_type == "json": completion_kwargs["format"] = "json" if response_format: completion_kwargs["response_format"] = response_format ``` # Hugging Face Backend The Hugging Face Local Backend allows running models locally using the Hugging Face Transformers library. The HF Local Backend is initialized as a class instance: ```python case "huggingface": self.llms.append(HfLocalBackend( model_name=config.name, max_gpu_memory=config.max_gpu_memory, eval_device=config.eval_device )) ``` It handles loading and running Hugging Face models locally, with options for GPU memory allocation. **Standard Text Input** For standard text requests, the backend uses the `generate()` method: ```python completed_response = model.generate(**completion_kwargs) return completed_response, True ``` **Tool Calls** As huggingface models do not natively support tool calls, the adapter merges tool information into messages before generation and decodes tool calls after generation. ```python if tools: new_messages = merge_messages_with_tools(messages, tools) completion_kwargs["messages"] = new_messages completed_response = model.generate(**completion_kwargs) # During processing if tools: if isinstance(model, HfLocalBackend): if finished: tool_calls = decode_hf_tool_calls(completed_response) tool_calls = double_underscore_to_slash(tool_calls) return LLMResponse( response_message=None, tool_calls=tool_calls, finished=finished ) ``` The [`merge_messages_with_tools()`](https://github.com/agiresearch/AIOS/blob/main/aios/llm_core/utils.py) function formats the tool information into the prompt, and [`decode_hf_tool_calls()`](https://github.com/agiresearch/AIOS/blob/main/aios/llm_core/utils.py) extracts tool calls from the text response. **JSON-Formatted Responses** JSON formatting is handled by merging the response format into the messages: ```python elif message_return_type == "json": new_messages = merge_messages_with_response_format(messages, response_format) completion_kwargs["messages"] = new_messages ``` The [`merge_messages_with_response_format()`](https://github.com/agiresearch/AIOS/blob/main/aios/llm_core/utils.py) function likely adds instructions for the model to respond in JSON format. # LLM Routing When agents send LLM requests to AIOS kernel, the agent can choose one LLM backend or choose multiple LLM backends. 1) If only one LLM backend is specified, then the request will be sent to the specified LLM; 2) If multiple LLM backends are specified, it means that the agent allows this request to be processed by any of the specified LLMs. In this case, AIOS provides two LLM routing strategies — Sequential Routing and Smart Routing — to decide which LLM among the specified LLMs will be chosen to process the request. ### Overview of Routing Strategies AIOS provides two routing strategies to distribute requests across multiple LLM backends: | Strategy Type | Description | | ---------------------- | ------------------------------------------------------------------------------------------------------------------------------- | | **Sequential Routing** | Sequentially cycles through the available models and selects an available one that is within the specific models | | **Smart Routing** | A cost-quality optimized strategy which smartly chooses the lowest cost LLM while maintaining the quality of request processing | ### Sequential Routing The `SequentialRouting` implements a basic model selection approach for load-balancing LLM requests. It sequentially cycles through the available models and selects an available models one that is within the specified models. #### Core Functions ```python def get_model_idxs(self, selected_llms: List[str], n_queries: int=1): model_idxs = [] for _ in range(n_queries): current = selected_llms[self.idx] for i, llm_config in enumerate(self.llm_configs): if llm_config["name"] == current["name"]: model_idxs.append(i) break self.idx = (self.idx + 1) % len(selected_llms) return model_idxs ``` ### Smart Routing The `SmartRouting` implements a sophisticated cost-quality optimized selection strategy for LLM requests, using historical performance data to predict which models will perform best for a given query while minimizing cost. It leverages a two-stage constrained optimization method. The figure below shows the overall pipeline of this smart routing strategy.

**Optimization Methods** * Uses Lagrangian dual optimization to globally optimize model selection * Balances overall performance against total cost * Ensures each query is assigned to exactly one model ```python def get_model_idxs(self, selected_llms, queries, input_token_lengths): model_idxs = [] for i in range(len(queries)): query = queries[i] input_token_length = input_token_lengths[i] # Get performance and length predictions perf_scores, length_scores = self.store.predict( query, selected_llms, n_similar=self.n_similar ) # Calculate cost scores cost_scores = [] for j in range(len(selected_llms)): pred_output_length = length_scores[0][j] input_cost = input_token_length * selected_llms[j].get("cost_per_input_token", 0) output_cost = pred_output_length * selected_llms[j].get("cost_per_output_token", 0) weighted_score = input_cost + output_cost cost_scores.append(weighted_score) # Select optimal model selected_idx = self.optimize_model_selection( selected_llms, perf_scores[0], np.array(cost_scores) ) # Find index in original llm_configs for idx, config in enumerate(self.llm_configs): if config["name"] == selected_llms[selected_idx]["name"]: model_idxs.append(idx) break return model_idxs ``` #### Usage Example ```python configs = [ {"name": "gpt-4o-mini", "backend": "openai", "cost_per_input_token": 0.00005, "cost_per_output_token": 0.00015}, {"name": "llama-3-8b", "backend": "ollama", "cost_per_input_token": 0.00001, "cost_per_output_token": 0.00002} ] strategy = SmartRouting( llm_configs=configs, performance_requirement=0.7, n_similar=16 ) queries = ["What is the capital of France?", "Explain quantum computing"] token_lengths = [8, 12] model_idxs = strategy.get_model_idxs(configs, queries, token_lengths) ``` For implementation details and experimental results, see our [official repository](https://github.com/agiresearch/ECCOS) and [research paper](https://arxiv.org/abs/2502.20576). ### Reference ``` @article{mei2025eccos, title={ECCOS: Efficient Capability and Cost Coordinated Scheduling for Multi-LLM Serving}, author={Mei, Kai and Xu, Wujiang and Lin, Shuhang and Zhang, Yongfeng}, journal={arXiv:2502.20576}, year={2025} } ``` # Scheduler Instead of placing separate queues within each processing module (e.g., LLM core(s) or memory manager), we centralize all the queues within the scheduler module. This aims to isolate the responsibility for request management from the individual modules, allowing each processing module to focus on its execution and simplifies the coordination of tasks across modules.

AIOS Scheduler manages the Request queue for each module, the Request queue includes both Queries (agent-to-kernel request) and Messages (module-to-module request)

The main purpose of the scheduler is to serve different agents queries in a fine-grained granularity (i.e., syscall level granuity) as below.

Scheduling example for syscalls for different agents (use LLM Syscall for illustration simplicity)

AIOS provides two main scheduler implementations to manage system calls from agents: | Feature | FIFOScheduler | RRScheduler | | ---------------------- | ------------------------------ | ---------------------------------- | | **Task Selection** | First-come, first-served | Round-robin rotation | | **Time Management** | No time limit enforcement | Fixed time slices per task | | **Task Preemption** | None (tasks run to completion) | Tasks can be suspended and resumed | | **Status Handling** | Only "executing"/"done" | "executing"/"done"/"suspending" | | **Context Management** | None | Uses SimpleContextManager | ### BaseScheduler The `BaseScheduler` is an abstract base class that defines the common interface and functionality for all scheduler implementations in AIOS. #### Abstract Methods | Method | Description | | ---------------------------- | ---------------------------------------------- | | `start_processing_threads()` | Start threads for different request processors | | `stop_processing_threads()` | Stop all processing threads gracefully | | `process_llm_requests()` | Abstract method to process LLM requests | | `process_memory_requests()` | Abstract method to process Memory requests | | `process_storage_requests()` | Abstract method to process Storage requests | | `process_tool_requests()` | Abstract method to process Tool requests | | `start()` | Abstract method to start the scheduler | | `stop()` | Abstract method to stop the scheduler | ### Different scheduler implementations * [FIFO Scheduler](/aios-docs/aios-kernel/scheduler/fifoscheduler) * [RR Scheduler](/aios-docs/aios-kernel/scheduler/rrscheduler) # FIFOScheduler The FIFO (First-In-First-Out) scheduler processes tasks in the order they arrive, with a 1-second timeout for each task. [Source code](https://github.com/agiresearch/AIOS/blob/main/aios/scheduler/fifo_scheduler.py) #### Core Functions ```python def _execute_syscall(self, syscall, executor, syscall_type): try: syscall.set_status("executing") self.logger.log(f"{syscall.agent_name} is executing {syscall_type} syscall.\n", "executing") syscall.set_start_time(time.time()) response = executor(syscall) syscall.set_response(response) syscall.event.set() syscall.set_status("done") syscall.set_end_time(time.time()) self.logger.log( f"Completed {syscall_type} syscall for {syscall.agent_name}. Thread ID: {syscall.get_pid()}\n", "done" ) return response # Error handling omitted for brevity ``` # RRScheduler ### RRScheduler The Round Robin scheduler ensures fair distribution of processing time among tasks by limiting each task's execution time and cycling through all tasks. #### Key Features * Configurable time slice per task (default: 1 second) * Fair CPU time distribution * Support for suspending and resuming long-running tasks * Preemptive scheduling #### Core Functions ```python def _execute_syscall(self, syscall, executor, syscall_type): try: syscall.set_time_limit(self.time_slice) syscall.set_status("executing") self.logger.log(f"{syscall.agent_name} is executing {syscall_type} syscall.\n", "executing") syscall.set_start_time(time.time()) response = executor(syscall) syscall.set_response(response) if response.finished: syscall.set_status("done") log_status = "done" else: syscall.set_status("suspending") log_status = "suspending" syscall.set_end_time(time.time()) syscall.event.set() self.logger.log( f"{syscall_type} syscall for {syscall.agent_name} is {log_status}. Thread ID: {syscall.get_pid()}\n", log_status ) return response # Error handling omitted for brevity ``` # Context The LLM generation process consists of two key stages: the **prefilling stage** and the **decoding stage**. 1. **Prefilling Stage**: During this stage, the LLM computes the key-value (KV) caches, which store intermediate results that can be reused during the subsequent decoding stage. 2. **Decoding Stage**: In this stage, the LLM generates tokens progressively, often working in a regressive manner to predict the next token based on the previously generated ones.

The `SimpleContextManager` class provides a practical implementation of context management for different types of LLM backends. #### Key Features * Support for different model types (API-based, OpenAI client, HuggingFace local) * Time limit enforcement for generations * Context saving and restoration * Handling of special input types (tools, JSON formatting) #### Context Switching Approaches The implementation supports two primary approaches to context switching: | Approach | Description | Compatibility | Implementation | | ------------------------------- | ------------------------------------------------ | --------------------------------------------------- | -------------------------------------------------------------------- | | **Text-Based Context Switch** | Stores generated text as intermediate results | Compatible with API-based models and OpenAI clients | Uses streaming responses with partial content accumulation | | **Logits-Based Context Switch** | Stores model state including KV cache and logits | Currently only supported for HuggingFace models | Saves generated tokens, past key values, and other state information | **Text-Based Context Switching** For API-based models and OpenAI clients, the context manager uses a streaming approach: ```python def process_completion_streaming_response(self, response, initial_content, time_limit): start_time = time.time() completed_response = initial_content finished = True for part in response: delta_content = part.choices[0].delta.content or "" completed_response += delta_content if time.time() - start_time > time_limit: if part.choices[0].finish_reason is None: finished = False break if not finished: self.context_dict[str(pid)] = completed_response else: self.clear_context(str(pid)) return completed_response, finished ``` This approach: * Accumulates partial content from streamed responses * Enforces time limits by breaking out of processing loop * Returns both the accumulated text and a completion status **Logits-Based Context Switching** For HuggingFace models, the context manager implements a more sophisticated approach: ```python def generate_with_time_limit_hf(self, model, messages, max_tokens, temperature, pid, time_limit): context_data = self.load_context(pid) if context_data: # Restore previous state start_idx = context_data["start_idx"] generated_tokens = context_data["generated_tokens"] past_key_values = context_data["past_key_values"] input_length = context_data["input_length"] else: # Initialize new generation # [Initialization code...] start_time = time.time() finished = True # Generate tokens incrementally with time checking for i in range(start_idx, max_tokens): if time.time() - start_time > time_limit: finished = False break # [Token generation code...] # Store state if not finished if not finished: self.context_dict[str(pid)] = { "generated_tokens": generated_tokens, "past_key_values": past_key_values, "start_idx": start_idx, "input_length": input_length } else: self.clear_context(str(pid)) return result, finished ``` This approach: * Saves and restores the full model state (generation state, position counters) * Continues generation from the exact point it was interrupted * Provides more efficient resumption as it doesn't repeat computation # Memory ### Two-Tier Memory Architecture for Agents The system employs a two-tier memory architecture consisting of: 1. **Memory Manager (RAM-based):** Handles transient, session-specific data for active agents. Memory is cleared when the agent session ends. 2. **Storage Manager (Persistent Storage):** Ensures long-term storage for evicted or overflowed memory blocks. ### **Memory Manager** The Memory Manager is responsible for managing temporary memory stored in RAM during an agent's active tasks or sessions. **Components** * **Compressed Block:** Primary unit of memory, containing interaction history in a compressed format. * **Memory Limit:** Defines the maximum amount of memory an agent can use in RAM. * **Eviction Strategy:** Implements a policy to evict blocks to persistent storage when memory usage exceeds the limit, ensuring efficient memory management.

How data swap occurs between memory and storage manager

**Memory Flow** 1. Interaction data is compressed into blocks and stored in the agent's memory. 2. Each agent's memory operates within a predefined limit. 3. When the accumulated memory reaches the limit: * Blocks are evicted based on the eviction strategy. * Evicted blocks are moved to the Storage Manager for persistent storage. #### Key Modules | Module | Purpose | Key Classes | | --------------------- | ----------------------- | --------------------------------------------- | | **Memory Processing** | Core memory operations | `BaseMemoryManager`, `MemoryNote` | | **Vector Retrieval** | Semantic search backend | `ChromaRetriever`, `SimpleEmbeddingRetriever` | | **Content Analysis** | Cognitive processing | `execute_memory_content_analyze` | | **Memory Evolution** | Autonomous adaptation | `execute_memory_evolve` | #### **TODO** * **Privacy Mechanism:** Add a system to restrict agents from accessing the memory of other agents, ensuring data privacy. * **Memory Optimization:** Improve the structure and efficiency of the memory management system. # Base Layer #### BaseMemoryManager Class **Core Functionality**: ```python class BaseMemoryManager: def add_memory(self, memory_note): ... def remove_memory(self, memory_id): ... def update_memory(self, memory_note): ... def get_memory(self, memory_id): ... def retrieve_memory(self, query): ... ``` **Key Features**: * Dual storage system (in-memory + ChromaDB) * Atomic memory operations * Automatic metadata synchronization * Thread-safe access patterns #### Memory Metadata Handling **Metadata Structure**: ```json { "context": "research/biology", "keywords": ["protein_folding", "AI_models"], "tags": ["scientific_breakthrough", "ML"], "category": "scientific_discovery", "timestamp": "202403151430" } ``` # Agentic Memory Operations | **System Behavior** | **Evolution Operation** | | ------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Automatic Merging of Highly Similar Memories** | Identifies memories with content/semantic duplication exceeding thresholds, generates unified memory nodes, and establishes knowledge graph associations. | | **Dynamic Context Association Updates** | Detects context drift, reconstructs semantic networks, and optimizes cross-scenario context anchors. | | **Intelligent Tag Unification** | Analyze tag semantic overlap, create standardized tag systems, and build multi-level classification indices. | | **Autonomous Linking** | Recognizes temporal patterns and constructs chronological memory chains. | **Execution Flow**: 1. Input sanitization 2. LLM-based semantic parsing 3. JSON schema validation 4. Error recovery mechanisms **Example Transformation**:json ```json # Input "Discovered new protein folding mechanism using AlphaFold2" # Output { "keywords": ["protein_folding", "AlphaFold2", "biochemical_discovery"], "context": "Scientific discovery in computational biology", "tags": ["AI_research", "biochemistry", "machine_learning"] } ``` **Evolution JSON Schema**: ```json { "should_evolve": true, "evolution_type": ["merge", "context_update"], "affected_memories": ["mem_123", "mem_456"], "evolution_details": { "new_context": "Advanced protein folding techniques", "new_relationships": ["ML_models", "drug_discovery"] } } ``` # Storage AIOS Storage Manager The storage manager in AIOS is built powered by the LLM-based semantic file system, which is published in ICLR2025, with its paper can be found at [From Commands to Prompts: LLM-based Semantic File System for AIOS](https://arxiv.org/pdf/2410.11843). ```python class StorageManager: def __init__(self, root_dir, use_vector_db=True, filesystem_type="lsfs"): self.use_vector_db = use_vector_db self.filesystem_type = filesystem_type self.root_dir = root_dir os.makedirs(self.root_dir, exist_ok=True) if filesystem_type == "lsfs": self.filesystem = LSFS(root_dir, use_vector_db) def address_request(self, agent_request): return self.filesystem.address_request(agent_request) ``` By default, the storage manager integrates the LLM-based semantic file system as its file system, which supports prompt-driven file management. Its original implementation can be found at . Available system calls are shown as below * [sto\_mount](/aios-docs/aios-kernel/storage/sto_mount) * [sto\_create\_file](/aios-docs/aios-kernel/storage/sto_create_file) * [sto\_create\_directory](/aios-docs/aios-kernel/storage/sto_create_directory) * [sto\_write](/aios-docs/aios-kernel/storage/sto_write) * [sto\_retrieve](/aios-docs/aios-kernel/storage/sto_retrieve) * [sto\_rollback](/aios-docs/aios-kernel/storage/sto_rollback) * [sto\_share](/aios-docs/aios-kernel/storage/sto_share)

@inproceedings{
    shi2025from,
    title={From Commands to Prompts: {LLM}-based Semantic File System for AIOS},
    author={Zeru Shi and Kai Mei and Mingyu Jin and Yongye Su and Chaoji Zuo and Wenyue Hua and Wujiang Xu and Yujie Ren and Zirui Liu and Mengnan Du and Dong Deng and Yongfeng Zhang},
    booktitle={The Thirteenth International Conference on Learning Representations},
    year={2025},
    url={https://openreview.net/forum?id=2G021ZqUEZ}
}

# sto\_mount ```python def sto_mount(collection_name: str, root_dir: str) -> str ``` [Source code](https://github.com/agiresearch/AIOS/blob/main/aios/storage/filesystem/lsfs.py) Description: Mount a specific directory for the filesystem Parameters: * collection\_name (str): Name of the collection/agent * root\_dir (str): Root directory to mount Returns: * Success message (str): "File system mounted successfully for agent: {collection\_name}" * Error message (str): "Error mounting file system: {error\_message}" # sto\_create\_file ```python def sto_create_file(file_name: str, file_path: str, collection_name: str = None) -> str ``` [Source code](https://github.com/agiresearch/AIOS/blob/main/aios/storage/filesystem/lsfs.py) Description: Create a file based on either file\_name or file\_path * If the file\_path is provided, it will create the file directly at the location of the file\_path * Else If the file\_name is provided, the file will be created at the path of os.path.join(self.root\_dir, file\_name) Parameters: * file\_name (str): Name of the file to create * file\_path (str): Full path where to create the file * collection\_name (str): Name of the collection/agent (optional) Returns: * Success message (str): "File has been created successfully at: {file\_path}" * Exists message (str): "File already exists at: {file\_path}" * Error message (str): "Error creating file: {error\_message}" # sto\_create\_directory ```python def sto_create_directory(self, dir_name: str, dir_path: str, collection_name: str = None) -> str ``` [Source code](https://github.com/agiresearch/AIOS/blob/main/aios/storage/filesystem/lsfs.py) Description: Create a directory based on either dir\_name or dir\_path * If the dir\_path is provided, it will create the file directly at the location of the dir\_path * Else If the dir\_name is provided, the directory will be created at the path of os.path.join(self.root\_dir, dir\_name) Parameters: * dir\_name (str): Name of the file to create * dir\_path (str): Full path where to create the file * collection\_name (str): Name of the collection/agent (optional) Returns: * Success message (str): "Directory has been created successfully at: {dir\_path}" * Exists message (str): "Directory already exists at: {dir\_path}" * Error message (str): "Error creating directory: {error\_message}" # sto\_write ```python def sto_write(file_name: str, file_path: str, content: str, collection_name: str = None) -> str ``` [Source code](https://github.com/agiresearch/AIOS/blob/main/aios/storage/filesystem/lsfs.py) Description: Write specific content into a file Parameters: * file\_name (str): Name of the file * file\_path (str): Full path to the file (optional) * content (str): Content to write * collection\_name (str): Name of the collection/agent (optional) Returns: * Success message (str): "Content has been written to file: {file\_path}" * Lock timeout message (str): "Timeout waiting for lock on {file\_path}" * Error message (str): "Error writing to file: {error\_message}" # sto\_retrieve ```python def sto_retrieve(collection_name: str, query_text: str, k: str = 3, keywords: str = None) -> list ``` [Source code](https://github.com/agiresearch/AIOS/blob/main/aios/storage/filesystem/lsfs.py) Descriptions: Retrieve relevant files based on either the query\_text or keywords Parameters: * collection\_name (str): Name of the collection/agent * query\_text (str): Text to search for * k (int): Number of results to return (default: 3) * keywords (str): Additional keywords for filtering (optional) Returns: * Successful retrieval (List): Matched Documents * Failed retrieval: Empty List # sto\_rollback ```python def sto_rollback(file_path: str, n: int = 1, time: str = None) -> str ``` [Source code](https://github.com/agiresearch/AIOS/blob/main/aios/storage/filesystem/lsfs.py) Descriptions: Rollback files to a specific version Parameters: * file\_path (str): Path to the file * n (int): Number of versions to roll back (default: 1) * time (str): ISO format timestamp to roll back to (optional) Returns: * Success message (str): "Successfully rolled back the file: {file\_path} to its previous {n} version" * Failure message (str): "Failed to roll back the file: {file\_path} to its previous {n} version" * Redis error (str): "Redis is not enabled. Please make sure the redis server has been installed and running." # sto\_share ```python def sto_share(file_path: str, collection_name: str = None) -> dict ``` [Source code](https://github.com/agiresearch/AIOS/blob/main/aios/storage/filesystem/lsfs.py) Parameters: * file\_path (str): Path to the file to share * collection\_name (str): Name of the collection/agent (optional) Returns: Success response (Dict): ```python { "file_name": str, # Name of the shared file "file_path": str, # Path to the file "share_link": str, # Public URL for accessing the file "expires_in": str, # "7 days" "last_modified": str # ISO format timestamp } ``` Failure response (Dict): ```python { "error": str # Error message } ``` # Tools AIOS Tool Manager The AIOS Tool Manager serves as a central hub for handling tool executions within the kernel. It provides a structured way to load and execute these tools while preventing conflicts. ### How Tool Execution Works 1. **Receives the tool request** with specific parameters 2. **Prevents tool conflicts** by tracking active tool executions 3. **Loads the appropriate tool instance** on demand 4. **Executes the tool** with the provided parameters 5. **Returns the result** back to the requesting component The Tool Manager implements a key method called `address_request` that handles incoming tool requests: ```python def address_request(self, syscall) -> None: tool_calls = syscall.tool_calls try: for tool_call in tool_calls: tool_org_and_name, tool_params = ( tool_call["name"], tool_call["parameters"], ) # Check if the tool is already being used to prevent conflicts if tool_org_and_name not in self.tool_conflict_map.keys(): # Mark the tool as in-use self.tool_conflict_map[tool_org_and_name] = 1 # Load the tool instance dynamically tool = self.load_tool_instance(tool_org_and_name) # Execute the tool with the provided parameters tool_result = tool.run(params=tool_params) # Release the tool for other uses self.tool_conflict_map.pop(tool_org_and_name) return Response( response_message=tool_result, finished=True ) except Exception as e: return Response( response_message=f"Tool calling error: {e}", finished=True ) ``` ### Dynamic Tool Loading One of the most powerful features of the Tool Manager is its ability to dynamically load tool instances when needed: ```python def load_tool_instance(self, tool_org_and_name): # Create a new instance of the requested tool tool_instance = AutoTool.from_preloaded(tool_org_and_name) return tool_instance ``` Details of how to use AutoTool can be found in . # Access AIOS Access Control Mechanism The access manager in the AIOS kernel aims to provide the following two key functionalities. ### Access Control The access manager controls read and write permissions when some agents request read and write operation of other agent's data such as memory. The access manager achieves this by assigning each agent to a specific privilege group. Agents can only access resources, such as LLM interaction history or tool usage logs of other agents only if they are within other agents' privilege group. To manage these permissions, the access manager uses a hashmap, where each agent's ID is mapped to its assigned privilege group. When an agent requests access to a resource, the access manager checks the hashmap to verify the agent’s permissions before allowing the request to proceed. ### User Intervention To prevent accidental or unintended operations when users interact with AIOS, particularly irreversible operations such as deletion, overwrite and privilege change, we provide a user intervention interface. This interface provides users with prompt checks to confirm before these irreversible operations are executed over files or tools. ### Static Access Control Each agent has a static access control list. The list is created on agent installation and never change unless agent developer changes the control list (TBD). ### Dynamic Access Control Each agent has a dynamic access control list. The list is created on agent instantiation and dynamically change when other agents are created or deleted (TBD). {% hint style="info" %} The access manager is still under development to have a more comprehensive guardrails over the lifecycle of agents. {% endhint %} # Syscalls Here are syscalls designed for achieving functionalities in the AIOS kernel. The `Syscall` class is a fundamental component in the AIOS framework, providing a standardized mechanism for asynchronous system calls between agents and system services. It extends the `Thread` class to handle concurrent operations and includes comprehensive tracking of call status, timing metrics, and response handling. ### Key Features * **Asynchronous Execution**: Built on Python's threading model for non-blocking operations * **Status Tracking**: Monitors the lifecycle of system calls from creation to completion * **Time Management**: Supports time limits and execution timing metrics * **Priority Handling**: Allows prioritization of system calls * **Event-Based Control**: Uses event synchronization for coordinated execution #### Initialization ```python def __init__(self, agent_name: str, query: Query): super().__init__() self.agent_name = agent_name self.query = query self.event = Event() # Call identification and status self.pid: Optional[int] = None self.aid: Optional[str] = None self.status: Optional[str] = None # Response and timing information self.response: Optional[Any] = None self.time_limit: Optional[float] = None # Timing metrics self.created_time: Optional[float] = None self.start_time: Optional[float] = None self.end_time: Optional[float] = None # Routing information self.source: Optional[str] = None self.target: Optional[str] = None self.priority: Optional[int] = None ``` #### Key Methods | Method | Description | | ------------------------ | ----------------------------------------------- | | `run()` | Executes the system call when the thread starts | | `set_status(status)` | Updates the call's current status | | `set_response(response)` | Sets the call's response data | | `get_response()` | Retrieves the call's response data | | `set_time_limit(limit)` | Sets maximum execution time | | `get_time_limit()` | Gets the call's time limit | #### Timing Management The class includes comprehensive timing tracking: ```python def set_created_time(self, time: float) -> None: self.created_time = time def set_start_time(self, time: float) -> None: self.start_time = time def set_end_time(self, time: float) -> None: self.end_time = time ``` These methods enable precise measurement of: * Time spent in queue (start\_time - created\_time) * Execution duration (end\_time - start\_time) * Total processing time (end\_time - created\_time) ### Usage Examples #### Creating a Custom Syscall ```python class LLMSyscall(Syscall): def __init__(self, agent_name: str, query: LLMQuery): super().__init__(agent_name, query) # Add LLM-specific attributes ``` #### Executing a Syscall ```python # Create the syscall syscall = LLMSyscall("agent_1", LLMQuery(...)) syscall.set_time_limit(30.0) # Set 30-second timeout # Start execution syscall.start() # Wait for completion syscall.join() # Get the result response = syscall.get_response() ``` # Terminal AIOS Terminal UI The terminal UI in AIOS is powered by the LLM-based semantic file system, which is published in ICLR2025, with its paper can be found at [From Commands to Prompts: LLM-based Semantic File System for AIOS](https://arxiv.org/pdf/2410.11843). Before running the AIOS terminal, please make sure you have read the instructions in [AIOS](https://github.com/agiresearch/AIOS) to set up the environment and launch the AIOS kernel. And you need to create a folder to put the files you want to manage, the default folder is `./root` After that, you can run the following command to start the AIOS terminal. ``` python scripts/run_terminal.py ``` When you successfully enter the system, the interface is as follows:

Then you can start interacting with the AIOS terminal by typing natural language commands. For instructions of how to use commands, refer to .

@inproceedings{
    shi2025from,
    title={From Commands to Prompts: {LLM}-based Semantic File System for AIOS},
    author={Zeru Shi and Kai Mei and Mingyu Jin and Yongye Su and Chaoji Zuo and Wenyue Hua and Wujiang Xu and Yujie Ren and Zirui Liu and Mengnan Du and Dong Deng and Yongfeng Zhang},
    booktitle={The Thirteenth International Conference on Learning Representations},
    year={2025},
    url={https://openreview.net/forum?id=2G021ZqUEZ}
}

# How to Use Agent To pipeline of how to use agents in AIOS is shown as the following figure.

AIOS first checks cache for the agent (typically the source code of the agent will be stored in \~/.cache/cerebrum/) and then installs the agent if it is not available. #### Running Agents **Run a Remote Agent** Use this command to run an agent from the AIOS Agent Hub: ```bash run-agent \ --mode remote \ --agent_author example \ --agent_name test_agent \ --agent_version 0.0.3 \ --task "What is the capital of United States?" \ --agenthub_url https://app.aios.foundation ``` **Run a Local Agent** For agents already available on your local device: ```bash run-agent \ --mode local \ --agent_path cerebrum/example/agents/test_agent \ --task "What is the capital of United States?" ``` #### Agent Management View agents available in the AIOS Agent Hub: ```bash list-agenthub-agents ``` View agents available locally: ```bash list-local-agents ``` Download an agent from the AIOS Agent Hub: ```bash download-agent \ --agent_author \ --agent_name \ --agent_version \ --agenthub_url ``` Upload an agent to the AIOS Agent Hub: ```bash upload-agents \ --agent_path \ --agenthub_url ``` # How to Develop Agents To develop new agents to run on AIOS, it provides the [native Agent-SDK](/aios-docs/aios-agent/how-to-develop-agents/develop-with-native-sdk) and various adapters to adapt different agent frameworks into AIOS. Below shows the agent frameworks that have been integrated and the current integration modules. The table below shows the agent frameworks we have integrated and different modules that have been integrated, and the details can be found in later sections.

Agent Framework	Integrated Module
AutoGen	`LLM`
Open-Interpreter	`LLM`
MetaGPT	`LLM`

Before developing agents, it is required to follow the guidelines of how to organize the agent files in the following structure. #### Agent Structure First, let's look at how to organize your agent's files. Every agent needs three mandatory components: ``` author/ └── agent_name/ │── entry.py # Your agent's main logic │── config.json # Configuration and metadata └── meta_requirements.txt # Additional dependencies ``` For example, if your name is demo\_author and you're building a demo\_agent that searches and summarizes articles, your folder structure would look like this: ``` demo_author/ └── demo_agent/ │── entry.py │── config.json └── meta_requirements.txt ``` Note: If your agent needs any libraries beyond AIOS's built-in ones, make sure to list them in meta\_requirements.txt. Apart from the above three files, you can have any other files in your folder. A minimal example of an agent is shown as below ```python from cerebrum.llm.apis import LLMQuery, LLMResponse from cerebrum.utils.communication import send_request, aios_kernel_url class TestAgent: def __init__(self, agent_name): self.agent_name = agent_name self.messages = [] def run(self, task_input): self.messages.append({"role": "user", "content": task_input}) tool_response = llm_chat( agent_name=self.agent_name, messages=self.messages, base_url=aios_kernel_url ) final_result = tool_response["response"]["response_message"] return final_result def __init__(self, agent_name, task_input, config_): super().__init__(agent_name, task_input, config_) def run(self): response = self.send_request( agent_name = self.agent_name, query=LLMQuery( messages=[{"role": "user", "content": "What is the capital of US? "}] ) ) print(response) return response ``` **\[Important]: Rules to be strictly followed in the entry.py** * The agent class must inherit the BaseAgent from * The agent class must have a `run` method as the main function to be executed * All the interactions with AIOS kernel must be through the `send_request` API, otherwise the services provided by the AIOS will not support for the agents. #### Configure the agent Your agent needs a config.json file that describes its functionality. Here's what it should include: ```json { "name": "demo_agent", "description": [ "Demo agent that can help search AIOS-related papers" ], "tools": [ "demo_author/arxiv" ], "meta": { "author": "demo_author", "version": "0.0.1", "license": "CC0" }, "build": { "entry": "agent.py", "module": "DemoAgent" } } ``` When setting up your agent, you'll need to specify which tools it will use. View tools available in the AIOS Tool Hub: ```bash list-toolhub-tools ``` View tools available locally: ```bash list-local-tools ``` To use these tools in your agent, use the [`AutoTool`](https://github.com/agiresearch/Cerebrum/blob/main/cerebrum/interface/__init__.py) provided by the AIOS SDK to load these tools within your agent logic. To load a tool from ToolHub in your code: ```python from cerebrum.interface import AutoTool tool = AutoTool.from_preloaded("example/arxiv", local=False) ``` To load a local tool in your code: ```python from cerebrum.tool import AutoTool tool = AutoTool.from_preloaded("google/google_search", local=True) ``` If you would like to create your new tools, you can either integrate the tool within your agent code or you can follow the tool examples in the tool folder to develop your standalone tools. The detailed instructions are in [How to develop new tools](/aios-docs/aios-agent-sdk/tool-api/how-to-develop-tools). # Develop with Native SDK ### 🚀 Develop and customize new agents Before starting developing your agents, make sure you have already read the instructions of the agent structures and rules for developing agents in [How to Use Agent](/aios-docs/aios-agent/how-to-use-agent) and [How to Develop Agents](/aios-docs/aios-agent/how-to-develop-agents). This guide will walk you through creating and running your own agents for AIOS with the native SDK. **Step 1: Set up the Agent Class** First, in the entry.py in your agent folder, create your agent class and implement the `run` method. ```python class TestAgent: def __init__(self, agent_name): self.agent_name = agent_name self.messages = [] def run(self, task_input): self.messages.append({"role": "user", "content": task_input}) llm_response = llm_chat( agent_name=self.agent_name, messages=self.messages, base_url="http://localhost:8000" ) final_result = llm_response["response"]["response_message"] return final_result ``` **Step 2: Use APIs provided by AIOS SDK to build your agents** LLM Core API Here are the LLM apis that can be imported from `cerebrum/llm/apis.py` ```python from cerebrum.llm.apis import llm_chat from cerebrum.llm.apis import llm_chat_with_json_output from cerebrum.llm.apis import llm_chat_with_tool_call_output from cerebrum.llm.apis import llm_call_tool from cerebrum.llm.apis import llm_operate_file ``` * Refer to [LLM-Core API](/aios-docs/aios-agent-sdk/llm-core-api) for detailed use Memory API Here are the Memory apis that can be imported from `cerebrum/memory/apis.py` ```python from cerebrum.memory.apis import create_memory from cerebrum.memory.apis import get_memory from cerebrum.memory.apis import update_memory from cerebrum.memory.apis import delete_memory from cerebrum.memory.apis import search_memories from cerebrum.memory.apis import create_agentic_memory ``` * Refer to [Memory API](/aios-docs/aios-agent-sdk/memory-api) for detailed use Storage API Here are the Storage apis that can be imported from `cerebrum/storage/apis.py` ```python from cerebrum.storage.apis import mount from cerebrum.storage.apis import create_file from cerebrum.storage.apis import create_dir from cerebrum.storage.apis import retrieve_file from cerebrum.storage.apis import rollback_file from cerebrum.storage.apis import share_file ``` * Refer to [Storage API](/aios-docs/aios-agent-sdk/storage-api) for detailed use Tool API ```python from cerebrum.tool.apis import call_tool ``` * Refer to [Tool API](/aios-docs/aios-agent-sdk/tool-api) for detailed use #### Step 3: Run and test your agents Run the agent you have just built ```bash run-agent \ --mode local \ --agent_path cerebrum/example/agents/test_agent \ --task "What is the capital of United States?" ``` # Develop with AutoGen ## Introduction AutoGen is an open-source programming framework for building AI agents and facilitating cooperation among multiple agents to solve tasks. We made it so that agent applications developed with Autogen can run on AIOS by adding just one line of code. (Only support AutoGen version\~0.2 now) ## Quick Start For installation and usage of AutoGen, please refer to [AutoGen](https://microsoft.github.io/autogen/docs/Getting-Started). If you want to run an application developed with Autogen on AIOS, please add `prepare_framework()` before you create an autogen agent, and select a framework type through `FrameworkType`. When you want to use AutoGen, you should use `FrameworkType.AutoGen`. Then create autogen agent. When running on AIOS, you don't need to supply parameter `llm_config`, this parameter configures the llm model that the agent will use. Because AIOS will deal with the calling of llms in the backend. ``` + with aios_starter(**vars(args)): + prepare_framework(FrameworkType.AutoGen) # Create the agent that uses the LLM. assistant = ConversableAgent("agent") # Create the agent that represents the user in the conversation. user_proxy = UserProxyAgent("user", code_execution_config=False) # Let the assistant start the conversation. It will end when the user types exit. assistant.initiate_chat(user_proxy, message="How can I help you today?") ``` More examples can be found in [.](https://github.com/agiresearch/Cerebrum/blob/main/cerebrum/community/adapter/autogen_adapter.py) # Develop with Open-Interpreter ## Introduction Open Interpreter lets language models run code. We made it so that agent applications developed with Open Interpreter can run on AIOS by adding just one line of code. ## Quick Start For installation and usage of open-interpreter, please refer to [Open Interpreter](https://docs.openinterpreter.com/getting-started/introduction). If you want to run an application developed with open-interpreter on AIOS, please add `prepare_interpreter()` before you use open-interpreter, and select a framework type through `FrameworkType`. When you want to use Open-Interpreter, you should use `FrameworkType.OpenInterpreter`. Then nothing needs to change, use interpreter as usual. ``` + with aios_starter(**vars(args)): + prepare_framework(FrameworkType.OpenInterpreter) interpreter.chat("In a group of 23 people, the probability of at least two having the same birthday is greater " "than 50%") ``` More examples can be found in [.](https://github.com/agiresearch/Cerebrum/blob/main/cerebrum/community/adapter/interpreter_adapter.py) # Develop with MetaGPT ## Intruduction MetaGPT is a multi-agent framework. We made it so that agent applications developed with MetaGPT can run on AIOS by adding just one line of code. ## Quick Start For installation and usage of open-interpreter, please refer to [MetaGPT](https://docs.deepwisdom.ai/main/en/). If you want to run an application developed with MetaGPT on AIOS, please add `prepare_framework()` before you use MetaGPT, and select a framework type through `FrameworkType`. When you want to use MetaGPT, you should use `FrameworkType.MetaGPT`. Then nothing needs to change, use MetaGPT as usual. ``` + with aios_starter(**vars(args)): + prepare_framework(FrameworkType.MetaGPT) repo: ProjectRepo = generate_repo("Create a 2048 game") # or ProjectRepo("") print(repo) ``` Or use Data Interpreter: ``` + with aios_starter(**vars(args)): + prepare_framework(FrameworkType.MetaGPT) async def di_main(): di = DataInterpreter() await di.run("Run data analysis on sklearn Iris dataset, include a plot") asyncio.run(di_main()) # or await main() in a jupyter notebook setting ``` More examples can be found in [.](https://github.com/agiresearch/Cerebrum/blob/main/cerebrum/community/adapter/metagpt_adapter.py) MetaGPT requires a longer output context to generate longer code, so you may need to use `--max_new_tokens` to set a larger output token length. ``` python scripts/aios-metagpt/example_aios_metagpt.py --llm_name gpt-4o-mini --max_new_tokens 4000 ``` # How to Publish Agents Before submitting your agents, make sure that you have read and followed the guidelines in [How to Develop Agents](/aios-docs/aios-agent/how-to-develop-agents) and check the following things 1. Tested your agent locally and verified its functionality 2. Organized agent files according to the required structure 3. Configured the correct versioning in your agent's configuration file After you develop your agents locally and organize the agent files in the structure, you can run the following file to upload your agent. ```python upload-agent \ --agent_path \ --agenthub_url ``` For example, if you run the following command ```python upload-agent \ --agent_path /home/user/xxx/Cerebrum/cerebrum/example/agents/academic_agent \ --agenthub_url https://app.aios.foundation ``` It is expected to show the following feedback if your agent has been uploaded successfully.

{% hint style="danger" %} It is important to mention that the version guidelines as below #### Version Format * Follow semantic versioning (MAJOR.MINOR.PATCH) * Example: 1.0.0, 1.1.0, 2.0.0 #### Important Version Rules 1. **Immutable Versions**: Similar to PyPI packages and Huggingface models, uploaded agent versions are immutable and cannot be modified 2. **Version Updates**: To upload changes: * Increment the version number in your agent's config.json * Major version (X.0.0): Breaking changes * Minor version (0.X.0): New features, backward compatible * Patch version (0.0.X): Bug fixes {% endhint %} # Overview The AIOS-Agent SDK (Cerebrum) provides a structured interface between user device applications and the AIOS kernel through different functional modules.

How agents leverage Agent-SDK to interact with AIOS kernel

All queries from these modules are ultimately channeled through a central `send_request(Query)` function in the AIOS-Agent SDK, which then communicates with the AIOS kernel via HTTP requests (either to localhost or a remote URL). On the user device side, the agent contains code snippets for planning, action, memory, and storage that interact with their respective modules through the Cerebrum Client interface. This creates a clean separation between the application logic and the kernel communication layer. **Please make sure you have read about how to use and develop agents before going deep into these APIs.** The SDKs contain the following APIs: * [LLM Core API](/aios-docs/aios-agent-sdk/llm-core-api) * [Memory API](/aios-docs/aios-agent-sdk/memory-api) * [Storage API](/aios-docs/aios-agent-sdk/storage-api) * [Tool API](/aios-docs/aios-agent-sdk/tool-api) # LLM Core API API for calling LLM Core(s) Within the LLM Core API, it relies on three important components: LLMQuery, LLMResponse and send\_request function ```python class LLMQuery(Query): query_class: str = "llm" llms: Optional[List[Dict[str, Any]]] = Field(default=None) messages: List[Dict[str, Union[str, Any]]] tools: Optional[List[Dict[str, Any]]] = Field(default_factory=list) action_type: Literal["chat", "tool_use", "operate_file"] = Field(default="chat") message_return_type: Literal["text", "json"] = Field(default="text") response_format: Optional[Dict[str, Any]] = Field(default=None) class Config: arbitrary_types_allowed = True ``` The LLM's response follows the `Response` class structure: ```python class LLMResponse(Response): response_class: str = "llm" response_message: Optional[str] = None tool_calls: Optional[List[Dict[str, Any]]] = None finished: bool = False error: Optional[str] = None status_code: int = 200 class Config: arbitrary_types_allowed = True ``` ### Common rules for LLM Core APIs When leveraging LLM Core APIs, you can specify which LLM to use with the `llms` parameter: ```python llms = [ { "name": "gpt-4o-mini", # Model name "backend": "openai" # Backend provider } ] ``` {% hint style="warning" %} It is important to mention that if you would like to pass multiple LLMs as the backends, the AIOS will automatically adopt the LLM routing to select a model from the LLMs you passed to address your queries. The selection is optimized on both model capability to solve the task and the cost to solve the task. This feature is detailed in [LLM router.](/aios-docs/aios-kernel/llm-cores/llm-routing) {% endhint %} # llm\_chat #### `llm_chat`: Basic Chat Interaction Enables simple text-based conversations with the language model. ```python def llm_chat( agent_name: str, messages: List[Dict[str, Any]], base_url: str = aios_kernel_url, llms: List[Dict[str, Any]] = None ) -> LLMResponse ``` **Parameters:** * `agent_name`: Identifier for the agent making the request * `messages`: List of message dictionaries (system, user, assistant) * `base_url`: API endpoint URL (default: configured AIOS kernel URL) * `llms`: Optional list of LLM configurations to use **Returns:** * `LLMResponse` object containing the model's text response **Example:** ```python # Basic chat interaction response = llm_chat( "my_assistant", messages=[ {"role": "system", "content": "You are a helpful AI assistant."}, {"role": "user", "content": "What are the main features of Python?"} ] ) print(response["response"]["response_message"]) ``` # llm\_chat\_with\_json\_output #### `llm_chat_with_json_output`: Structured JSON Responses Gets structured JSON responses from the language model according to a specified schema. ```python def llm_chat_with_json_output( agent_name: str, messages: List[Dict[str, Any]], base_url: str = aios_kernel_url, llms: List[Dict[str, Any]] = None, response_format: Dict[str, Dict] = None ) -> LLMResponse ``` **Parameters:** * `agent_name`: Identifier for the agent making the request * `messages`: List of message dictionaries * `base_url`: API endpoint URL * `llms`: Optional list of LLM configurations * `response_format`: JSON schema specifying the required output format **Returns:** * `LLMResponse` object containing the structured JSON response **Example:** ```python # Extract keywords from a text response = llm_chat_with_json_output( "content_analyzer", messages=[ {"role": "system", "content": "Extract key information from the text."}, {"role": "user", "content": "AIOS is a new operating system for AI agents."} ], response_format={ "type": "json_object", "schema": { "type": "object", "properties": { "keywords": { "type": "array", "items": {"type": "string"} }, "summary": {"type": "string"} }, "required": ["keywords", "summary"] } } ) print(response["response"]["response_message"]) # JSON string containing keywords and summary ``` # llm\_chat\_with\_tool\_call\_output #### `llm_chat_with_tool_call_output`: Chat with Tool Integration Allows the language model to decide which tools to use based on the conversation context. ```python def llm_chat_with_tool_call_output( agent_name: str, messages: List[Dict[str, Any]], tools: List[Dict[str, Any]], base_url: str = aios_kernel_url, llms: List[Dict[str, Any]] = None ) -> LLMResponse ``` **Parameters:** * `agent_name`: Identifier for the agent making the request * `messages`: List of message dictionaries * `tools`: List of available tools and their specifications * `base_url`: API endpoint URL * `llms`: Optional list of LLM configurations **Returns:** * `LLMResponse` object containing tool calls made by the model **Example:** ```python # Chat that may trigger tool use when needed response = llm_chat_with_tool_call_output( "research_assistant", messages=[ {"role": "system", "content": "Help the user with research tasks."}, {"role": "user", "content": "I need to find recent papers about transformer architectures."} ], tools=[{ "name": "scholar_search", "description": "Search for academic papers on a topic", "parameters": { "type": "object", "properties": { "query": {"type": "string"}, "year_start": {"type": "integer"}, "max_results": {"type": "integer"} }, "required": ["query"] } }] ) # Check if tool calls were made tool_calls = json.loads(response["response"]["tool_calls"]) if tool_calls: for tool_call in tool_calls: print(f"Tool: {tool_call['name']}") print(f"Parameters: {tool_call['parameters']}") ``` # llm\_call\_tool #### `llm_call_tool`: Direct Tool Invocation Explicitly instructs the language model to use specified tools with user input. ```python def llm_call_tool( agent_name: str, messages: List[Dict[str, Any]], tools: List[Dict[str, Any]], base_url: str = aios_kernel_url, llms: List[Dict[str, Any]] = None ) -> LLMResponse ``` **Parameters:** * `agent_name`: Identifier for the agent making the request * `messages`: List of message dictionaries * `tools`: List of available tools and their specifications * `base_url`: API endpoint URL * `llms`: Optional list of LLM configurations **Returns:** * `LLMResponse` object containing tool calls and their results **Example:** ```python # Use weather tool to get forecast response = llm_call_tool( "weather_agent", messages=[ {"role": "user", "content": "What's the weather like in New York today?"} ], tools=[{ "name": "weather_service/get_forecast", "description": "Get weather forecast for a location", "parameters": { "type": "object", "properties": { "location": {"type": "string"}, "units": {"type": "string", "enum": ["metric", "imperial"]} }, "required": ["location"] } }] ) # Process the tool response print(response["response"]["response_message"]) ``` # llm\_operate\_file #### `llm_operate_file`: File Management Operations Uses the language model to perform file operations based on user instructions. ```python def llm_operate_file( agent_name: str, messages: List[Dict[str, Any]], tools: List[Dict[str, Any]] = None, base_url: str = aios_kernel_url, llms: List[Dict[str, Any]] = None ) -> LLMResponse ``` **Parameters:** * `agent_name`: Identifier for the agent making the request * `messages`: List of message dictionaries * `tools`: Optional list of tools (usually not needed for file operations) * `base_url`: API endpoint URL * `llms`: Optional list of LLM configurations **Returns:** * `LLMResponse` object containing file operation results **Example:** ```python # Create a text file response = llm_operate_file( "file_assistant", messages=[ {"role": "user", "content": "Create a new file called notes.txt with a todo list for today"} ] ) # Process the response print(response["response"]["response_message"]) # Contains information about the file operation ``` ### Common Message Formats All functions accept messages in the following format: ```python messages = [ { "role": "system", # Can be "system", "user", or "assistant" "content": "You are a helpful assistant specialized in...", # Message content "name": "example_name" # Optional name for the message sender }, { "role": "user", "content": "Please help me with..." } ] ``` # Memory API The Memory API provides a structured way to manage and retrieve agent memories through CRUD operations and semantic search capabilities. This documentation explains core functionality and usage patterns. ### Core Concepts * **Memory**: A structured data object containing: * Content: Textual information * Metadata: Key-value pairs for organization * Unique memory ID * **Operations**: Create, Read, Update, Delete, and Semantic Search * **Agent Context**: Memories are stored per agent namespace #### Functional Details **1. Standard Memory APIs** | API | Parameters | Outputs | Use Case Example | | ----------------- | ------------------------------- | ------------------ | -------------------------------- | | `create_memory` | `content`, `metadata` | Memory ID | Storing meeting notes | | `read_memory` | `memory_id` | Content + Metadata | Recalling specific events | | `update_memory` | `memory_id`, `content/metadata` | Success status | Correcting outdated information | | `delete_memory` | `memory_id` | Success status | GDPR compliance requests | | `search_memories` | `query`, `k` (results count) | Ranked memory list | Finding related project memories | **2. Agentic Memory APIs** | API | Special Features | Cognitive Components | Autonomous Behaviors | | ----------------------- | ----------------------------------------------- | -------------------------------------------- | -------------------------------------------- | | `create_agentic_memory` |

- Dynamic linking
- Cognitive tagging

Activation levels
Retention scores

Auto-clustering
Context adaptation

| # create\_memory Create Memory **Function**: `create_memory()`\ **Purpose**: Store new memory with optional metadata\ **Parameters**: * `agent_name`: Namespace identifier (required) * `content`: Memory text (required) * `metadata`: Optional key-value pairs (tags, priority, etc.) * `base_url`: API endpoint (default: configured kernel URL) **Example**: ```python metadata = { "tags": ["project-planning", "urgent"], "context": "Q4 strategy meeting" } response = create_memory( agent_name="project_bot", content="Decided to accelerate feature rollout timeline by 2 weeks", metadata=metadata ) ``` **Response Structure** (MemoryResponse): ```json { "success": true, "memory_id": "mem_abc123", "content": null, "metadata": null, "error": null } ``` # get\_memory **Function**: `get_memory()`\ **Purpose**: Fetch memory contents by ID\ **Parameters**: * `agent_name`: Namespace identifier * `memory_id`: Target memory ID * `base_url`: API endpoint **Example**: ```python response = get_memory( agent_name="project_bot", memory_id="mem_abc123" ) ``` **Response**: ```json { "success": true, "memory_id": "mem_abc123", "content": "Decided to accelerate...", "metadata": { "tags": ["project-planning", "urgent"], "context": "Q4 strategy meeting" }, "error": null } ```
# update\_memory **Function**: `update_memory()`\ **Purpose**: Modify existing memory content/metadata\ **Parameters**: * `agent_name`: Namespace identifier * `memory_id`: Target memory ID * `content`: New text (optional) * `metadata`: New/updated fields (optional) **Example**: ```python update_memory( agent_name="project_bot", memory_id="mem_abc123", content="Updated timeline decision", metadata={"priority": "critical"} ) ``` **Response**: ```json { "success": true, "memory_id": "mem_abc123", "content": null, "metadata": null, "error": null } ``` # delete\_memory **Function**: `delete_memory()`\ **Purpose**: Remove memory by ID\ **Parameters**: * `agent_name`: Namespace identifier * `memory_id`: Target memory ID **Example**: ```python delete_memory( agent_name="project_bot", memory_id="mem_abc123" ) ``` **Response**: ```json { "success": true, "error": null } ``` # search\_memories **Function**: `search_memories()`\ **Purpose**: Find relevant memories using natural language\ **Parameters**: * `agent_name`: Namespace identifier * `query`: Search phrase * `k`: Maximum results (default:5) **Example**: ```python search_memories( agent_name="project_bot", query="timeline changes", k=3 ) ``` **Response**: ```json { "success": true, "search_results": [ { "memory_id": "mem_abc123", "content": "Updated timeline decision...", "score": 0.92, "metadata": {...} } ], "error": null } ``` # create\_agentic\_memory ### Overview The Agentic Memory system provides advanced memory management capabilities that enable LLM agents to dynamically organize and utilize memories through autonomous decision-making. Unlike traditional static memory storage, this system implements intelligent memory operations inspired by cognitive architectures. ### Key Capabilities ✨ #### Dynamic Memory Organization * 🔄 Zettelkasten-inspired knowledge networking * 🔗 Auto-generated memory relationships * 🧩 Context-aware memory clustering #### Intelligent Memory Operations * 🔍 Semantic indexing with multi-modal tagging * 📈 Adaptive retention/forgetting mechanisms * 🤖 Autonomous memory refinement cycles #### Agent-Memory Interaction * 🌐 State-aware memory activation * 🎯 Task-specific memory prioritization * 🔄 Bidirectional memory-agent influence *** ### API Integration #### Create Agentic Memory **Function**: `create_agentic_memory()`\ **Purpose**: Store memories with autonomous organization capabilities **Parameters**: | Parameter | Type | Description | | ------------ | ---- | ----------------------------------- | | `agent_name` | str | Agent namespace identifier | | `content` | str | Memory text with contextual signals | | `metadata` | dict | Structured attributes (see below) | | `base_url` | str | API endpoint | **Example**: ```python response = create_agentic_memory( agent_name="research_bot" content="Breakthrough in protein folding: Achieved 92% accuracy...", metadata={ "system": "scientific_discovery" } ) ``` **Response Structure**: ```json { "success": true, "memory_id": "mem_abc123", "content": null, "metadata": null, "error": null } ``` AIOS Kernel will automatically use agentic memory to help you organize your memory. **Reference**: ``` @article{xu2025mem, title={A-mem: Agentic memory for llm agents}, author={Xu, Wujiang and Liang, Zujie and Mei, Kai and Gao, Hang and Tan, Juntao and Zhang, Yongfeng}, journal={arXiv preprint arXiv:2502.12110}, year={2025} } ``` For implementation details and experimental results, see our [official repository](https://github.com/agiresearch/A-mem) and [research paper](https://arxiv.org/pdf/2502.12110). # Storage API This part introduces the Storage API functions available in AIOS. These functions allow agents to interact with the file system for various operations such as mounting storage, retrieving, creating, modifying, and sharing files. Interactions with the storage layer are handled through the `StorageQuery` class: ```python class StorageQuery(BaseModel): params: List[Dict[str, Union[str, Any]]] # params operation_type: str = Field(default="text") # Operation type specification class Config: arbitrary_types_allowed = True # Enables complex type usage ``` All storage functions return a `StorageResponse` object with these key attributes: * `response_message`: Contains the operation result or requested data * `finished`: Boolean indicating whether the operation completed successfully * `error`: Error message (if any) * `status_code`: HTTP status code ```python class StorageResponse(Response): response_class: str = "storage" response_message: Optional[str] = None finished: bool = False error: Optional[str] = None status_code: int = 200 ``` # mount #### `mount`: Mount Storage Directory Mounts a directory as the root storage location for an agent. ```python def mount( agent_name: str, root_dir: str, base_url: str = aios_kernel_url ) -> StorageResponse ``` **Parameters:** * `agent_name`: Identifier for the agent making the request * `root_dir`: Directory path to mount as storage root * `base_url`: API endpoint URL (default: configured AIOS kernel URL) **Returns:** * `StorageResponse` object containing operation result **Example:** ```python # Mount a personal storage directory for the agent response = mount( "research_agent", "/data/research_projects" ) print(response["response"]["response_message"]) ``` # create\_file #### `create_file`: Create Empty File Creates a new empty file at the specified path. ```python def create_file( agent_name: str, file_path: str, base_url: str = aios_kernel_url ) -> StorageResponse ``` **Parameters:** * `agent_name`: Identifier for the agent making the request * `file_path`: Path where the new file should be created * `base_url`: API endpoint URL **Returns:** * `StorageResponse` object containing creation status **Example:** ```python # Create a new Python script file response = create_file( "developer_agent", "projects/data_analyzer/main.py" ) print(response["response"]["response_message"] ``` # create\_dir #### `create_dir`: Create Directory Creates a new directory at the specified path. ```python def create_dir( agent_name: str, dir_path: str, base_url: str = aios_kernel_url ) -> StorageResponse ``` **Parameters:** * `agent_name`: Identifier for the agent making the request * `dir_path`: Path where the new directory should be created * `base_url`: API endpoint URL **Returns:** * `StorageResponse` object containing creation status **Example:** ```python # Create a project directory structure response = create_dir( "project_manager", "projects/new_webapp/src" ) print(response["response"]["response_message"]) ``` # write\_file #### `write_file`: Write Content to File Writes content to a file, creating it if it doesn't exist. ```python def write_file( agent_name: str, file_path: str, content: str, base_url: str = aios_kernel_url ) -> StorageResponse ``` **Parameters:** * `agent_name`: Identifier for the agent making the request * `file_path`: Path to the file to write * `content`: Text content to write to the file * `base_url`: API endpoint URL **Returns:** * `StorageResponse` object containing write operation status **Example:** ```python # Write a simple HTML file html_content = """ AIOS Project

Welcome to AIOS

This is a sample project page.

""" response = write_file( "web_developer", "projects/website/index.html", html_content ) print(response["response]["response_message"]) ``` # retrieve\_file #### `retrieve_file`: Search and Retrieve Files Searches for files matching query criteria and returns the results. ```python def retrieve_file( agent_name: str, query_text: str, n: int, keywords: List[str] = None, base_url: str = aios_kernel_url ) -> StorageResponse ``` **Parameters:** * `agent_name`: Identifier for the agent making the request * `query_text`: Text to search for in files * `n`: Maximum number of results to return * `keywords`: Optional list of specific keywords to match * `base_url`: API endpoint URL **Returns:** * `StorageResponse` object containing matched files **Example:** ```python # Find Python files related to data processing response = retrieve_file( "data_scientist", "data processing pipeline", n=5, keywords=["pandas", "numpy", "dataframe"] ) print(response["response"]["response_message"]) ``` # rollback\_file #### `rollback_file`: Restore Previous File Version Reverts a file to a previous version. ```python def rollback_file( agent_name: str, file_path: str, n: int, base_url: str = aios_kernel_url ) -> StorageResponse ``` **Parameters:** * `agent_name`: Identifier for the agent making the request * `file_path`: Path to the file to roll back * `n`: Number of versions to roll back (1 for previous version) * `base_url`: API endpoint URL **Returns:** * `StorageResponse` object containing rollback status **Example:** ```python # Roll back a configuration file to its previous version response = rollback_file( "system_agent", "config/app_settings.json", n=1 ) print(response["response"]["response_message"]) ``` # share\_file #### `share_file`: Share Files Makes a file accessible in the AIOS environment. ```python def share_file( agent_name: str, file_path: str, base_url: str = aios_kernel_url ) -> StorageResponse ``` **Parameters:** * `agent_name`: Identifier for the agent sharing the file * `file_path`: Path to the file to be shared * `base_url`: API endpoint URL **Returns:** * `StorageResponse` object containing sharing status **Example:** ```python # Share a dataset file with other agents response = share_file( "data_engineer", "datasets/processed_data.csv" ) print(response["response"]["response_message"]) ``` # Tool API ### Structure of Tool Query and Response The `ToolQuery` class defines the structure for tool operation requests: ```python class ToolQuery(Query): query_class: str = "tool" tool_calls: List[Dict[str, Union[str, Any]]] class Config: arbitrary_types_allowed = True ``` The `ToolResponse` class defines the structure for tool operation responses: ```python class ToolResponse(Response): response_class: str = "tool" response_message: Optional[str] = None finished: bool = False error: Optional[str] = None status_code: int = 200 ``` ### Available Tool Management Commands **List Hub Tools** ```bash # View all tools available in the AIOS Tool Hub list-toolhub-tools ``` **List Local Tools** ```bash # View tools installed on your local system list-local-tools ``` **Download a Tool** ```bash # Get a specific tool from the AIOS Tool Hub download-tool \ --tool_author \ --tool_name \ --tool_version \ --toolhub_url ``` **Upload a Tool** ```bash # Share your custom tool with the AIOS community upload-tool \ --tool_path \ --toolhub_url ``` These commands help you discover, install, and share tools that extend AIOS functionality. Tools can provide specialized capabilities like data processing, external API access, and advanced computational features. # How to Develop Tools Similar as [developing new agents](/aios-docs/aios-agent/how-to-develop-agents), developing tools also need to follow a simple directory structure: ``` demo_author/ └── demo_tool/ │── entry.py # Contains your tool's main logic └── config.json # Tool configuration and metadata ``` {% hint style="danger" %} To use the agents in your local device, you need to put the tool folder under the cerebrum/tool/core folder and register your tool in the cerebrum/tool/core/registry.py {% endhint %} #### Create Tool Class In `entry.py`, you'll need to implement a tool class which is identified in the config.json with two essential methods: 1. `get_tool_call_format`: Defines how LLMs should interact with your tool 2. `run`: Contains your tool's main functionality Here's an example: ```python class Wikipedia: def __init__(self): super().__init__() self.WIKIPEDIA_MAX_QUERY_LENGTH = 300 self.top_k_results = 3 self.lang = "en" self.load_all_available_meta: bool = False self.doc_content_chars_max: int = 4000 self.wiki_client = self.build_client() def build_client(self): try: import wikipedia wikipedia.set_lang(self.lang) except ImportError: raise ImportError( "Could not import wikipedia python package. " "Please install it with `pip install wikipedia`." ) return wikipedia def run(self, params) -> str: """Run Wikipedia search and get page summaries.""" query = params["query"] page_titles = self.wiki_client.search(query, results=self.top_k_results) summaries = [] for page_title in page_titles[: self.top_k_results]: if wiki_page := self._fetch_page(page_title): if summary := self._formatted_page_summary(page_title, wiki_page): summaries.append(summary) if not summaries: return "No good Wikipedia Search Result was found" return "\n\n".join(summaries)[: self.doc_content_chars_max] @staticmethod def _formatted_page_summary(page_title: str, wiki_page: Any) -> Optional[str]: return f"Page: {page_title}\nSummary: {wiki_page.summary}" def get_tool_call_format(self): tool_call_format = { "type": "function", "function": { "name": "wikipedia", "description": "Provides relevant information about the destination", "parameters": { "type": "object", "properties": { "query": { "type": "string", "description": "Search query for Wikipedia" } }, "required": [ "query" ] } } } return tool_call_format ``` #### How to publish tools to the toolhub Before publishing tools, you need to set up the configurations as the following: ```json { "name": "wikipedia", "description": [ "Search information in the wikipedia" ], "meta": { "author": "example", "version": "0.0.1", "license": "CC0" }, "build": { "entry": "tool.py", "module": "Wikipedia" } } ``` then you can use the following command to upload tool ```python python cerebrum/commands/upload_tool.py \ --tool_path \ # tool path to the tool directory --toolhub_url # the url of the toolhub, default is https://app.aios.foundation ``` # Access API API for agents to set access controls for its resources such as memory, storage, etc. The access APIs are under development along with user account system. # Post API API for agents to send posts to each other API for Agent to Agent Communication (under development) # Agent API ### List agents from agenthub ```bash list-agenthub-agents ``` ### List agents from local ``` list-local-agents ``` ### Download agents ```bash download-agent \ --agent_author \ --agent_name \ --agent_version \ --agenthub_url ``` ### Upload agent ```bash upload-agents \ --agent_path \ --agenthub_url ``` # How to Contribute Thank you for your interest in AIOS! Here's a guide to help you contribute to this project. ### 1. Get Started #### Fork the repository At first, you need to fork this copy and create your own version of repo. #### Clone the repository and install the dependencies. #### Installing dependencies with pip ```bash pip install -r requirements.txt ``` #### Installing pre-commit We strongly recommend installing [pre-commit](https://pre-commit.com/) to ensure proper formatting during development ### 2. Developing and Testing #### Create a branch Create a new branch for developing your creative features ```shell git checkout -b your-feature ``` #### Make changes and testing You can develop new features and then you need to make sure everything works as expected. Run our provided tests and make sure the existing ones go well. Your new tests are encouraged. #### Run tests Add your test code into the `tests/` directory if any, then run test via [pytest](https://docs.pytest.org/en/8.0.x/) ``` pytest -v tests/ ``` **Sample Output** ``` ============================================================================================================================= test session starts ============================================================================================================================== platform darwin -- Python 3.11.9, pytest-8.1.1, pluggy-1.5.0 -- "" cachedir: .pytest_cache rootdir: "" plugins: anyio-4.3.0 collected 7 items tests/test_llms.py::test_closed_llm PASSED [ 14%] tests/test_llms.py::test_open_llm PASSED [ 28%] tests/test_lru_k_replacer.py::test_update_access_history PASSED [ 42%] tests/test_lru_k_replacer.py::test_evict PASSED [ 57%] tests/test_lru_k_replacer.py::test_set_evictable PASSED [ 71%] tests/test_lru_k_replacer.py::test_invalid_block_id PASSED [ 85%] tests/test_memory.py::test_mem_alloc PASSED ``` ### 3. Submitting Changes #### Code format check Please ensure your code is formatted correctly using pre-commit #### Git commit format We strongly recommend your git commit follows the format below ```bash git commit -m : ``` | | | | ---------- | ---------------------------------------------------------------------- | | `feat` | Add new features | | `fix` | Fix bugs | | `docs` | Modify documents like README, CONTRIBUTE | | `style` | Modify code format like space and comma without changing code logic | | `refactor` | Refactor code structure without adding new features or fixing new bugs | | `perf` | Improve performance or user experience | | `test` | Test features, including unit test and integration test | | `chore` | Change the build procedure or add dependencies | | `revert` | Revert to the previous version | #### Create a Pull Request 1. Visit your forked AIOS repository on GitHub and click the "Compare & pull request" button to initiate the process of submitting your changes to the original repository for review and potential merging. 2. Choose the base branch and the compare branch (your feature branch).💡 Note that when you add new features, it is recommended to choose the (`dev`) branch and if your change does not affect original functions, you may consider choosing the (`main`) branch. 3. Write a title and describe your changes in the description. And it is recommended to select the label of the change to make it more clear. ### 4. Style Guide This is the style guide determining how code should be formatted. As more code is added, this documentation will be updated. #### Line Length Each line should not exceed 80 characters. #### Spaces Each function or class is to have a space between the line and any preceding code if it is not directly associated with the function or class. For example: ```py # this is a useful comment for the bar function def foo(): pass ``` #### Commenting Each directory is required to describe the purpose of the directory and each file in the directory in README.md. Each file is required to have a header describing the purpose at the top. For example: ```py # This file has helpful math functions that we will use in the foo module in # AIOS def add(a, b): return a + b def subtract(a, b): return a - b ``` Any comments for imports should be preceded by a `#` as well. Spaces should be applied as necessary. Each function is required to have a comment right under the header which describes what it does, unless the function encompasses few lines. For example: ```py def foo(): """This function does something""" pass ``` The same applies to classes: ```py class Bar: """This class does something""" def baz(self): """This function does something""" pass pass ``` Lines that are to be commented out can be marked with `#`. For example: ```py # def foo(): # """This function does something""" # pass ``` It is not recommended to comment out lines, but if you must make sure that the purpose is understood by the reader. Any comments for a specific line are additionally to be marked with `#`: ```py a = 1 b = 3 c = a + b # This adds a and b and stores the result in c ``` A comment describing multiple lines in a particular function can be represented by `#` as well. ### 5. Review and Approval Our maintainers will have a review of that and might give some suggestions or ask for more details. After they approve, your commitment can be incorporated into AIOS! If you need some ideas on what to get started with, take a look at the roadmaps in [issues](https://github.com/agiresearch/AIOS/issues).