# Quickstart
Be sure to complete the [installation instructions](https://docs.aios.foundation/aios-docs/getting-started/installation) before continuing with this guide.
Before starting AIOS, you need to make sure you have installed the LLM backends that you would like to run. Here are the LLM providers for supported backends for AIOS.
| Provider 🏢 | Model Name 🤖 | Open Source 🔓 | Model String ⌨️ | Backend ⚙️ | Required API Key |
| ----------- | --------------------------------------------- | -------------- | ------------------------------------- | ----------- | ------------------- |
| Anthropic | Claude 3.5 Sonnet | ❌ | claude-3-5-sonnet-20241022 | anthropic | ANTHROPIC\_API\_KEY |
| Anthropic | Claude 3.5 Haiku | ❌ | claude-3-5-haiku-20241022 | anthropic | ANTHROPIC\_API\_KEY |
| Anthropic | Claude 3 Opus | ❌ | claude-3-opus-20240229 | anthropic | ANTHROPIC\_API\_KEY |
| Anthropic | Claude 3 Sonnet | ❌ | claude-3-sonnet-20240229 | anthropic | ANTHROPIC\_API\_KEY |
| Anthropic | Claude 3 Haiku | ❌ | claude-3-haiku-20240307 | anthropic | ANTHROPIC\_API\_KEY |
| Deepseek | Deepseek-reasoner (R1) | ❌ | deepseek-reasoner | deepseek | DEEPSEEK\_API\_KEY |
| Deepseek | Deepseek-chat (V3) | ❌ | deepseek-chat | deepseek | DEEPSEEK\_API\_KEY |
| OpenAI | GPT-4 | ❌ | gpt-4 | openai | OPENAI\_API\_KEY |
| OpenAI | GPT-4 Turbo | ❌ | gpt-4-turbo | openai | OPENAI\_API\_KEY |
| OpenAI | GPT-4o | ❌ | gpt-4o | openai | OPENAI\_API\_KEY |
| OpenAI | GPT-4o mini | ❌ | gpt-4o-mini | openai | OPENAI\_API\_KEY |
| OpenAI | GPT-3.5 Turbo | ❌ | gpt-3.5-turbo | openai | OPENAI\_API\_KEY |
| Google | Gemini 1.5 Flash | ❌ | gemini-1.5-flash | google | GEMINI\_API\_KEY |
| Google | Gemini 1.5 Flash-8B | ❌ | gemini-1.5-flash-8b | google | GEMINI\_API\_KEY |
| Google | Gemini 1.5 Pro | ❌ | gemini-1.5-pro | google | GEMINI\_API\_KEY |
| Google | Gemini 1.0 Pro | ❌ | gemini-1.0-pro | google | GEMINI\_API\_KEY |
| Groq | Llama 3.2 90B Vision | ✅ | llama-3.2-90b-vision-preview | groq | GROQ\_API\_KEY |
| Groq | Llama 3.2 11B Vision | ✅ | llama-3.2-11b-vision-preview | groq | GROQ\_API\_KEY |
| Groq | Llama 3.1 70B | ✅ | llama-3.1-70b-versatile | groq | GROQ\_API\_KEY |
| Groq | Llama Guard 3 8B | ✅ | llama-guard-3-8b | groq | GROQ\_API\_KEY |
| Groq | Llama 3 70B | ✅ | llama3-70b-8192 | groq | GROQ\_API\_KEY |
| Groq | Llama 3 8B | ✅ | llama3-8b-8192 | groq | GROQ\_API\_KEY |
| Groq | Mixtral 8x7B | ✅ | mixtral-8x7b-32768 | groq | GROQ\_API\_KEY |
| Groq | Gemma 7B | ✅ | gemma-7b-it | groq | GROQ\_API\_KEY |
| Groq | Gemma 2B | ✅ | gemma2-9b-it | groq | GROQ\_API\_KEY |
| Groq | Llama3 Groq 70B | ✅ | llama3-groq-70b-8192-tool-use-preview | groq | GROQ\_API\_KEY |
| Groq | Llama3 Groq 8B | ✅ | llama3-groq-8b-8192-tool-use-preview | groq | GROQ\_API\_KEY |
| ollama | [All Models](https://ollama.com/search) | ✅ | model-name | ollama | - |
| vLLM | [All Models](https://docs.vllm.ai/en/latest/) | ✅ | model-name | vllm | - |
| HuggingFace | [All Models](https://huggingface.co/models/) | ✅ | model-name | huggingface | HF\_HOME |
### Configuration
**Set up configuration file directly (Recommended)**
You need API keys for services like OpenAI, Anthropic, Groq and HuggingFace. The simplest way to configure them is to edit the `aios/config/config.yaml`.
> \[!TIP] It is important to mention that, we strongly recommend using the `aios/config/config.yaml` file to set up your API keys. This method is straightforward and helps avoid potential sychronization issues with environment variables.
A simple example to set up your API keys in `aios/config/config.yaml` is shown below:
```yaml
api_keys:
openai: "your-openai-key"
gemini: "your-gemini-key"
groq: "your-groq-key"
anthropic: "your-anthropic-key"
huggingface:
auth_token: "token to authorize specific models for use"
home: "path to store downloaded model weights"
```
To obtain these API keys:
1. Deepseek API:
2. OpenAI API:
3. Google Gemini API:
4. Groq API:
5. HuggingFace Token:
6. Anthropic API:
**Configure LLM Models**
You can configure which LLM models to use in the same `aios/config/config.yaml` file. Here's an example configuration:
```yaml
llms:
models:
# Ollama Models
- name: "qwen2.5:7b"
backend: "ollama"
hostname: "http://localhost:11434" # Make sure to run ollama server
# vLLM Models
- name: "meta-llama/Llama-3.1-8B-Instruct"
backend: "vllm"
hostname: "http://localhost:8091/v1" # Make sure to run vllm server
```
**Using Ollama Models:**
1. First, download ollama from
2. Start the ollama server in a separate terminal:
```bash
ollama serve
```
3. Pull your desired models from :
```bash
ollama pull qwen2.5:7b # example model
```
> Ollama supports both CPU-only and GPU environments. For more details about ollama usage, visit [ollama documentation](https://github.com/ollama/ollama)
**Using vLLM Models:**
1. Install vLLM following their [installation guide](https://docs.vllm.ai/en/latest/getting_started/installation.html)
2. Start the vLLM server in a separate terminal:
```bash
vllm serve meta-llama/Llama-3.1-8B-Instruct --port 8091
```
> vLLM currently only supports Linux and GPU-enabled environments. If you don't have a compatible environment, please choose other backend options. To enable the tool calling feature of vllm, refer to
**Using HuggingFace Models:** You can configure HuggingFace models with specific GPU memory allocation:
```yaml
- name: "meta-llama/Llama-3.1-8B-Instruct"
backend: "huggingface"
max_gpu_memory: {0: "24GB", 1: "24GB"} # GPU memory allocation
eval_device: "cuda:0" # Device for model evaluation
```
### Launch AIOS
After you setup the required keys, you can run the following command to launch the AIOS kernel.
```bash
bash runtime/launch_kernel.sh
```
And then you can start a client to interact with the AIOS kernel using Terminal or WebUI.