# LLM Core(s)

AIOS wraps different LLMs as LLM Cores to support the unified interface for addressing requests to both cloud LLM apis and locally-hosted LLMs. 

AIOS supports three main categories of backends:

1. LiteLLM compatible backends (cloud and local)
2. vLLM backends (local)
3. Huggingface backends (local)

Each backend type handles different input scenarios, including standard text generation, tool calling, and JSON-formatted responses. This document explains how each backend processes these inputs and the implementation details.

| Feature        | LiteLLM Compatible                 | vLLM                               | Huggingface                              |
| -------------- | ---------------------------------- | ---------------------------------- | ---------------------------------------- |
| Standard Input | Uses completion function           | Uses OpenAI client                 | Uses generate method                     |
| Tool Calls     | Native support via tools parameter | Native support via tools parameter | Uses message merging and custom decoding |
| JSON Responses | Uses format="json"                 | Uses format="json"                 | Uses message merging                     |

Different backend details are as below

* [Litellm compatible backends](/aios-docs/aios-kernel/llm-cores/litellm-compatible-backend.md)
* [vLLM backends](/aios-docs/aios-kernel/llm-cores/vllm-backend.md)
* [Huggingface local backends](/aios-docs/aios-kernel/llm-cores/hugging-face-backend.md)


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.aios.foundation/aios-docs/aios-kernel/llm-cores.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.