Instead of placing separate queues within each processing module (e.g., LLM core(s) or memory manager), we centralize all the queues within the scheduler module. This aims to isolate the responsibility for request management from the individual modules, allowing each processing module to focus on its execution and simplifies the coordination of tasks across modules.
AIOS Scheduler manages the Request queue for each module, the Request queue includes both Queries (agent-to-kernel request) and Messages (module-to-module request)
The main purpose of the scheduler is to serve different agents queries in a fine-grained granularity (i.e., syscall level granuity) as below.
Scheduling example for syscalls for different agents (use LLM Syscall for illustration simplicity)
AIOS provides two main scheduler implementations to manage system calls from agents:
Feature
FIFOScheduler
RRScheduler
Task Selection
First-come, first-served
Round-robin rotation
Time Management
No time limit enforcement
Fixed time slices per task
Task Preemption
None (tasks run to completion)
Tasks can be suspended and resumed
Status Handling
Only "executing"/"done"
"executing"/"done"/"suspending"
Context Management
None
Uses SimpleContextManager
BaseScheduler
The BaseScheduler is an abstract base class that defines the common interface and functionality for all scheduler implementations in AIOS.
