Docs
Concepts: the runtime
One API, models running in parallel, pinned vs just-in-time, and one memory budget across everything.
Bedrova is a runtime, not just a chat window. Four ideas explain the whole product. The rest of the docs assume you’ve read this page.
One API, routed by type
Everything is reachable on localhost through one API that speaks both the OpenAI and
Anthropic dialects. The endpoint decides which model answers:
| You call | Bedrova routes to |
|---|---|
/v1/chat/completions, /v1/responses | a chat / reasoning model |
/v1/embeddings | an embedding model |
/v1/images/generations | an image model |
/v1/audio/transcriptions | a speech-to-text model |
/v1/audio/speech | a text-to-speech model |
/api/v1/search, /api/v1/rerank | a local search / rerank provider |
/api/v1/perception/{task} | a Core ML perception provider |
/anthropic/v1/messages | a chat model, Anthropic-style |
You make one kind of call; Bedrova picks the model. Full request and response shapes are in the API reference.
Models run in parallel
A scheduler gives each workload its own lane: text, vision, embeddings and search, perception, speech, image and video. Different model types run at the same time instead of queuing behind one another. GGUF is not a separate lane; it is a text-runtime choice. Before any expensive work starts, an admission gate checks memory. When a lane is saturated, it returns a retryable response instead of crashing the app. See Parallel requests & scheduling.
Pinned vs just-in-time
You don’t have to load models by hand. Pin the ones you use constantly — they stay resident and answer with no load delay. Everything else loads just-in-time on its first request and auto-evicts after an idle timeout you set. Pinned models are never evicted. The mechanics are in Memory & loading.
One memory budget
Apple Silicon shares a single pool of unified memory across the CPU and GPU. Bedrova treats that pool as one budget and tracks live usage for every loaded model — weights, KV cache, and working set. Two rules follow from this:
- Installed is not loaded. A model on disk costs disk. It only costs memory once it’s loaded. Live memory totals count loaded and resident models only.
- A fit-check runs before every load and download. It reserves headroom for the rest of the system (roughly a quarter of unified memory), then allows, warns, or refuses — and when something won’t fit it suggests a smaller quant or model. See What your Mac can run.
Screenshot: the memory monitor showing several models sharing the budget — added at launch.