Skip to content
Bedrova bedrova
FeaturesCompatibilityDevelopersCompareDocs Coming soon

Docs

Memory & loading

Just-in-time loading, idle TTL, auto-evict, pinning, live memory accounting, and guardrail modes.

Bedrova manages which models are in memory so you don’t have to babysit it. This page explains the loading lifecycle, the live accounting behind the memory bar, and the guardrails that keep a load from taking the app down.

The loading lifecycle

  • Just-in-time (JIT). A model that isn’t resident loads on its first request — a chat message, an API call, a tool that needs it. The first request waits for the load; later ones are fast.
  • Idle TTL. After a model sits unused for a timeout you set, it auto-evicts to free memory. Set the TTL in Settings → Server/API.
  • Pin. Mark a model to stay resident. Pinned models load eagerly and are never auto-evicted — use this for the model you chat with or code against all day.

The vocabulary in the app mirrors this: fetch a model (download), stay (pin/keep loaded), drop it (unload).

Live memory accounting

Bedrova tracks live unified-memory use for every loaded model — weights, KV cache, and working set — and shows the combined total. Two rules matter:

  • Loaded, not installed. A model on disk costs disk, not memory. Live totals count loaded and resident models only. A library full of installed models uses no RAM until you load one.
  • Everything shares one budget. A pinned chat model, a GGUF text context, an embedding or search worker, a Core ML perception model, and an image model all draw from the same pool. The monitor shows them together. Background: Concepts and What your Mac can run.

Guardrails

Before any load, the fit-check estimates peak memory against what’s already resident, then allows, warns, or blocks. How strict that is comes from the guardrail mode in Settings → Model Loading Guardrails:

ModeBehavior
StrictBlock anything that wouldn’t fit with full headroom.
BalancedWarn on tight loads, block clear overcommits (default).
RelaxedWarn but let you proceed past most warnings.
OffNo guardrail — you’re on your own.
CustomSet your own size and warning thresholds.

The point of the guardrail is that Bedrova would rather decline a load than crash. If you override it, you accept the memory pressure that follows.

When memory is tight

  • Let an idle model auto-evict, or unload one by hand.
  • Drop to a smaller quant — a 4-bit build frees a lot versus higher precision.
  • Shorten context length to shrink the KV cache.
  • Don’t pin more than you need; each pinned model is permanent memory until you unpin it.

See also