Docs
Vision & perception
Ask questions about images with local vision models, or run Core ML perception tasks for typed image analysis.
Vision-language models let you ask about images, screenshots, and photos — locally. Core ML perception models handle typed image-analysis tasks such as classification, detection, segmentation, depth, OCR, and pose.
Screenshot: a vision chat with an attached image — added at launch.
Models
Qwen-VL and other MLX-VLM models power chat about images. Core ML perception packages power analysis artifacts. Text-only models explicitly refuse image input, so you always know what you’re running.
How it works
Attach one or more images to a chat. Bedrova validates and orientation-corrects them, caps
the pixel budget, and packs multiple images per message. Repeated chats reuse cached vision
features. Images can be sent as OpenAI-compatible image_url content blocks or native media
inputs.
For perception, choose an Analyze action in Media or call the native API. Results are typed: labels, boxes, masks, depth maps, OCR text, or pose keypoints, with artifacts saved locally.
API
Send image content blocks to POST /v1/chat/completions (or /anthropic/v1/messages) with
a vision model selected. Speculative decoding is disabled for vision models.
Run perception via POST /api/v1/perception/{task} with image-classification,
object-detection, segmentation, depth-estimation, ocr, or pose-detection.