LocalEngine API Reference

Overview

Enable the server in Settings → Local API, then copy the generated token. The API is disabled by default and listens on loopback only. Select the appropriate chat, embedding, decision, or image model in the app before calling its endpoint.

Base URL

HTTP:       http://127.0.0.1:38680
OpenAI:     http://127.0.0.1:38680/v1
Anthropic:  http://127.0.0.1:38680
WebSocket:  ws://127.0.0.1:38680/v1/chat/completions/ws

The port is configurable in Settings. OpenAI clients use the base URL ending in /v1; Anthropic clients use the server root.

Authentication

A token is required for every inference, model-list, and status request. Copy it from Settings → Local API; a dummy key will fail. Tokens persist in the macOS Keychain. Regenerate replaces the token and immediately invalidates the previous one.

export LOCALENGINE_TOKEN="le-REPLACE_WITH_YOUR_TOKEN"

# Use one of these HTTP headers:
Authorization: Bearer <token>
x-api-key: <token>
x-privy-token: <token>

Only the four health and discovery routes below are public. Browser clients should use Authorization or x-api-key; CORS preflight permits these and the supported OpenAI and Anthropic SDK headers. Error responses also carry CORS headers.

Health & discovery

GET /health
GET /v1/health
GET /v1/capabilities
GET /v1/openapi.json

curl http://127.0.0.1:38680/health

Health reports server liveness and the installed app version, not model readiness. Capabilities lists registered routes; openapi.json describes the running server's routes.

{
  "status": "ok",
  "app": "LocalEngine",
  "version": "0.7.0"
}

GET /v1/status

Reports the active chat runtime and model. Runtime is localengine-llama or localengine-mlx. When no chat model is selected, ready is false and active_model is omitted.

curl http://127.0.0.1:38680/v1/status \
  -H "Authorization: Bearer $LOCALENGINE_TOKEN"

{
  "runtime": "localengine-llama",
  "active_model": "qwen3.5-9b-ud-q4_k_xl.gguf",
  "ready": true,
  "local_api": "running"
}

GET /v1/models

Lists registered models and installed image models. If the model registry is empty, runtime aliases are returned. Sending anthropic-version switches to the Anthropic list shape with display_name, created_at, has_more, first_id, and last_id.

{
  "object": "list",
  "data": [
    {
      "id": "qwen3.5-9b-ud-q4_k_xl.gguf",
      "object": "model",
      "created": 0,
      "owned_by": "localengine"
    }
  ]
}

For chat, model selects a runtime, not a different loaded model. A registered name or ID uses its runtime; names containing mlx or llama select that runtime; otherwise the active runtime is used. Select the desired model in the app. Replies report the active model.

POST /v1/chat/completions

messages must be a non-empty array. Supported roles are system, developer, user, assistant, and tool. model defaults to localengine and stream defaults to false. temperature is optional; max_completion_tokens takes precedence over max_tokens, then the app's token limit is used.

curl http://127.0.0.1:38680/v1/chat/completions \
  -H "Authorization: Bearer $LOCALENGINE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "localengine",
  "messages": [
    {
      "role": "user",
      "content": "Hello!"
    }
  ],
  "temperature": 0.7,
  "max_tokens": 512,
  "stream": false
}'

{
  "id": "chatcmpl-example",
  "object": "chat.completion",
  "created": 1793404800,
  "model": "qwen3.5-9b-ud-q4_k_xl.gguf",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Hello."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 2,
    "completion_tokens": 2,
    "total_tokens": 4
  }
}

content accepts a string or text/image parts. Images require a vision-capable model and use base64 data URLs, local file URLs, or absolute paths accessible to the app. Remote image URLs are rejected.

{
  "role": "user",
  "content": [
    {
      "type": "text",
      "text": "Describe this image."
    },
    {
      "type": "image_url",
      "image_url": {
        "url": "data:image/png;base64,REPLACE_WITH_BASE64",
        "detail": "auto"
      }
    }
  ]
}

With stream: true, SSE data contains chat.completion.chunk objects and ends with [DONE]. stream_options.include_usage adds a final chunk with empty choices and usage totals. Chat usage is an estimate of roughly four characters per token.

data: {"id":"chatcmpl-example","object":"chat.completion.chunk","created":1793404800,"model":"localengine","choices":[{"index":0,"delta":{"content":"Hello"}}]}

data: [DONE]

Tools & reasoning

Function tools work with the LlamaKit runtime; MLX answers with text. tool_choice accepts auto, none, required, or a named function. Required or named calls are best effort. Return tool results with role: tool and the matching tool_call_id.

{
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get weather for a city",
        "parameters": {
          "type": "object",
          "properties": {
            "city": {
              "type": "string"
            }
          },
          "required": [
            "city"
          ]
        }
      }
    }
  ],
  "tool_choice": "auto"
}

Tool replies contain message.tool_calls and finish_reason: tool_calls. Streaming calls arrive together in delta.tool_calls once their markup is complete. Reasoning is separated into message.reasoning_content or delta.reasoning_content; this is a LocalEngine extension.

WS /v1/chat/completions/ws

Send the chat request JSON as a text frame. Non-streaming requests return one completion frame; streaming requests return chunk frames followed by [DONE]. Browser clients can authenticate with the bearer subprotocol pair or a token query parameter.

const socket = new WebSocket(
  "ws://127.0.0.1:38680/v1/chat/completions/ws",
  ["bearer", token] // token copied from Settings → Local API
);
socket.onopen = () => socket.send(JSON.stringify({
  model: "localengine",
  messages: [{ role: "user", content: "Hello!" }],
  stream: true
}));
socket.onmessage = ({ data }) => {
  if (data === "[DONE]") return;
  console.log(JSON.parse(data));
};

// Alternative URL:
// ws://127.0.0.1:38680/v1/chat/completions/ws?token=le-YOUR_TOKEN

POST /v1/messages

Anthropic-compatible messages require a positive max_tokens and non-empty user/assistant messages. system accepts text or text blocks. Use the server root as the Anthropic base URL and the generated token as the API key.

export ANTHROPIC_BASE_URL="http://127.0.0.1:38680"
export ANTHROPIC_API_KEY="$LOCALENGINE_TOKEN"

curl http://127.0.0.1:38680/v1/messages \
  -H "x-api-key: $LOCALENGINE_TOKEN" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d '{"model":"localengine","max_tokens":512,"messages":[{"role":"user","content":"Hello!"}]}'

{
  "id": "msg_example",
  "type": "message",
  "role": "assistant",
  "model": "qwen3.5-9b-ud-q4_k_xl.gguf",
  "content": [
    {
      "type": "text",
      "text": "Hello."
    }
  ],
  "stop_reason": "end_turn",
  "stop_sequence": null,
  "usage": {
    "input_tokens": 2,
    "output_tokens": 2
  }
}

Content supports text, base64 images, text documents, assistant tool_use, and user tool_result blocks. URL images and PDFs are unsupported. Client tools use name, description, and input_schema; server tools are skipped. tool_choice supports auto, none, any, and tool; forced calls are best effort on LlamaKit.

temperature and stop_sequences are honored. thinking.type enabled or adaptive returns thinking blocks with an empty signature; budget_tokens is ignored. top_p, top_k, metadata, service_tier, and cache_control are ignored. Usage is estimated; hitting max_tokens still reports end_turn.

Streaming emits message_start, content_block_start, content_block_delta, content_block_stop, message_delta, and message_stop, with ping events while waiting. Deltas may be text_delta, thinking_delta, or input_json_delta. Stop reasons are end_turn, tool_use, or stop_sequence; failures after streaming starts arrive as error events.

POST /v1/messages/count_tokens

Accepts the Messages request body without requiring max_tokens. Returns a length-based input token estimate, not an exact tokenizer count.

curl http://127.0.0.1:38680/v1/messages/count_tokens \
  -H "Authorization: Bearer $LOCALENGINE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "localengine",
  "messages": [
    {
      "role": "user",
      "content": "Hello!"
    }
  ]
}'

{
  "input_tokens": 2
}

POST /v1/embeddings

Import a Llama/GGUF embedding model and choose Use for Embeddings. input accepts a string or a non-empty array of strings. Omit model to use the selected embedding model, or pass its exact ID or name. A missing or mismatched model returns model_not_found.

curl http://127.0.0.1:38680/v1/embeddings \
  -H "Authorization: Bearer $LOCALENGINE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
  "input": [
    "Hello world",
    "Local inference"
  ]
}'

Returns an OpenAI list with data entries containing object: embedding, index, and a numeric embedding array, plus model and estimated usage.prompt_tokens and usage.total_tokens. Token-ID inputs, dimensions, and encoding_format are not implemented.

POST /v1/systemone

Select Clef or Clef Flash with Use for Decisions. state accepts text or JSON; questions maps IDs to typed questions. choice uses an option-to-description object (up to 255 options), score uses 2–10 level descriptions, and noul is a true/false decision. instructions defaults to the question ID.

curl http://127.0.0.1:38680/v1/systemone \
  -H "Authorization: Bearer $LOCALENGINE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "clef",
  "state": "Checkout is failing and orders are blocked.",
  "questions": {
    "team": {
      "type": "choice",
      "criteria": {
        "billing": "Payments or invoices",
        "technical": "Bugs or outages"
      }
    },
    "urgency": {
      "type": "score",
      "criteria": [
        "Can wait",
        "This week",
        "Today"
      ]
    },
    "outage": {
      "type": "noul",
      "instructions": "Is a service down?"
    }
  }
}'

{
  "model": "clef",
  "answers": {
    "team": {
      "type": "choice",
      "choice": "technical",
      "confidence": 0.96,
      "probabilities": {
        "billing": 0.02,
        "technical": 0.98
      }
    },
    "urgency": {
      "type": "score",
      "score": 1.9,
      "confidence": 0.85,
      "legend": {
        "0": "Can wait",
        "1": "This week",
        "2": "Today"
      },
      "probabilities": {
        "0": 0.01,
        "1": 0.08,
        "2": 0.91
      }
    },
    "outage": {
      "type": "noul",
      "noul": 0.97
    }
  },
  "usage": {
    "input_tokens": 212,
    "output_tokens": 0
  }
}

Answers include probabilities; score is an expected level index and noul is the probability of true. model is echoed, while the selected decision model performs inference. Optional images are base64 data URLs and require the vision projector; videos are unsupported. Requests run one at a time with an 8,192-token context limit.

POST /v1/images/generations

Download or import an image model first. prompt must be non-empty; only n: 1 and response_format: b64_json are supported. Omit model to use the active image model, or supply an installed image model ID. Omitted size or auto uses the model family's default size.

curl http://127.0.0.1:38680/v1/images/generations \
  -H "Authorization: Bearer $LOCALENGINE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
  "prompt": "A watercolor landscape with mountains and a lake",
  "n": 1,
  "size": "1024x1024",
  "response_format": "b64_json",
  "seed": 42
}'

{
  "created": 1793404800,
  "data": [
    {
      "b64_json": "<base64 PNG>",
      "seed": 42
    }
  ],
  "output_format": "png",
  "size": "1024x1024"
}

Local sampling options are steps, guidance_scale (or cfg_scale), negative_prompt, seed, sampling_method, and flow_shift. Defaults depend on the model family. quality, output_format, output_compression, background, and partial_images are forwarded to the image provider; support depends on that provider.

stream: true emits an image_generation.completed SSE event with b64_json, created_at, size, output_format, and optional seed. Keepalive comments arrive every 10 seconds; failures arrive as error events. This route does not emit chat chunks or [DONE]. Allow time for local image generation.

Errors & compatibility

Missing or invalid tokens return 401 invalid_token. Invalid input, missing models, and context overflow return 400. Unavailable image generation returns 503 image_generation_unavailable. Standard errors contain error.code and error.message; Messages routes use the Anthropic error envelope.

{
  "error": {
    "code": "invalid_token",
    "message": "Invalid or missing bearer token"
  }
}

{
  "type": "error",
  "error": {
    "type": "authentication_error",
    "message": "Invalid or missing bearer token"
  }
}

Compatibility covers the routes documented here, not the entire OpenAI or Anthropic API. The current server has no /v1/responses, /v1/translate, or /v1/translate/batch route. To translate, send a translation instruction through chat completions or Messages.

curl http://127.0.0.1:38680/v1/chat/completions \
  -H "Authorization: Bearer $LOCALENGINE_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
  "model": "localengine",
  "messages": [
    {
      "role": "system",
      "content": "Translate the user text into Simplified Chinese. Return only the translation."
    },
    {
      "role": "user",
      "content": "Hello world"
    }
  ]
}'