Open Source AI Tooling Go SDK + model server
Go With Your Own Intelligence
Use Go for hardware-accelerated local inference with llama.cpp — and extend into speech and image generation through Bucky and Malina. Kronk provides a high-level SDK and a production-ready model server so you can run AI with fewer moving parts.
brew install ardanlabs/kronk/kronkkronk server start✓ installed kronk✓ model server ready- listen : http://localhost:11435- api : OpenAI + Anthropic compatible- llama.cpp
- yzma
- GGUF
- CUDA
- Metal
- Vulkan
- ROCm
- OpenAI API
- Anthropic Messages
- OpenWebUI
- OpenCode
- Claude Code
- whisper.cpp
- stable-diffusion.cpp
Watch the Overview
See the Kronk ecosystem in action
Bill Kennedy walks through Kronk from the ground up: running Go-native inference on hardware you control, managing and testing models in the browser UI, and how context caching keeps generations fast.
- Go-native AI inference
- Model server & web UI
- Context caching
Why Kronk
Built for production-minded Go teams
Kronk explores a simpler path to production AI: bring inference closer to your Go application, reduce unnecessary services, and keep performance, cost, and control in your hands.
Talk to an EngineerFewer moving parts
Bring inference into your Go application instead of treating the model server as a permanent dependency.
Run on your hardware
GPU acceleration via CUDA, Metal, Vulkan, and ROCm — with automatic hardware detection and verified library installs.
Familiar APIs
OpenAI-compatible Chat Completions, Responses, embeddings, and reranking — plus an Anthropic-compatible Messages API.
Built for Go teams
Designed for engineers who want local-first systems they can own, operate, and ship with confidence.
SDK or model server
Same public APIs. Two ways to run.
The model server is built on the same SDKs available to Go applications. Choose the path that matches how your system needs to own inference.
Embedded
Use the SDK when you need
- 01 Your Go application
- 02 Kronk SDK
- 03 Local model + GPU
- Inference inside a Go process
- Direct control over model loading and lifetime
- No separate server process
- Application-specific caching and concurrency
Served
Use the model server when you need
- 01 OpenWebUI · OpenCode · Claude Code
- 02 kronk server :11435
- 03 Local model + GPU
- HTTP APIs for one or more clients
- OpenAI- and Anthropic-compatible endpoints
- Browser-based model management and testing
- Authentication, rate limiting, metrics, and tracing
API surface
Compatible where it matters
Point existing clients and agents at a local Kronk endpoint, without rewriting your tool chain around a new protocol.
Explore Kronk Examples-
Chat Completions
/v1/chat/completionsOpenAI-compatible chat for tools, agents, and existing OpenAI clients pointed at a local endpoint. -
Responses API
/v1/responsesModern response workflows for applications that need structured generation beyond basic chat turns. -
Anthropic Messages
/v1/messagesAnthropic-compatible Messages API so Claude-oriented tooling can talk to local models through Kronk. -
Embeddings & reranking
/v1/embeddings · /v1/rerankLocal embedding and reranking surfaces for RAG pipelines without sending documents off-network. -
Decisions
/v1/decideEvaluate typed decision questions with a decision model and get structured answers back. -
Audio transcription
/v1/audio/transcriptionsOpenAI-style speech-to-text and translation powered by Bucky and whisper.cpp.
Capabilities
What you get
Hardware accelerated
GPU acceleration across major backends so models run where your capacity already lives.
Streaming & reasoning
Stream tokens as they generate, with support for reasoning-style workloads in the Go SDK.
Tool calling
Native function calling so models can invoke your Go functions with structured input and output.
Vision models
Prompt multimodal models with images alongside text for grounded local inference.
Model server & Browser UI
Manage models, test prompts, and integrate with OpenWebUI, OpenCode, and Claude Code.
Security & observability
Authentication, rate limiting, metrics, and tracing when you need a multi-client production surface.
The local AI stack
One foundation. Growing capabilities.
Kronk is the core. Bucky and Malina extend the same local-first philosophy into speech and image generation while keeping Go developers close to the system.
Get started
Install Kronk
On macOS or Linux, Homebrew is the recommended path. Start the model server, open the Browser UI, and the first model or SDK example can download compatible native libraries automatically.
brew install ardanlabs/kronk/kronkgo install github.com/ardanlabs/kronk/cmd/kronk@latestkronk server start# Browser UI → http://localhost:11435git clone https://github.com/ardanlabs/kronk.gitcd kronkmake example-questionPlatforms
Where Kronk runs
Compatible native library bundles are downloaded per host. Use the CLI as the source of truth for combinations supported by your install.
Linux
- CPU
- amd64 · arm64
- GPU
- CUDA Vulkan ROCm (amd64)
macOS
- CPU
- amd64 · arm64
- GPU
- Metal (Apple Silicon)
Windows
- CPU
- amd64 · arm64
- GPU
- CUDA Vulkan ROCm (amd64)
From the Lab · Featured Read
Kronk AI: A Simpler Way to Build and Run AI Applications
Go deeper into the engineering question that started Kronk: why local-first inference changes the architecture, and what happens when the application owns more of its execution path.
“Why should you need a model server to deploy an AI application? Why can’t your AI app be the model server as well?”Read the Article
Build AI systems you can own
Talk with Ardan Labs about local inference, private AI infrastructure, and production systems beyond API prototypes.