rubyllm.com

# RubyLLM Documentation

> Developer documentation for RubyLLM 2.1.1 (stable).

These docs cover RubyLLM 2.1.1. RubyLLM is the Ruby AI framework: chats, tools, agents, MCP, structured output, evaluations, images, video, audio, OCR, embeddings, reranking, and Rails integration behind one API.

## Quick Reference

```ruby
# Install: bundle add ruby_llm
RubyLLM.configure do |config|
  config.openai_api_key = ENV["OPENAI_API_KEY"]
end

# Chat
chat = RubyLLM.chat
chat.with_instructions("Answer in one sentence.").ask "What is Ruby?"
chat.ask "What's in this image?", with: "photo.jpg"

# Tool: parameters come from the keyword arguments of execute
class Weather < RubyLLM::Tool
  description "Gets current weather for a location"

  def execute(latitude:, longitude:)
    WeatherService.current(latitude:, longitude:)
  end
end

chat.with_tools(Weather).ask "Should I bike to work in Berlin today?"

# Agent: reusable instructions, tools, and options
class SupportAgent < RubyLLM::Agent
  instructions "Accept returns of unopened items within 30 days."
  tools Weather
end

SupportAgent.new.ask "Can I return an unopened item after 14 days?"

# Rails: bin/rails generate ruby_llm:install && bin/rails db:migrate
class Chat < ApplicationRecord
  acts_as_chat
end

Chat.create!(model: "gpt-5.6-luna").ask "Hello"
```

- Check the installed version with `RubyLLM::VERSION`. The root docs cover the latest release, `/next/` covers `main`, and `/v1/` covers 1.x.
- Use `instructions`, never `system_prompt`. Use `max_output_tokens`, not `max_tokens`, and `thinking`, not `reasoning`.
- Every `with_*` method returns the chat, so calls chain. Pass `provider:` as a keyword. Keep `provider_options` for fields only one provider has.
- Read structured output with `message.parsed`, tokens with `message.tokens`, and cost with `message.cost`. An unknown cost is `nil`.
- Verify model IDs with `RubyLLM.models.find(id)` instead of guessing them.

The canonical docs cover installation, configuration, providers, core features, agents, evaluations, observability, Rails integration, and the complete reference. Also available: the [API reference Markdown index](https://rubyllm.com/api/index.md), the live [model registry as JSON](https://rubyllm.com/models.json), the [source](https://github.com/crmne/ruby_llm), and the [RubyGems package](https://rubygems.org/gems/ruby_llm).

## Documentation by Area

- [Tools and agents](https://rubyllm.com/llms-tools-and-agents.txt): Tools, MCP, agents, prompt templates, the agentic loop, durable agents, code execution, skills, judgments, and evaluations.
- [Rails](https://rubyllm.com/llms-rails.txt): Active Record persistence, Active Storage attachments, Hotwire streaming, generators, and background jobs.
- [Individual operations](https://rubyllm.com/llms-operations.txt): Images, video, speech, transcription, OCR, moderation, embeddings, reranking, RAG, files, batches, tokenization, and hosted research.
- [Reference](https://rubyllm.com/llms-reference.txt): Provider setup, models, custom providers, releases, and upgrade guides.
- [Core](https://rubyllm.com/llms-core.txt): Installation, configuration, chat, streaming, attachments, structured output, thinking, caching, cost tracking, errors, and observability.
- [Complete documentation](https://rubyllm.com/llms-full.txt): Every page in one file.

## Overview

- [RubyLLM](https://rubyllm.com/index.md): Build and evaluate AI features in Ruby and Rails with chats, tools, agents, images, audio, and video across 19 providers.

## Start Here

- [Getting Started](https://rubyllm.com/getting-started.md): Install RubyLLM and build with chats, tools, agents, images, video, audio, and document processing in Ruby and Rails.
- [How RubyLLM Works](https://rubyllm.com/overview.md): Understand RubyLLM's public API, its providers and protocols, and the services they share.
- [Configuration](https://rubyllm.com/configuration.md): Set provider credentials, choose default models, and share configuration across the RubyLLM API.
- [Provider Setup and Custom Endpoints](https://rubyllm.com/configuration-providers.md): Configure provider credentials and connect to hosted or local APIs through compatible endpoints.
- [Connection, Logging and Contexts](https://rubyllm.com/configuration-connection.md): Timeouts, retries, proxies, connection reuse, logging, the model registry file, and isolated multi-tenant contexts.
- [What's New in 2.1](https://rubyllm.com/whats-new-in-2-1.md): Connect to MCP servers, ask typed judgments, evaluate your agents, and trace them with OpenTelemetry in RubyLLM 2.1.
- [What's New in 2.0](https://rubyllm.com/whats-new-in-2-0.md): Explore the expanded provider coverage, new AI operations, conversation controls, and Rails integration in RubyLLM 2.0.
- [AI Coding Assistants](https://rubyllm.com/ai-coding-assistants.md): Install the RubyLLM skill to help your coding assistant use the API and documentation that match your application.

## Core Features

- [Chat](https://rubyllm.com/chat.md): Start a conversation, give it instructions, and switch models while keeping the same Ruby API.
- [Attachments](https://rubyllm.com/attachments.md): Ask questions about images, recordings, videos, and documents through one attachment API.
- [Stream Responses](https://rubyllm.com/streaming.md): Learn how to display AI responses in real-time as they're generated
- [Structured Output](https://rubyllm.com/structured-output.md): Get AI responses that match an exact JSON schema with required fields and types
- [Extended Thinking](https://rubyllm.com/thinking.md): Give reasoning models more time and budget to deliberate, with optional access to thinking output
- [Citations](https://rubyllm.com/citations.md): Get verifiable answers with normalized citations pointing at documents and web sources, on every provider that supports them
- [Prompt Rendering](https://rubyllm.com/prompt-rendering.md): Render reusable ERB prompt templates from app/prompts with RubyLLM.render_prompt
- [Prompt Caching](https://rubyllm.com/prompt-caching.md): Reuse stable prompt prefixes with automatic or explicit prompt caching
- [Advanced Request Control](https://rubyllm.com/chat-request-control.md): Reach provider-specific features with custom parameters, wire protocols, request hooks, and HTTP headers
- [Tokens and Costs](https://rubyllm.com/cost-and-usage-tracking.md): Read normalized token counts and costs for every response, chat, and provider attempt, including retries and cancellations.
- [Chat Event Handlers](https://rubyllm.com/chat-callbacks.md): Hook into the chat lifecycle with additive callbacks for UI updates, logging, and analytics
- [Tools](https://rubyllm.com/tools.md): Let AI call your Ruby code. Connect to databases, APIs, or any external system with function calling.
- [Tool Parameters](https://rubyllm.com/tool-parameters.md): Declare tool arguments - from inferred signatures to full JSON Schema, rich return values, and provider-specific metadata.
- [Controlling Tool Execution](https://rubyllm.com/tool-execution.md): Choose tools, require approval, run calls concurrently, and observe their results
- [MCP Client](https://rubyllm.com/mcp.md): Connect your chats and agents to Model Context Protocol servers and use their tools, resources, and prompts
- [Provider Tools](https://rubyllm.com/provider-tools.md): Search the web, run code, and connect remote MCP tools through the same chat API
- [Tool Search](https://rubyllm.com/tool-search.md): Give a model hundreds of tools without sending them all, and let it search for the ones it needs.
- [Embeddings](https://rubyllm.com/embeddings.md): Create vectors from text, images, audio, video, and documents for search and similarity
- [Reranking](https://rubyllm.com/rerank.md): Order candidate documents by how well they answer a query, the second stage of a retrieval pipeline
- [Image Generation](https://rubyllm.com/image-generation.md): Generate and edit images from text prompts, reference images, and masks
- [Judgments](https://rubyllm.com/judgments.md): Ask typed questions about your application data and receive probabilities, choices, and scores
- [Video Generation](https://rubyllm.com/video-generation.md): Generate, edit, and extend videos from prompts, images, and audio
- [Audio Transcription](https://rubyllm.com/audio-transcription.md): Convert speech to text with support for multiple languages and speaker diarization
- [Text to Speech](https://rubyllm.com/text-to-speech.md): Convert text into spoken audio
- [Document OCR](https://rubyllm.com/ocr.md): Extract text, tables, and images from PDFs and scans as clean markdown
- [Moderation](https://rubyllm.com/moderation.md): Screen text and images for harmful content before it reaches your models
- [Files](https://rubyllm.com/files.md): Upload files once, reuse them in chats and batches, and download provider results.
- [Tokenization](https://rubyllm.com/tokenization.md): Inspect token IDs and count a conversation's input tokens before generating a response.

## Building Real Apps

- [Rails Integration](https://rubyllm.com/rails.md): Use the RubyLLM API with Active Record, Active Storage, Hotwire, and your existing Rails jobs.
- [Persistence with acts_as](https://rubyllm.com/rails-persistence.md): Persist application-owned chats and messages while RubyLLM manages its supporting records.
- [Streaming with Hotwire/Turbo](https://rubyllm.com/rails-streaming.md): Broadcast AI responses token by token to the browser with Turbo Streams and background jobs.
- [Generators and App Conventions](https://rubyllm.com/rails-generators.md): Scaffold chats, a chat UI, agents, tools, and schemas with RubyLLM's Rails generators and conventions.
- [Advanced Rails Configuration](https://rubyllm.com/rails-advanced-config.md): Route models through different providers, use per-tenant contexts, persist cache boundaries, adjust provider payloads per request, and run fiber-safe.
- [Agents](https://rubyllm.com/agents.md): Give agents instructions, tools, and schemas in a Ruby class, then use them in scripts, services, and Rails jobs.
- [Agentic Workflows](https://rubyllm.com/agentic-workflows.md): Compose agents and AI operations with ordinary Ruby methods, from sequential steps to routing and parallel work.
- [Retrieval-Augmented Generation (RAG)](https://rubyllm.com/rag.md): Retrieve relevant context from your own documents, then answer with that context
- [Durable Agents](https://rubyllm.com/durable-agents.md): Run agent turns as background jobs that survive crashes, deploys, and the wait for a human decision.
- [Memory](https://rubyllm.com/memory.md): Give agents short-term memory through the transcript and long-term memory your app owns, recalled by meaning.
- [Batches](https://rubyllm.com/batches.md): Process chats and embeddings in batches, then collect their results when they are ready.
- [Scale with Async](https://rubyllm.com/async.md): Run AI jobs with Solid Queue fiber workers, use Async for concurrent Ruby calls, and choose Async::Job for higher throughput.
- [Instrumentation and Observability](https://rubyllm.com/instrumentation.md): Observe RubyLLM model operations, provider attempts, tool calls, workflows, batches, and model refreshes
- [OpenTelemetry](https://rubyllm.com/opentelemetry.md): Trace RubyLLM model calls, tools, and workflows with your application's OpenTelemetry setup
- [Error Handling](https://rubyllm.com/error-handling.md): Rescue provider errors, fall back to other models, and let automatic retries absorb transient failures
- [Hosted Research](https://rubyllm.com/hosted-research.md): Run a hosted research agent with remote tools and collect its report through a durable job ID
- [Evaluations](https://rubyllm.com/evaluations.md): Measure how well your agents answer, with reusable datasets, model grading, and Ruby assertions
- [Evaluation Datasets](https://rubyllm.com/evaluation-datasets.md): Define evaluation cases in YAML, JSON, JSONL, or application code
- [Evaluating Conversations and Tools](https://rubyllm.com/evaluation-conversations.md): Evaluate multi-turn conversations, saved transcripts, tool calls, and your own result objects
- [Evaluators](https://rubyllm.com/evaluation-evaluators.md): Decide what a good answer means and which model, Agent, or Judge assesses it
- [Running Evaluations](https://rubyllm.com/evaluation-running.md): Run evaluations from Rake, RSpec, Minitest, or Ruby and read their results, tokens, and costs
- [Evaluation Progress and Monitoring](https://rubyllm.com/evaluation-progress.md): Show evaluation progress in your application, save trials as they finish, and trace runs
- [Evaluation Examples](https://rubyllm.com/evaluation-examples.md): Complete evaluations for formatting, grounded answers, and safe tool use that you can adapt

## Reference

- [API Reference](https://rubyllm.com/api-reference.md): RDoc documentation for every public RubyLLM class and method
- [Model Registry](https://rubyllm.com/models.md): Find models by provider or capability, inspect their pricing, and refresh the registry in plain Ruby and Rails.
- [Model Resolution](https://rubyllm.com/model-resolution.md): How RubyLLM turns a model name into a concrete model and provider, step by step, covering aliases, the registry, provider preference, and unlisted models.
- [Custom Endpoints and Unlisted Models](https://rubyllm.com/custom-endpoints.md): Target OpenAI-compatible endpoints and use model IDs the registry doesn't list
- [Custom Providers and Protocols](https://rubyllm.com/custom-providers.md): Teach RubyLLM to talk to a new AI service by writing a provider, a protocol, or both, and ship it as a gem.
- [Upgrading](https://rubyllm.com/upgrading.md): Move a RubyLLM 2.0 application to 2.1, one release at a time.
- [RubyLLM Ecosystem](https://rubyllm.com/ecosystem.md): Extend RubyLLM with structured schemas, instrumentation, monitoring and community-built tools for production AI apps.

## Pages

- [About RubyLLM](https://rubyllm.com/about.md): RubyLLM is an open source Ruby gem for building AI applications across major AI providers with one consistent Ruby framework.