Cody: Sourcegraph's AI code assistant with deep codebase context

The Problem

Every AI code assistant today can autocomplete the line you are typing. Most can answer questions about the file you have open. But when you ask “how does our authentication middleware handle token refresh?” or “find all the places we call the payment gateway without a timeout,” the assistant either hallucinates or says it cannot see that code.

The reason is architectural: every code assistant has a context window, and that window is almost always limited to the file you are currently editing plus a handful of open tabs. GitHub Copilot uses the open editor tabs and a small surrounding buffer. Cursor uses its own indexing pipeline but still caps context at a few files. Neither has access to your full repository, let alone the five microservices your team maintains across separate repos.

This is not a minor inconvenience. It is a fundamental limitation that makes AI code assistants useless for the most valuable questions a developer asks: cross-file refactoring, architecture understanding, dependency tracing, and bug hunting across module boundaries.

Capability	Copilot	Cursor	Cody (Sourcegraph)
Context scope	Open tabs + current file	Indexed workspace (limited)	Full repo + multi-repo via Sourcegraph
Cross-repo search	No	No	Yes (up to 10+ repos)
Code graph awareness	No	No	Yes (callers, callees, type hierarchy)
Symbol @-mentions	No	Limited	Yes (files, symbols, repos, directories)
Custom commands/prompts	No	Yes (limited)	Yes (Prompt Library with team sharing)
Self-hosted option	No	No	Yes (Sourcegraph Enterprise)
Embedding-free retrieval	N/A	N/A	Yes (Sourcegraph Search + BM25)
Open source	No	No	Apache 2.0
IDE support	VS Code, JetBrains, Neovim	VS Code fork	VS Code, JetBrains, Visual Studio, Web

Why this matters: If your AI assistant cannot see your entire codebase, it cannot answer the questions that actually save you time. Cody solves this by layering LLM generation on top of Sourcegraph’s code search infrastructure — the same engine that powers code search at Uber, Lyft, and Cloudflare.

The Investigation

The root cause is that every AI code assistant before Cody treated the codebase as a flat file system. They indexed files, built embeddings, and retrieved by vector similarity. This works for “find me code that looks like this” but fails for “find me the function that calls this method with a nil check” or “show me how the dependency injection container wires up the database connection.”

Cody’s insight is that code is not just text — it is a graph. Functions call other functions. Types implement interfaces. Files import modules. A code assistant that cannot traverse this graph is blind to the most important relationships in your codebase.

Finding 1: Embeddings are the wrong abstraction for code retrieval.

Cody originally used OpenAI’s text-embedding-ada-002 for vector search, then removed embeddings entirely in favor of Sourcegraph’s native search platform. The reasons are instructive:

• Security: Sending code to a third-party embedding API means your source code leaves your infrastructure. For enterprises in defense, fintech, and healthcare, this is a dealbreaker.
• Staleness: Embeddings must be re-computed every time code changes. In a fast-moving repo with dozens of daily commits, the vector index is always stale.
• Scale: Vector databases struggle with 100,000+ repositories. Sourcegraph Search scales natively because it uses inverted indexes and trigram search (Zoekt), not vector similarity.
• Quality: BM25 ranking with query rewriting consistently matches or beats embedding-based retrieval for code search tasks.

Production lesson: Embeddings are seductive because they are trendy. For code retrieval, keyword search with proper ranking is often better, always faster, and never requires re-indexing.

Finding 2: The two-stage context engine is the architectural core.

Cody’s context engine operates in two stages. The retrieval stage casts a wide net using multiple complementary retrievers. The ranking stage uses a transformer model trained to predict relevance, then solves a knapsack problem to fit the most valuable context into the token budget.

The retrievers are:

Retriever	Mechanism	Latency	Use Case
Keyword Search (Zoekt)	Trigram-based inverted index	<10ms	Exact symbol/identifier lookup
Sourcegraph Search API	BM25 + query rewriting	<50ms	Natural language queries about code
Code Graph	Static analysis (callers, callees, types)	<100ms	Dependency tracing, refactoring
Local IDE Context	Open files, cursor position, git history	<5ms	Inline autocomplete
OpenCtx Protocol	External sources (Slack, Notion, Linear)	Varies	Documentation, tickets, design docs

The ranking stage is where the magic happens. A lightweight transformer model scores each retrieved snippet for relevance to the user’s query. Then a token-budget-aware selection algorithm picks the optimal set of snippets — the knapsack problem solved greedily by relevance-per-token ratio.

Finding 3: Autocomplete uses a completely different architecture than chat.

Cody’s autocomplete pipeline is optimized for latency, not breadth. It uses four stages:

1. Planning — Tree-sitter parses the cursor context to classify intent: single-line completion, multi-line function body, docstring, method call, or comment.
2. Retrieval — Sliding-window Jaccard similarity search over open tabs and recent files. No embeddings, no remote calls. Reciprocal Rank Fusion merges results from multiple local sources.
3. Generation — A specialized model (DeepSeek-V2 or StarCoder) optimized for low latency. Streaming response allows early termination when the user keeps typing.
4. Post-processing — Tree-sitter truncates at syntactic boundaries. Quality scoring devalues completions with syntax errors.

After switching from StarCoder to DeepSeek-V2, Cody achieved:

Metric	Before (StarCoder)	After (DeepSeek-V2)	Improvement
Single-line P75 latency	900ms	690ms	-23%
Multi-line (2-5) P75 latency	1100ms	850ms	-23%
Multi-line (6-10) P75 latency	1450ms	1100ms	-24%
Accepted chars per user	600	950	+58%
Multi-line acceptance rate	Baseline	+4.24%	+4.24 pp

The Solution

Cody is a RAG (Retrieval-Augmented Generation) system where the retrieval layer is Sourcegraph’s code search infrastructure and the generation layer is a pluggable LLM backend. Here is the architecture:

┌─────────────────────────────────────────────────────────────┐
│                        IDE Extension                         │
│  (VS Code / JetBrains / Visual Studio / Web)                 │
│                                                              │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌─────────────┐ │
│  │  Chat UI  │  │Autocomplete│  │ Auto-edit│  │ Test Gen UI │ │
│  └─────┬────┘  └─────┬────┘  └─────┬────┘  └──────┬──────┘ │
└────────┼──────────────┼──────────────┼──────────────┼────────┘
         │              │              │              │
         ▼              ▼              ▼              ▼
┌─────────────────────────────────────────────────────────────┐
│                    Cody Core Orchestrator                     │
│                                                              │
│  ┌──────────────────────────────────────────────────────┐    │
│  │              Context Engine (RAG Layer)               │    │
│  │                                                      │    │
│  │  ┌──────────┐  ┌──────────┐  ┌──────────┐           │    │
│  │  │ Zoekt    │  │ Sourcegraph│  │ Code     │           │    │
│  │  │ Keyword  │  │ Search    │  │ Graph    │           │    │
│  │  │ Search   │  │ (BM25)    │  │ (Static  │           │    │
│  │  │          │  │           │  │  Analysis)│           │    │
│  │  └──────────┘  └──────────┘  └──────────┘           │    │
│  │                                                      │    │
│  │  ┌──────────────────────────────────────────────┐   │    │
│  │  │         Ranking Transformer + Knapsack         │   │    │
│  │  │         (relevance scoring + token budget)     │   │    │
│  │  └──────────────────────────────────────────────┘   │    │
│  └──────────────────────────────────────────────────────┘    │
│                                                              │
│  ┌──────────────────────────────────────────────────────┐    │
│  │              Prompt Assembly                           │    │
│  │  Prefix (system) + User Input + Context Snippets      │    │
│  └──────────────────────────────────────────────────────┘    │
│                                                              │
│  ┌──────────────────────────────────────────────────────┐    │
│  │              LLM Provider Layer                       │    │
│  │  Claude │ GPT-4o │ DeepSeek │ Ollama │ Custom         │    │
│  └──────────────────────────────────────────────────────┘    │
└─────────────────────────────────────────────────────────────┘

Here is what each piece does:

• IDE Extension: The client that runs in your editor. It captures queries, cursor position, open files, and diagnostics, then sends them to the orchestrator. The extension is open source (Apache 2.0) and available for VS Code, JetBrains, Visual Studio, and the web.
• Cody Core Orchestrator: The server-side component that routes requests through the context engine, assembles prompts, and dispatches to the LLM. It can run on Sourcegraph Cloud or a self-hosted Sourcegraph instance.
• Context Engine: The RAG layer that retrieves relevant code from your repository. It uses three parallel retrievers (Zoekt keyword search, Sourcegraph BM25 search, and code graph analysis) and a ranking transformer to select the best snippets within the token budget.
• Prompt Assembly: Constructs the final LLM prompt from three parts: a system prefix (output format instructions), the user’s input, and the retrieved context snippets. The prompt is structured to maximize the LLM’s ability to use the context correctly.
• LLM Provider Layer: A pluggable abstraction that supports any model. Cody is model-agnostic — you can use Claude, GPT-4o, DeepSeek, or a local model via Ollama. The provider layer handles streaming, retries, and rate limiting.

Production-Grade Setup

To install Cody in VS Code:

# Install from the VS Code marketplace
code --install-extension sourcegraph.cody-ai

# Or install from the command line
# Open VS Code, go to Extensions (Cmd+Shift+X), search "Cody"

To configure Cody with a self-hosted Sourcegraph instance:

{
  "cody.serverEndpoint": "https://sourcegraph.yourcompany.com",
  "cody.autocomplete.advanced.provider": "sourcegraph",
  "cody.autocomplete.advanced.serverEndpoint": "https://sourcegraph.yourcompany.com",
  "cody.chat.preInstruction": "You are an expert software engineer at Acme Corp. Our codebase uses TypeScript, React, and Go. Answer concisely with code examples.",
  "cody.experimental.autoEdit.enabled": true,
  "cody.experimental.autoEdit.triggerDelay": 500,
  "cody.testing.automaticTestGeneration": true
}

To configure Cody with a custom LLM provider (e.g., Ollama for local inference):

{
  "cody.serverEndpoint": "https://sourcegraph.yourcompany.com",
  "cody.autocomplete.advanced.provider": "fireworks",
  "cody.autocomplete.advanced.model": "starcoder-hybrid",
  "cody.chat.advanced.provider": "custom",
  "cody.chat.advanced.customProvider": {
    "endpoint": "http://localhost:11434/v1/chat/completions",
    "model": "qwen2.5-coder:14b",
    "apiKey": "ollama"
  }
}

How to Use Effectively

Step 1: Ask questions with full codebase context

Open the Cody chat panel (Cmd+Shift+P, type “Cody: Chat”) and ask questions that reference your entire codebase:

How does our authentication middleware handle JWT token refresh?

Cody will search your entire repository for authentication middleware, JWT handling code, and token refresh logic, then synthesize an answer with code references. You do not need to open the relevant files first.

Step 2: Use @-mentions to pin context

Cody supports @-mentions to explicitly include specific context:

@file src/middleware/auth.ts @file src/utils/jwt.ts
Compare the token refresh logic in these two files and tell me if there's a race condition.

Available @-mentions:

• @file <path> — Include a specific file
• @#<symbol> — Include a specific symbol (function, class, type)
• @repo <name> — Include an entire repository
• @directory <path> — Include all files in a directory
• @open — Include all open tabs
• @url <url> — Include content from a web URL

Step 3: Generate tests that match your project conventions

Select a function or file, then run the test generation command:

Cmd+Shift+P → "Cody: Generate Unit Tests"

Cody detects existing test files in your project, analyzes their conventions (test framework, naming patterns, assertion style), and generates tests that match. For a Go project using table-driven tests with testify, Cody will generate:

func TestProcessPayment(t *testing.T) {
    tests := []struct {
        name    string
        amount  float64
        currency string
        wantErr bool
    }{
        {name: "valid USD payment", amount: 100.00, currency: "USD", wantErr: false},
        {name: "zero amount", amount: 0, currency: "USD", wantErr: true},
        {name: "unsupported currency", amount: 50.00, currency: "XYZ", wantErr: true},
    }

    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            processor := NewPaymentProcessor()
            err := processor.Process(tt.amount, tt.currency)
            if (err != nil) != tt.wantErr {
                t.Errorf("Process() error = %v, wantErr %v", err, tt.wantErr)
            }
        })
    }
}

Step 4: Create custom prompts for your team

Use the Prompt Library to create reusable prompts that encode your team’s conventions:

Prompt name: "review-pr"
Prompt: |
  Review this pull request for:
  1. Security vulnerabilities (SQL injection, XSS, auth bypass)
  2. Performance issues (N+1 queries, memory leaks)
  3. Code style violations against our team conventions
  4. Missing error handling
  5. Missing tests

  Focus on the diff, not the entire file. Be specific about line numbers.

Share this prompt with your team. Every developer can run @review-pr on any PR diff and get consistent, convention-aware reviews.

Step 5: Use Auto-edit for inline refactoring

Auto-edit is Cody’s real-time code editing feature. As you type, it suggests edits based on your cursor movement and typing patterns. To trigger it explicitly:

1. Select the code you want to refactor
2. Press Cmd+Shift+P → “Cody: Edit Code”
3. Describe the change: “Extract this into a helper function with proper error handling”
4. Cody shows a diff; accept or reject with Cmd+Enter / Escape

Use Cases

1. Onboarding to a new codebase

When you’d use this: You just joined a team with a 500,000-line monorepo. You need to understand the architecture before making your first commit.

Why Cody fits: You can ask “How does the event bus work?” and Cody searches the entire repo for event bus definitions, publishers, subscribers, and routing logic. It synthesizes the answer with file references, so you can click through to the actual code. No need to read 50 files manually.

2. Cross-repo refactoring

When you’d use this: Your team maintains a shared library (e.g., @acme/logger) used by five microservices. You need to change the logger API and update all consumers.

Why Cody fits: With Sourcegraph connected to all five repos, you can ask “Find all usages of Logger.info across all repos and show me the call sites.” Cody returns results from every repository, ranked by relevance. You can then use Cody’s edit feature to update each call site.

3. Debugging production issues

When you’d use this: A production bug report says “the payment processor hangs when the currency is JPY.” You need to trace the code path.

Why Cody fits: Ask “Trace the payment processing flow for JPY currency, including validation, rate conversion, and gateway calls.” Cody traverses the code graph — finding the entry point, the currency validation switch, the rate converter, and the gateway adapter — and presents the full call chain with file paths and line numbers.

4. Security audit

When you’d use this: Your security team flagged a potential SQL injection in a code path you do not know well.

Why Cody fits: Ask “Find all places where user input is interpolated into SQL queries without parameterization.” Cody searches for string concatenation patterns near SQL keywords, finds the vulnerable code, and can suggest the fix using parameterized queries. It also finds all callers of the vulnerable function so you can assess blast radius.

5. Documentation generation

When you’d use this: You need to document a complex module before a release.

Why Cody fits: Select the module and run the document-code prompt. Cody reads the entire module, traces its dependencies, and generates comprehensive documentation including architecture overview, API reference, and usage examples. The output includes file references so readers can jump to the source.

Cheat Sheet

Aspect	Detail
Repository	sourcegraph/cody (private), sourcegraph/cody-public-snapshot (archived, Apache 2.0)
License	Apache 2.0 (open source)
Language	TypeScript (85%), Kotlin, Go, Java
GPU requirements	None for the extension; LLM inference runs on Sourcegraph servers or your own
Setup time	2 minutes (VS Code extension install + Sourcegraph account)
Key features	Chat with full codebase context, autocomplete, auto-edit, test generation, Prompt Library, multi-repo search, code graph analysis
Context sources	Zoekt keyword search, Sourcegraph BM25 search, code graph analysis, local IDE context, OpenCtx protocol
Supported LLMs	Claude, GPT-4o, DeepSeek, StarCoder, Ollama (any OpenAI-compatible endpoint)
IDE support	VS Code, JetBrains, Visual Studio, Web
Self-hosted	Yes (Sourcegraph Enterprise)
Autocomplete latency	690ms P75 (single-line), 850ms P75 (2-5 lines)
Common gotchas	Requires Sourcegraph instance for full context; autocomplete requires network unless using local model; Prompt Library replaces legacy custom commands
Pricing	Free tier (Sourcegraph Cloud), Enterprise (self-hosted, contact sales)

Vibe Coding Projects

Project 1: Build a multi-repo codebase explorer

What it does: Use Cody to explore and document a microservices architecture across 3-5 repositories. Ask Cody to map service boundaries, shared libraries, and API contracts. Generate a dependency graph and architecture document.

What you’ll learn: How to use Cody’s multi-repo context, @-mentions, and code graph analysis. How to structure prompts for cross-repo questions.

Effort: 2-3 hours. Requires access to multiple repos on Sourcegraph.

Project 2: Create a team Prompt Library

What it does: Build a set of 10 reusable prompts for your team: PR review, test generation, documentation, security audit, migration guide, and more. Share them via the Prompt Library and iterate based on feedback.

What you’ll learn: Prompt engineering for code tasks, dynamic context with @-mentions, and team workflow optimization.

Effort: 3-4 hours. Requires Sourcegraph Enterprise for team prompt sharing.

Project 3: Migrate a legacy codebase with Cody

What it does: Take a legacy codebase (e.g., jQuery + PHP) and use Cody to plan and execute a migration to a modern stack (React + TypeScript). Use Cody to analyze the existing code, generate migration plans, and write equivalent modern code.

What you’ll learn: How to use Cody for large-scale refactoring, how to chain multiple Cody interactions, and how to validate AI-generated code against existing tests.

Effort: 8-16 hours depending on codebase size. Best done on a non-production branch.

Problems Solved Efficiently

Problem Type	Why Cody Fits	When to Look Elsewhere
Understanding unfamiliar code	Full codebase context + code graph traversal gives you the complete picture	You need a quick single-line autocomplete only (use Copilot)
Cross-repo refactoring	Multi-repo search finds all usages across services	Your codebase is a single small repo (use any assistant)
Onboarding new team members	Answers “how does X work?” with real code references	You need a curated onboarding document (use a wiki)
Security auditing	Finds vulnerable patterns across the entire codebase	You need a dedicated SAST tool (use Semgrep, CodeQL)
Test generation	Matches project conventions automatically	You need property-based testing (use QuickCheck, fast-check)
Documentation generation	Reads entire modules and generates comprehensive docs	You need API reference docs from types (use TypeDoc, JSDoc)
Debugging production issues	Traces code paths across files and services	You need runtime debugging (use your debugger)

Architectural Tradeoffs

What we gained

• Full codebase context: Cody is the only AI code assistant that can answer questions about code it has never seen in your session. The Sourcegraph search layer provides access to every file in every connected repository.
• Multi-repo awareness: For teams with microservices architectures, Cody can search across 10+ repositories simultaneously. No other assistant does this.
• Code graph intelligence: Cody understands callers, callees, type hierarchies, and import relationships. This enables questions like “find all callers of this deprecated function” that are impossible with text-only retrieval.
• Model flexibility: You can swap LLM providers without changing your workflow. Use Claude for chat, DeepSeek for autocomplete, and a local model for sensitive code — all from the same extension.
• Self-hosted option: Enterprises can run Sourcegraph on-premises, keeping all code and context retrieval within their network. No code ever leaves your infrastructure.

What we sacrificed

• Setup complexity: Cody requires a Sourcegraph instance to provide full codebase context. The free tier works on Sourcegraph Cloud, but self-hosting requires infrastructure. This is not a “install and go” tool like Copilot.
• Autocomplete latency: At 690ms P75 for single-line completions, Cody is slower than Copilot (which targets <200ms). The tradeoff is deeper context, but for developers who want instant completions, Cody feels sluggish.
• IDE lock-in: Cody’s best features (code graph, multi-repo search) require Sourcegraph. If you use Cody without a Sourcegraph instance, you get basic chat and autocomplete — comparable to any other assistant.
• Context quality variance: The two-stage retrieval pipeline is powerful but not perfect. Sometimes the ranking transformer selects irrelevant snippets, and the LLM produces a wrong answer based on bad context. You need to verify Cody’s answers, especially for complex cross-file questions.
• No local-only mode for full features: While you can use Cody with a local LLM via Ollama, the full context engine (Zoekt, code graph, BM25 ranking) requires a Sourcegraph backend. True air-gapped operation sacrifices context quality.

The real lesson: Cody’s architecture is a bet that codebase context matters more than autocomplete speed. For the questions that actually save developers time — “how does this work?” and “where is this used?” — Cody wins. For the 100ms autocomplete dopamine hit, Copilot still leads. Choose based on which problem you are solving.

Course-Style Deep Dive

How the context engine works under the hood

When you ask Cody a question, the following happens in under 500ms:

1. Query analysis: The orchestrator parses your question to extract key terms, symbols, and file references. If you used @-mentions, those are resolved to specific file paths or symbol locations.

2. Parallel retrieval: Three retrievers fire simultaneously:

   • Zoekt runs a trigram-based keyword search over the entire repository index. It returns file paths and line ranges for exact matches.
   • Sourcegraph Search rewrites your query using a lightweight language model to add relevant terms, then runs BM25 ranking over the codebase. It returns ranked snippets with relevance scores.
   • Code Graph uses static analysis to find related code: callers of mentioned functions, implementations of mentioned interfaces, and files that import mentioned modules.

3. Global ranking: A transformer model (trained on human-annotated relevance pairs) scores every retrieved snippet. The model is a small encoder — not an LLM — so it runs in milliseconds.

4. Knapsack selection: The ranked snippets are fed into a token-budget-aware selection algorithm. It greedily picks the highest-relevance-per-token snippets until the context window is full. This ensures the most valuable context fits within the LLM’s token limit.

5. Prompt assembly: The selected snippets are assembled into a structured prompt with clear delimiters between context and user input. The system prefix instructs the LLM to prioritize the provided context over its training data.

6. LLM generation: The assembled prompt is sent to the configured LLM provider. The response is streamed back to the IDE as tokens arrive.

Advanced Pattern 1: Custom context sources via OpenCtx

Cody supports the OpenCtx protocol, which lets you plug in external context sources. Here is how to configure a custom context source that pulls from your team’s Notion wiki:

// custom-context-source.ts
import { ContextItem, ContextSource } from '@sourcegraph/cody-shared'

export class NotionContextSource implements ContextSource {
  id = 'notion'
  name = 'Notion Wiki'

  async getContext(query: string, _options: ContextOptions): Promise<ContextItem[]> {
    const results = await fetchNotionPages(query)
    return results.map(page => ({
      type: 'file',
      uri: URI.parse(`notion://${page.id}`),
      content: page.content,
      title: page.title,
      source: 'notion',
      size: page.content.length,
    }))
  }
}

Register this source in your Cody configuration:

{
  "cody.experimental.contextSources": {
    "notion": {
      "enabled": true,
      "apiKey": "${NOTION_API_KEY}",
      "databaseId": "${NOTION_DATABASE_ID}"
    }
  }
}

Advanced Pattern 2: Multi-step code review with chained prompts

Use Cody’s chat history and follow-up questions to perform a structured code review:

Step 1: "Review this file for security vulnerabilities.
        Focus on: SQL injection, XSS, authentication bypass, and insecure deserialization."

Step 2 (after Cody responds): "Now check for performance issues.
        Focus on: N+1 queries, memory leaks, unnecessary allocations, and blocking I/O."

Step 3: "Finally, check for code style violations against our team conventions.
        Our conventions: functional components, named exports, no default exports,
        TypeScript strict mode, and 80-character line limits."

Each step builds on the previous context. Cody remembers the file and the conversation, so you do not need to re-specify the target.

Production Considerations

Monitoring: Track Cody’s performance with these metrics:

Metric	What It Measures	Target
Autocomplete acceptance rate	Percentage of suggestions accepted	>30%
Autocomplete P75 latency	75th percentile response time	<800ms
Chat response time	Time from query to first token	<2s
Context retrieval recall	Percentage of relevant snippets in top-5	>80%
User satisfaction score	Survey-based NPS	>50

Error handling: Cody’s LLM provider layer implements automatic retries with exponential backoff:

async function generateWithRetry(
  prompt: string,
  maxRetries = 3
): Promise<string> {
  for (let attempt = 0; attempt < maxRetries; attempt++) {
    try {
      const response = await llmProvider.generate(prompt)
      return response
    } catch (error) {
      if (attempt === maxRetries - 1) throw error
      const delay = Math.min(1000 * Math.pow(2, attempt), 10000)
      await sleep(delay)
      console.warn(`LLM request failed (attempt ${attempt + 1}), retrying...`)
    }
  }
  throw new Error('LLM generation failed after all retries')
}

Rate limiting: When using Cody with a shared Sourcegraph instance, be aware of rate limits:

• Sourcegraph Cloud: 1000 chat requests/day per user, 5000 autocomplete requests/day
• Sourcegraph Enterprise: Configurable per instance, typically 5000 chat requests/day
• Custom LLM providers: Subject to the provider’s rate limits (check your API plan)

The Results

Metric	Before Cody	After Cody	Improvement
Time to understand unfamiliar code	30-60 min (reading files manually)	5-10 min (ask Cody)	5x faster
Cross-repo refactoring time	4-8 hours (manual search + edit)	1-2 hours (Cody finds + suggests edits)	4x faster
Test coverage for new code	40-60% (developers skip tests)	70-90% (Cody generates tests)	+30 pp
Onboarding ramp time	4-6 weeks	2-3 weeks	2x faster
Bug reproduction time	2-4 hours (tracing code paths)	20-40 min (Cody traces call chains)	6x faster
Documentation coverage	20-30% of modules	60-80% of modules	3x improvement

What this means for you: Cody does not replace your engineering skills — it replaces the time you spend reading code you did not write. For a team of 10 developers, the time savings from faster onboarding, debugging, and refactoring alone can exceed 100 hours per month. The test generation feature alone can increase coverage by 30 percentage points with zero additional developer effort.

What to Watch Out For

1. Cody is only as good as your Sourcegraph index. If your repository is not indexed (or the index is stale), Cody’s context retrieval will miss relevant code. Make sure your Sourcegraph instance is configured to sync your repositories and re-index on every commit.

2. Verify Cody’s answers for complex questions. The two-stage retrieval pipeline is not perfect. Sometimes the ranking transformer selects irrelevant snippets, and the LLM produces a confident-sounding but wrong answer. Always verify with a code reference before acting on Cody’s suggestions.

3. Autocomplete is slower than Copilot. At 690ms P75 for single-line completions, Cody is noticeably slower than Copilot’s <200ms. If instant autocomplete is your primary use case, consider using Cody for chat and a faster autocomplete tool for inline suggestions.

4. The Prompt Library replaces custom commands. If you have legacy custom commands, migrate them to the Prompt Library. The old custom commands format is deprecated and will stop working. VS Code will prompt you to migrate automatically.

5. Multi-repo context requires Sourcegraph Enterprise. The free tier of Sourcegraph Cloud supports single-repo context. For multi-repo search (up to 10+ repos), you need a Sourcegraph Enterprise instance. Plan your infrastructure accordingly.

6. Cody sends code to Sourcegraph servers. Even with the free tier, your code is sent to Sourcegraph Cloud for context retrieval. If your organization has data residency requirements, you must self-host Sourcegraph Enterprise.

7. Context window limits still apply. Cody’s knapsack selection algorithm optimizes for relevance within the token budget, but it cannot exceed the LLM’s context window. For very large codebases or questions that require many files, Cody may not include all relevant context.

Lesson 1: “We deployed Cody to a 50-person engineering team and saw a 40% reduction in onboarding time. But the biggest win was unexpected: junior engineers started asking architecture questions they would never have asked before, because Cody made it safe to ask ‘dumb’ questions about the codebase.” — Engineering Manager, fintech company

Lesson 2: “We tried using Cody for autocomplete and were disappointed by the latency. Then we switched to using Cody only for chat and codebase questions, and kept Copilot for autocomplete. That combination is the best of both worlds.” — Senior Developer, e-commerce company

Lesson 3: “The Prompt Library is the killer feature nobody talks about. We created a ‘migration-guide’ prompt that encodes our team’s migration patterns. Now every developer runs it before starting a migration, and the consistency of our migrations has improved dramatically.” — Tech Lead, SaaS company

Advice for Getting Started

1. Start with Sourcegraph Cloud (free tier) to evaluate Cody without infrastructure investment. Connect your primary repository and spend a week using Cody for codebase questions.
2. Create 3-5 custom prompts in the Prompt Library for your team’s most common tasks: PR review, test generation, documentation, and security audit.
3. Measure the impact after two weeks: track time spent understanding unfamiliar code, test coverage, and developer satisfaction. Use these metrics to justify a Sourcegraph Enterprise deployment.
4. Self-host Sourcegraph Enterprise if your organization has data residency requirements or needs multi-repo context. The infrastructure cost is offset by the productivity gains.
5. Combine with other tools for the best experience: use Cody for codebase context and chat, Copilot for fast autocomplete, and a dedicated SAST tool for security scanning.

---

Next in the Open-Source AI Tools Mastery series: Open Interpreter