This project used to be called Finny. It's called Scout now, and since this sub is basically where it started, I wanted to write up what actually changed in the fork. Most of it wasn't what I expected going in.

The problem with a stock coding agent doing trading research is that it's very good at producing something that looks right. Ask for a momentum strategy and you get clean Python and a backtest with a great Sharpe. When I dug into those results, the number almost always came from one of three places: lookahead (deciding on a bar using that same bar's close), picking the best run out of a parameter sweep, or the model already knowing how that ticker moved. Nothing in a normal coding loop tells the agent any of that is wrong. The code runs, so as far as the agent is concerned it's done.

So most of my changes are about getting in the agent's way.

1. A fixed strategy contract. Every strategy is a class with an on_bar method. On the current bar it should only decide using bar["open"], and orders fill on the next bar. To be clear, that's a contract the agent writes against, not something the engine enforces. Still, giving the model one simple rule to write against was the cheapest fix I found, and it makes lookahead much easier to spot when it does show up.

2. A validation gate before anything runs. It checks syntax and class shape, and blocks os, subprocess, socket, requests, shutil and exec/eval inside strategies. There's no override flag. I went back and forth on that and decided an override would just get used every time.

3. A skeptical default. After the base backtest it runs walk-forward, Monte Carlo and lookahead checks, and the default verdict is research_only. Most ideas stay there, which honestly is the most useful thing it does.

The part that's probably most relevant here: I ran my own small-sample benchmark, ScoutBench, comparing Scout with stock OpenCode, Claude Code and Codex using GPT-5.5 in each harness. Across 5 tasks, 20 checks per task and 3 trials, the mean scores were Scout 53.3, Claude Code 46.3, Codex 44.1 and OpenCode 38.6. These are benchmark scores, not trading results.

Most of that gap comes from two tasks: repairing a lookahead bug (70 vs 51.7 for Claude Code, 45 Codex, 35 OpenCode) and an end-to-end BTC strategy build (81.7 vs 55). So the domain changes help where they're supposed to and not much anywhere else. Treat the results as directional: the sample is small, Scout lost on numeric discipline (30 vs 45), and no harness scored above 28.3 on hallucination resistance, mine included. OpenCode scoring lowest mostly says the tasks are narrow and it's general-purpose; it was a really good base to build on.

The full method, task breakdown and results are in my benchmark write-up: https://www.polarline.io/blog/finnybench-agent-benchmark

Disclosure: I built this, I'm the founder of Polarline. It's source-available (not open source). The CLI command is still finny from before the rename.

Has anyone else bent OpenCode toward a domain where "the code runs" isn't the same as "the answer is right"? Curious how you handled verification.

I made a plugin that adds Claude Code's /goal command to OpenCode v2. You give the agent a finish line like "/goal npm test passes with 0 failures" and it keeps working turn after turn until the goal is actually met, no need to keep typing "continue".

After every turn an evaluator checks the conversation and decides if the goal is met, not met or impossible. If it's not met, the agent gets the reason and keeps going on its own.

It's token efficient: the evaluator reuses the session's cached context, so each check costs little more than a short answer.

It also has safety rails (pauses when the agent stops making progress, retries on network errors, turn limit) and a status indicator in the TUI.

Tell me what you think about it and feedback is appreciated.

Here's the repo and npm package:

https://github.com/EL4CTEO/opencode-goal

https://www.npmjs.com/package/@el4cteo/opencode-goal

https://research.meta.ai/blog/muse-spark-open

Point of the question is that. Some models are cheaper on paper but somehow can rack up costs quickly. GLM-5.3 is my main complaint in this regard. Its on paper cheap. But it generates so much thinking tokens that it kind of nullifies the cost.

I know ppl use cost per task. But I am not running benches. I'm simply collecting metrics and comparing. And since I use different models for different tasks. I dont think I can correctly say x model uses n times more tokens unless I run multiple models against the same task regularly.

Hey r/opencodeCLI! 👋

With OpenCode v2 rolling out, the way plugins interact with the core runtime has taken a massive leap forward. Today, we’re excited to announce opencode-models-discovery v2.0.0-rc.2 — completely rebuilt to bring full, native support for OpenCode v2 while making dynamic model discovery faster, safer, and truly reactive.

If you use local models or custom gateways (vLLM, LiteLLM, Ollama, LM Studio, Bifrost, etc.), we’d love for you to take rc.2 for a spin!

🚀 What Changes in the V1 ➔ V2 Evolution?

In OpenCode v1, model discovery relied on disk-based caches, file locks, and static configuration updates that often required app restarts.

In v2.0.0, we’ve redesigned the plugin around OpenCode v2’s native architecture:

• ⚡ Native Reactive Catalog (Zero Restarts)
• Models from your OpenAI-compatible endpoints are discovered and transformed directly through OpenCode v2’s dynamic provider controller. No config rewrites, no restarting the CLI or TUI.
• 🔒 Native ctx.storage Engine (No More Disk-Cache Baggage)
• Gone are the legacy XDG disk-cache files and file-lock contention. Caching now uses OpenCode v2’s isolated storage API. Discovered caches and enrichments are managed cleanly without leaking credentials or API keys to disk.
• 🎛️ Live Model Overrides in Console
• Fine-tune model capabilities, context windows, and reasoning tiers right from the OpenCode model console. Custom tweaks are persisted under clean, scoped override keys instead of manual config edits.
• 🔑 Seamless /connect Integration
• First-class credential resolution: works out-of-the-box with OpenCode v2’s managed connections, integration IDs, and environment variables without needing duplicate API keys in your config.
• 🛡️ Hardened & Standardized Token Limits
• Whether you're pulling metadata from models.dev, LiteLLM, LM Studio, or raw OpenAI /v1/models responses, context and output token limits are normalized to match OpenCode v2’s token budgeting specifications.

📦 Quick Start with OpenCode v2

You can try 2.0.0-rc.2 today by adding it to your opencode.json:

{
  "$schema": "https://opencode.ai/config.json",
  "plugins": [
    "npm:opencode-models-discovery@2.0.0-rc.2"
  ],
  "providers": {
    "local-gateway": {
      "name": "Local Gateway",
      "package": "@opencode-ai/ai/providers/openai-compatible",
      "settings": {
        "baseURL": "http://127.0.0.1:8000/v1",
        "modelsDiscovery": {
          "enabled": true,
          "smartModelName": true,
          "modelInfoFormat": "models.dev"
        }
      }
    }
  }
}

💬 Help Us Test Before Final v2.0.0 GA!

We are putting the final touches on v2.0.0 and want to ensure rock-solid compatibility across all OpenCode v2 setups, environments, and local model servers.

• GitHub Repository: https://github.com/yuhp/opencode-models-discovery
• Migration Guide: Check out our Migrate V1 to V2 Guide for details on the transition.

Give v2.0.0-rc.2 a spin and let us know your thoughts, feedback, or any issues you encounter in the comments below!