Tool sets in VS Code: Stop agent hallucinations

Visual Studio Code agents operate on a strict triad: built-in tools, Model Context Protocol (MCP) integrations, and extension utilities. The difference between a hallucinating bot and a productive engineer often comes down to how you curate this list. Agents do not guess; they select from what you enable. If you leave the floodgates open, the model will dredge up irrelevant capabilities or, worse, attempt dangerous operations on sensitive endpoints.

Security and accuracy start before the first token is generated. You must explicitly enable functions within the Chat view, filtering availability via the search box or by installing specific MCP servers. This isn't just housekeeping; it's a hard boundary against capability hallucination.

Beyond simple on/off switches, the real power lies in tool sets. These allow you to group utilities into single, referenceable entities, letting you override input parameters via the confirmation dialog before a single command hits the shell. This control layer ensures that while the agent handles the heavy lifting of searching code or fetching web content, you retain final authority. Modern development demands this balance: automation without oversight is just a faster way to break production.

The Role of Tool Sets in Agent Autonomy

Defining Tool Sets, Built-in Tools, and MCP Tools

Stop enumerating individual functions. A tool set groups multiple utilities into a single referenceable entity, simplifying prompt execution across chat prompts, prompt files, and custom chat agents. Instead of listing ten separate commands, you invoke one named set. This abstraction cuts token overhead and standardizes agent behavior, ensuring your custom chat agents act consistently every time.

Availability hinges on source distinctions. Built-in tools like read, write, and bash come pre-enabled, handling basic filesystem or shell operations without external dependencies. MCP tools extend this baseline by connecting to installed Model Context Protocol servers, bridging the gap to external data sources or specialized APIs. While built-ins handle local code manipulation immediately, MCP tools expand the agent's reach to external systems through installed servers.

Automatic tool selection is a gamble when precision matters. Relying on it risks bloating your context window with irrelevant options. Explicitly referencing a tool set in a prompt forces the agent to prioritize specific capabilities, mitigating the risk of the model selecting a suboptimal tool from the global pool. Precision requires upfront investment: define these sets via .jsonc configuration files. Selecting only the tools necessary for a prompt improves results, balancing the flexibility of a large repertoire against the predictability of curated sets.

Using # References and Configuring Tool Sets

Typing # followed by a specific name in the chat input triggers explicit tool invocation. This syntax bypasses the agent's autonomous selection logic, forcing the application of an assigned capability. A dropdown list appears containing built-in utilities, MCP tools from external servers, and defined tool sets. Direct references eliminate ambiguity when multiple tools share similar functional descriptions or when precise execution order matters.

To create custom collections, run the Chat: Configure Tool Sets command within the Command Palette. Selecting Create new tool sets file generates a .jsonc configuration where you map logical names to arrays of underlying functions. Grouped entities simplify prompt engineering by allowing a single token, the set name, to enable complex context-gathering workflows instantly. This abstraction reduces the cognitive load required to manage large inventories of available actions during extended sessions.

Control doesn't end at invocation. Granular control over tool permissions is necessary for managing how the agent requests approval and operates with autonomy. Autonomous loops risk executing destructive filesystem changes or exposing environment variables without strict confirmation policies. Increased interaction frequency is the cost of this safety layer, requiring builders to balance autonomy levels against operational risk tolerance carefully.

Inside the Agent Tool Selection and Execution Loop

Agents pull from the enabled tools set, picking utilities that match the prompt context and request scope. This autonomous selection weighs built-in tools, MCP integrations, and extension tools to handle jobs like running terminal commands or searching codebases. Users override that automatic choice by referencing a tool with the # symbol followed by the name, forcing the agent to apply specific functionality such as web fetching for external queries or code search for internal analysis. Explicit referencing narrows the search space so the agent focuses computational resources on the intended operation rather than guessing from the full tool set.

Configuring and Executing Tools for Development Workflows

Defining Tool Set Structure in JSONC Configuration Files

Map specific built-in tools, MCP utilities, and extension capabilities into named groups by constructing a .jsonc file. Generate the initial file structure by executing the Chat: Configure Tool Sets command or accessing the ellipsis menu in the Chat view. The resulting configuration object requires a tools array listing exact identifiers, a description string for the picker interface, and an icon reference from the Product Icon library. This schema enables the tool set to function as a single selectable entity when prefixed with # in prompts. Organizing utilities this way reduces prompt token consumption by avoiding repetitive tool enumeration during complex queries. Grouping too many heavy utilities into one set can obscure individual tool failures during debugging sessions. Developers must balance granular control against the convenience of bulk enabling related functions. The structure supports custom tool creation via helper functions, extending beyond standard built-in tools like grep or bash. Precise naming in the tools array is mandatory; a single typo prevents the agent from locating the underlying executable.

Executing Terminal Commands via Integrated VS Code Terminals

Invoke the terminal tool to execute dependency installs and build scripts within the integrated shell environment. Developers initiate this workflow by enabling specific utilities in the tools picker before submitting a request to the chat interface. When the model determines a command is necessary, it renders the proposed script directly in the conversation thread for review. Users inspect output inline by selecting Show Output or switch to the full integrated terminal view via the Show Terminal action. Tool call details stay collapsed by default: select the tool summary line in chat to expand them, or change the default with the chat.agent.thinking.collapsedTools setting (experimental). This dual-view architecture supports both quick verification and deep debugging without leaving the editor context.

1. Select the Configure Tools button in the chat input field.
2. Toggle the terminal switch to enable command execution capabilities.
3. Submit a prompt requiring environment changes, such as installing packages.
4. Review the generated command and select Allow to proceed.
5. Click Continue in Background if the process requires extended runtime.

Run long-running processes like development servers asynchronously, freeing the agent to perform other diagnostic tasks while monitoring status. Background execution prevents blocking the conversational loop during watch-mode builds. Pushing commands to the background introduces state management complexity, as the agent must later retrieve output from a detached session context. OpenCode, a terminal-first alternative, provides built-in bash capabilities, yet the VS Code approach tightly couples execution visibility with chat history. The constraint favors immediate context retention over the raw process isolation found in standalone terminal-based ai tools.

Validating Background Execution Times and Output Locations

Stop waiting for terminal output once a specified timeout expires, returning only the data collected until that interruption. Operators must explicitly configure this limit within the tool parameters to prevent indefinite hangs during long-running builds or server startups.

1. Open the tool confirmation dialog and expand the input details using the chevron control.
2. Modify the timeout value to match the expected duration of the background task.
3. Verify the chat.tools.terminal.outputLocation setting directs logs to the integrated terminal for continuous monitoring.

Edit these parameters before execution to ensure the permission every step of the way philosophy remains intact while accommodating asynchronous workflows. A common failure mode involves agents attempting to read partial files before the background process flushes buffers. Aggressive timeouts produce incomplete build artifacts that force manual re-execution. Proper validation requires testing against actual compilation durations rather than estimated values.

Optimizing Tool Usage and Resolving Common Configuration Errors

The 128-Tool Limit, Virtual Thresholds and Recovery via the Tools Picker

A single chat request supports a maximum of 128 enabled tools before the system rejects the configuration. When an agent exceeds this count, users must manually deselect specific MCP servers or individual utilities via the tools picker to restore functionality. This hard constraint prevents context window overflow but creates friction for developers relying on extensive plugin ecosystems. To address this scaling issue, the github.copilot.chat.virtualTools.threshold setting allows automatic management of large tool sets without manual intervention. Enabling every available utility is rarely optimal because excessive tool definitions degrade agent reasoning speed and increase token consumption. Automatic thresholding may hide specialized tools the agent needs for niche tasks, requiring occasional manual verification. Operators should group related functions into logical sets rather than enabling all extensions globally. This approach balances the breadth of available actions with the strict input limits of the underlying model.

Recovery from the "Cannot have more than 128 tools per request" error runs through the tools picker. Operators must open the Chat view, select the Configure Tools button, and deselect non-necessary utilities or entire server groups. Typing # in the input field reveals the full inventory, allowing precise identification of active components before removal. Virtual thresholds solve the count error automatically, yet they shift the bottleneck from memory limits to runtime resolution speed and add latency to the first tool invocation. Developers extending functionality through custom tool creation must account for this ceiling during architecture planning. AI Agents News recommends auditing tool sets quarterly to align active definitions with current workflow requirements rather than retaining legacy integrations.

Shell Integration Failures with Default cmd and sh Configurations

Shell integration fails when agents default to cmd on Windows or sh on macOS, severing the feedback loop required for reliable automation. Because these legacy shells lack modern escape sequence support, the agent cannot distinguish command output from the prompt, forcing reliance on idle timeouts rather than explicit completion signals. This architectural gap creates a "slow and flaky experience" where execution timing becomes unpredictable. Operators configuring profiles via chat.tools.terminal.terminalProfile.windows or equivalent Linux settings must recognize that overriding defaults to use PowerShell or zsh is mandatory for stable workflows. Some terminal-first environments like OpenCode allow explicit shell configuration to bypass host constraints, but the native VS Code agent remains bound by the host terminal's capabilities. The cost of retaining sh is measurable: agents miss error codes hidden in unstructured text streams, leading to cascading logic failures in multi-step tasks. Builders must treat shell selection as a hard dependency, not a preference, to prevent silent execution drift. AI Agents News recommends validating terminal profiles before deploying complex tool sets.

References

• GitHub Copilot Teams new to AI-assisted development Inline editing
• AI Lab cli #52 58.0% Subscription ✗ ✗ $17-200/mo