Table of Contents

Using MCP Servers via the MCP Client Tool

This guide walks you through configuring an agent to use the MCP Client Tool and testing it locally. By the end you will have an agent that connects to a remote MCP server (using the Microsoft Learn MCP server as an example) and returns results through the standard FoundationaLLM completions flow.

Overview

The Model Context Protocol (MCP) is an open standard that lets AI agents call external tools, read resources, and execute prompts hosted by MCP-compliant servers. The FoundationaLLM MCP Client Tool wraps the official Python MCP SDK so that any FoundationaLLM agent can interact with an MCP server without custom code.

Use this guide when you want to:

  • Connect an agent to an MCP server — for example, Microsoft Learn, a custom internal tool server, or any MCP-compliant endpoint.
  • Test MCP tool invocations locally — step through the full request flow in your IDE with breakpoints.
  • Understand the configuration surface — learn what goes into the agent definition, tool properties, and (optionally) intelligent_execute resources.

Prerequisites

  1. Local development environment configured per Configure local development environment.

  2. Environment variables set as described in Running FoundationaLLM Services Locally:

    • FOUNDATIONALLM_APP_CONFIGURATION_URI — your Azure App Configuration URI
    • FOUNDATIONALLM_VERSION — your deployment version

  3. Python virtual environment for LangChainAPI with local plugin sources and the mcp package installed:

    cd C:\Repos\foundationallm\src\python\LangChainAPI
.\.venv\Scripts\Activate.ps1
pip install -e ..\plugins\core\pkg
pip install -e ..\plugins\agent_core\pkg
pip install mcp==1.17.0
    Important

    The pip install -e (editable install) commands make the foundationallm and foundationallm_agent_plugins packages resolve to your local source code instead of the pre-built .whl files downloaded from Azure Blob Storage. Both packages must be installed in editable mode — the core package contains the PluginManager that loads agent plugins, and will prefer locally-installed packages over the .whl from blob storage. This ensures that any local changes to the plugin code (such as newly registered tool classes) are picked up immediately without rebuilding the wheel.

  4. Entra ID authentication — you need an Entra ID bearer token for agent management operations.

  5. An MCP server to target — this guide uses the publicly available Microsoft Learn MCP server (https://learn.microsoft.com/api/mcp). No API key is required.

Step 1: Start the Required Local Services

The following services are listed for convenience so you have the full local environment available:

Service Port Role
CoreAPI 63279 Entry point for completion requests
ManagementAPI 63267 Agent configuration and administration
OrchestrationAPI 7324 Loads agent definition, builds orchestration context
ContextAPI 6004 Context operations (skill management, tool sessions)
LangChainAPI 8765 Executes the agent workflow, instantiates and invokes the MCP client tool
UserPortal 3000 End-user chat interface
ManagementPortal 3001 Administrative configuration interface

Before starting the local services, configure URL exceptions so that your requests are routed to your local service instances instead of the deployed ones. Enable exceptions for the following services using your User Principal Name (UPN):

For step-by-step instructions, see Configuring URL Exceptions for Local Development.

Use the local services management script

The fastest way to start multiple services is with the management script. See Using the Local Services Management Script for full details.

cd C:\Repos\foundationallm\devtools\scripts\powershell

# Start all services
.\Manage-LocalServices.ps1 -Action Start -Services CoreAPI,OrchestrationAPI,LangChainAPI,ManagementPortal,UserPortal,ManagementAPI,ContextAPI

Verify they are running:

.\Manage-LocalServices.ps1 -Action Status

Option B: Start services manually

Follow the instructions in Running FoundationaLLM Services Locally to start each service individually.

Step 2: Configure an Agent with the MCP Client Tool

This section walks through adding the MCP Client Tool to an existing agent using the Management Portal.

Open the agent editor

  1. Open the Management Portal (default: http://localhost:3001 if running locally).
  2. In the sidebar, select Agents > Agents.
  3. Select the pencil icon for the agent you want to configure.

Add the MCP Client Tool

  1. Open the Tools tab and select Add Tool.
  2. In Add Tool to Agent, select an MCP tool from the library, or create one first on Agents > Tools:
    • Select Create Tool, choose MCP Tool, and select Next.
    • Under Connection, choose Inline URL.
    • Set MCP Server URL to https://learn.microsoft.com/api/mcp and Transport to Streamable HTTP. Leave HTTP Headers empty.
    • Under Tool Selection and Configuration, select List Tools, pick the Microsoft Learn search tool, and select Configure. Or check Enable Intelligent Execute and select a model.
    • Select Next, then Save to Library.
  3. Add that library tool to the agent and select Save Changes.

Register the server on Agents > MCP Servers and choose Registered endpoint when several tools share the same URL or credentials. See MCP Servers and Tools.

Important

After saving, the agent will use the MCP Client Tool on the next completion request. There is no restart required — the LangChainAPI picks up the updated agent definition dynamically. If you have an existing conversation going with that agent, simply start a new conversation and the tool will be picked up automatically.

Tool properties explained

Property Value Purpose
transport streamable_http Protocol used to connect. Also supports http_sse. (stdio is supported via API only — not configurable in the Management Portal.)
streamable_http.url https://learn.microsoft.com/api/mcp The MCP server endpoint.
streamable_http.headers {} HTTP headers sent with every request. Use config:// prefixed values for secrets stored in Azure App Configuration or Key Vault.
streamable_http.timeout_seconds 30 HTTP connection timeout in seconds.
streamable_http.sse_read_timeout_seconds 300 Read timeout for streaming responses.
Operation timeout 30 How long a single MCP call can run. Set Operation Timeout (seconds) on the tool.
Note

Header values and STDIO environment variables that begin with config:// are resolved through Azure App Configuration / Key Vault at runtime. Values beginning with env:// resolve to environment variables. This keeps secrets out of agent definitions.

Step 3: Test the Agent via the User Portal

With the agent configured and services running, submit a request through the User Portal to verify the MCP Client Tool is working end-to-end.

  1. Open the User Portal (default: http://localhost:3000 if running locally).

  2. Sign in with your Entra ID credentials.

  3. Click New Session (or +) to start a new conversation.

  4. In the Agent selector at the top of the chat, choose the agent you configured in Step 2.

  5. In the message input, type a prompt that will trigger the MCP tool — for example:

    Search Microsoft Learn for how to use MCP with Azure

  6. Press Enter (or click Send) to submit the message.

  7. Wait for the agent to respond. Behind the scenes, the agent's LLM determines that it should invoke the MCP Client Tool, which connects to the Microsoft Learn MCP server, executes the search, and returns the results.

  8. The response should contain information sourced from Microsoft Learn documentation.

Tip

If you are running LangChainAPI locally with breakpoints set (see Debugging Tips), the debugger will pause at those breakpoints as the MCP tool executes. This is a good way to inspect the MCP session, the operation arguments, and the response payload.

What happens under the hood

The request flows through the following chain:

CoreAPI  →  OrchestrationAPI  →  LangChainAPI
                                      ↓
                                 GenericAgent
                                      ↓
                                 ToolFactory → FoundationaLLMAgentToolPluginManager
                                      ↓
                                 FoundationaLLMMCPClientTool._arun()
                                      ↓
                                 MCP Server (learn.microsoft.com)
  1. CoreAPI validates the request, checks quotas, and forwards to OrchestrationAPI.
  2. OrchestrationAPI loads the agent definition from Cosmos DB (including the MCP tool configuration), assembles resources, and forwards to LangChainAPI.
  3. LangChainAPI creates a GenericAgent, which uses ToolFactory to instantiate the FoundationaLLMMCPClientTool via the plugin manager.
  4. The agent's LLM decides to invoke the MCP tool based on the user prompt.
  5. The MCP client tool opens a session to the configured MCP server, executes the operation, and returns the result.

Step 4: Use Intelligent Execute (Optional)

For more sophisticated scenarios, the MCP client tool supports an intelligent_execute operation that uses an LLM to automatically discover MCP tools provided by the MCP server, plan which ones to call, execute them, and synthesize the results.

Configuration in the tool editor

  1. Open the MCP tool from Agents > Tools, or from the agent editor's Tools tab.
  2. Check Enable Intelligent Execute.
  3. Under Intelligent Execute Resources, select the main model. The main and router prompts default to Platform-Agent-Tool-MCP-Main and Platform-Agent-Tool-MCP-Router.
  4. Select Save, then Save Changes on the agent if you edited it there.

Prompt content is edited on Agents > Prompts. See the MCP Orchestration Prompt reference and the Sample Routing Prompt for MS Learn MCP Tool.

How it works

Instead of manually calling list_tools and call_tool, you send a single natural language prompt:

{
  "operation": "intelligent_execute",
  "arguments": {
    "prompt": "Find Azure Functions deployment documentation"
  }
}

The tool then:

  1. Discovers available MCP tools via list_tools
  2. Plans an execution strategy using the configured LLM
  3. Executes the planned MCP operations
  4. Synthesizes the results into a coherent response

Token usage across the planning and synthesis phases is tracked and reported in the FoundationaLLMToolResult.

Debugging Tips

Set breakpoints in the MCP client tool

If you are running LangChainAPI locally, you can set breakpoints in:

  • src/python/plugins/agent_core/pkg/foundationallm_agent_plugins/tools/foundationallm_mcp_client_tool.py — the main tool implementation
  • _arun() — the async execution entrypoint
  • _open_session() — MCP session lifecycle
  • _intelligent_execute() — the LLM-powered orchestration flow

Check service status

cd C:\Repos\foundationallm\devtools\scripts\powershell
.\Manage-LocalServices.ps1 -Action Status

View LangChainAPI logs

If running with --reload, uvicorn outputs logs to the terminal. Look for:

  • mcp.operation — which MCP operation was invoked
  • mcp.transport — which transport was used
  • Exception traces if the MCP server is unreachable

Common issues

Problem Cause Fix
ToolException: Invalid MCP client tool configuration Malformed properties in the agent tool definition Verify the transport-specific config block matches the selected transport value
MCP client invocation failed: Connection refused MCP server endpoint is unreachable Check the URL is correct and accessible from your machine
Unknown tool class: FoundationaLLMMCPClientTool The pre-built .whl from blob storage does not include the MCP client tool class Install the agent_core plugin from local source with pip install -e ..\plugins\agent_core\pkg in the LangChainAPI venv (see Prerequisites step 3)
Requests not reaching local LangChainAPI URL exception not configured or not enabled Follow Configuring URL Exceptions and verify with the Management Portal
ToolException: LLM not configured when using intelligent_execute Missing resource_object_ids for main_model Add AI model and prompt resources to the tool's resource_object_ids in the agent definition

Using Other MCP Servers

The Microsoft Learn server is just one example. To connect to a different MCP server, adjust the tool properties:

Remote HTTP server (with authentication)

{
  "transport": "streamable_http",
  "streamable_http": {
    "url": "https://my-internal-server.example.com/mcp",
    "headers": {
      "Authorization": "config://MCP-MyServer-ApiKey"
    },
    "timeout_seconds": 30,
    "sse_read_timeout_seconds": 300,
    "terminate_on_close": true
  }
}

HTTP + SSE server

{
  "transport": "http_sse",
  "http_sse": {
    "url": "https://sse-server.example.com/mcp",
    "headers": {},
    "timeout_seconds": 15,
    "sse_read_timeout_seconds": 120
  }
}