Sandbox Deployment

This guide covers the steps required to run FoundationaLLM secure sandboxes in an instance, for both:

Platform use — User Portal agents create a sandbox for a conversation.
External MCP use — a client such as Cursor talks to the shared sandbox MCP over streamable HTTP, with no FoundationaLLM user identity.

Management API owns sandbox authorization, configuration, and lifecycle. The MCP adapter is not a second control plane. It calls Management API. Context API and the AKS backend create and run the Kata pod.

The product contract is documented in Secure Sandbox Contract v1.

If the instance already has a v1 sandbox (Kata pods and older guest images) and you are rolling current Management / Context, use Upgrade a v1-only sandbox environment for images, App Config, cluster wiring, and class/policy republish.

Two caller paths

Path	Who Management API authorizes	How the Sandbox Configuration is chosen
Platform	The signed-in user (forwarded as `X-USER-IDENTITY`)	The agent's administrator-selected configuration ID
External MCP	The MCP server managed identity	Trusted header `X-FLLM-SANDBOX-CONFIGURATION-ID`

Do not mix the paths. If the MCP server sends X-USER-IDENTITY or has a fallback user identity configured, Management API treats the call as platform-mediated and ignores the configuration header.

Prerequisites

Complete these before assigning roles or connecting a client.

Platform build

Management API, Authorization API, and Context API must come from a build that:

Knows sandbox resources (sandboxes, sandboxConfigurations, sandboxClasses, sandboxPolicies) and their authorizable actions.
Accepts a managed-identity token that has an object ID but no UPN. Authorization keys access by object ID. If the APIs reject an empty UPN, create fails with HTTP 403 and a blank user name before roles are checked.
Lets Context API create a sandbox without an agent name (environment_type = sandbox).

Older builds (for example 1.0.0-rc108) cannot enable the external MCP path even when every role below is assigned.

Azure and cluster

Requirement	Purpose
AKS cluster with Kata (`kata-vm-isolation`)	Isolated guest VM per sandbox
At least one sandbox node pool	Where pods schedule. A configuration can pin a pool
Context API identity with cluster access	The only identity that execs into sandbox pods
Published container image for the sandbox class	Guest image the class revision references

Node-pool SKU and Ultra Disk capability must match the storage mode on the Sandbox Policy (ephemeral, standard SMB, shared SSD, dedicated SSD, or dedicated disk). Do not enable Ultra on a general-purpose pool unless a configuration is meant to use it.

Shared MCP endpoint (external path only)

Deploy the Management API–backed MCP adapter (not the older in-cluster Kubernetes MCP). It must:

Authenticate to Management API with its workload / managed identity.
Have no Kubernetes credentials and no local sandbox database.
Have no FLLM_AGENT_NAME.
Have no SANDBOX_USER_IDENTITY_UPN or SANDBOX_USER_IDENTITY_USER_ID (those impersonate a person and discard the configuration header).

Deployment details for the adapter live in the foundationallm-aks-pods-sandbox repository (docs/plans/aks-deployment-plan.md and the repository README).

Shared setup in the Management Portal

These resources are used by both paths. Publish them before granting access.

1. Publish a Sandbox Class

Sandboxes → Classes. A class is an immutable revision: image, CPU, memory, GPU, runtime class (kata-vm-isolation), and workspace/tmp size.

2. Publish a Sandbox Policy

Sandboxes → Policies. A policy is an immutable revision: persistence mode, I/O, seccomp/inner filter, package installs, and network exceptions.

3. Publish a Sandbox Configuration

Sandboxes → Configurations. A configuration is the administrator-controlled combination of one class revision and one policy revision. Optionally pin a node pool. Each save creates a new published revision (ETag-protected). New sandboxes use the class and policy revision that was current at create time. Idle timeout and maximum lifetime are editable in place and apply to running sandboxes after resource-provider caches refresh.

Create one configuration per intended audience. Example: secure-sandbox-default for ordinary use.

4. Confirm a node pool can schedule

Sandboxes → Sessions (capacity) or Sandboxes → Node pools. The pool named on the configuration (or the instance default if the configuration leaves it empty) must have capacity. Scale idle Ultra or dedicated-disk pools to zero when they are not in use.

Enable sandboxes for the platform (agents and User Portal)

Platform callers are people (or groups). They already have a UPN. The default instance policy assignment for “own resources” is usually on a security group, so users inherit ownership. Managed identities do not inherit that group policy.

1. Grant people the sandbox role

Assign Sandboxes Contributor to the users or groups who may create and use their own sandboxes.

Item	Value
Role	Sandboxes Contributor
Role definition ID	`a8f3e2d1-4c5b-6a7e-8f9d-0e1f2a3b4c5d`
Principal type	User or Group
Scope	`/instances/<instance-id>`

Actions granted: sandboxes/read, sandboxes/write, sandboxes/delete.

Administrators who author classes, policies, and configurations also need Sandboxes Admin (b9e4f3a2-5d6c-7b8e-9f0a-1b2c3d4e5f6a) at instance scope.

See Role Assignments.

2. Confirm the UPN ownership policy for people

Sandbox create requires the built-in User Principal Name (UPN) match policy on the sandboxes resource type, not only a role.

Item	Value
Policy	User Principal Name (UPN) match
Policy definition ID	`00000000-0000-0000-0001-000000000001`
Typical principal	The instance users security group
Scope	`/instances/<instance-id>/providers/FoundationaLLM.Platform/sandboxes`

If this default assignment already exists for your users group, do not duplicate it.

3. Bind an agent to a configuration

In the Management Portal, open the agent and set Sandbox Configuration to a published configuration (for example secure-sandbox-default). Do not put class, policy, image, or network fields on the model-visible tool schema.

The agent must use the FoundationaLLM MCP client against the sandbox MCP (or the equivalent platform tool wiring). FoundationaLLM sends X-FLLM-SESSION-ID (conversation id) and X-USER-IDENTITY on every call.

4. Smoke-test in the User Portal

Sign in as a user who has Sandboxes Contributor.
Start a conversation with that agent.
Ask it to use the sandbox (create, then run a simple command).
Confirm Sandboxes → Sessions shows a Running Kata pod for that conversation.
Delete the conversation or the sandbox when finished.

How an admin enables sandbox via MCP directly

This is the external builder path: Cursor or another MCP client talks to the shared sandbox MCP over streamable HTTP. FoundationaLLM does not send a user identity. The MCP server’s managed identity is the caller that Management API authorizes.

Step 1 — Identify the MCP server identity

In Azure, open the managed identity used by the shared MCP.

Copy the Object ID. That is the principal you assign roles to.

Example (LemonDesert): id-akssandbox-mcp-server-lemondesert, object ID 4d5f8d68-5171-461b-885d-777488bcef93.

Step 2 — Let that identity call Management API (Entra)

The MCP authenticates to Management API as itself (client credentials / workload identity), not as the human using the MCP client.

Open the FoundationaLLM-Management-API app registration.
Confirm the MCP identity has an app-role grant such as MCP.Invoke or Data.Manage.
If it is missing, grant it and wait a few minutes for token caches to refresh.

This only lets the MCP reach the API. It does not let it create sandboxes.

Step 3 — Assign FoundationaLLM roles (PBAC)

Do this against the live Management API for that instance (Management Portal or API). Do not assign only through a local debug API. The cloud Authorization service keeps its own in-memory cache and will not see those grants until it is updated or restarted.

Assign to the MCP Object ID, principal type Service Principal.

A. Create and manage sandboxes

Item	Value
Role	Sandboxes Contributor
Role definition ID	`/providers/FoundationaLLM.Authorization/roleDefinitions/a8f3e2d1-4c5b-6a7e-8f9d-0e1f2a3b4c5d`
Scope	`/instances/<instance-id>`

This allows sandboxes/read, sandboxes/write, and sandboxes/delete.

B. Read the configuration the client is allowed to pick

Item	Value
Role	Reader
Role definition ID	`/providers/FoundationaLLM.Authorization/roleDefinitions/00a53e72-f66e-4c03-8f81-7e885fd2eb35`
Scope	`/instances/<instance-id>/providers/FoundationaLLM.Platform/sandboxConfigurations/<configuration-id>`

Example configuration id: secure-sandbox-default.

Do not grant read on every configuration if you want the client locked to one SKU and policy.

Sandboxes Admin also works because it includes configuration read, but it is broader than needed.

Least privilege that matches an external harness: Sandboxes Contributor on the instance, Reader on one configuration, and the UPN-match policy in the next step. Broad instance Contributor (*) is not required once Authorization API knows the sandbox actions.

Step 4 — Assign the ownership policies (required)

These are PBAC policies, not just roles. People usually get them through a security group. A managed identity is not in those groups, so they must be assigned directly.

A. Sandbox create

Without this, create returns HTTP 500 about missing policy assignments on sandboxes/__new__.

Item	Value
Policy	User Principal Name (UPN) match
Policy definition ID	`/providers/FoundationaLLM.Authorization/policyDefinitions/00000000-0000-0000-0001-000000000001`
Principal	The same MCP Object ID
Principal type	Service Principal
Scope	`/instances/<instance-id>/providers/FoundationaLLM.Platform/sandboxes`

B. Conversation ownership (required for download_file)

download_file persists a copy into the conversation file store. That load also requires the UPN-match policy on conversations, plus a role that can write conversations.

Item	Value
Policy	User Principal Name (UPN) match
Policy definition ID	`/providers/FoundationaLLM.Authorization/policyDefinitions/00000000-0000-0000-0001-000000000001`
Principal	The same MCP Object ID
Principal type	Service Principal
Scope	`/instances/<instance-id>/providers/FoundationaLLM.Conversation/conversations`

Item	Value
Role	Conversations Contributor
Role definition ID	`/providers/FoundationaLLM.Authorization/roleDefinitions/d0d21b90-5317-499a-9208-3a6cb71b84f9`
Scope	`/instances/<instance-id>`

Without the conversation policy, download returns HTTP 500 about missing PBAC on conversations/{session-id}. External MCP sessions are not User Portal chats; the platform then treats the agent name as external and still returns the file bytes.

Step 5 — Configure the MCP server

On the shared MCP deployment:

Point it at the live Management API base URL and instance ID.
Attach the managed identity from step 1.
Do not set SANDBOX_USER_IDENTITY_UPN or SANDBOX_USER_IDENTITY_USER_ID.
Do not set FLLM_AGENT_NAME.

Step 6 — Configure the MCP client

Use streamable HTTP (not stdio). Send these headers on every call:

Header	What to put
`X-FLLM-API-Key`	The MCP API key from Key Vault
`X-FLLM-SESSION-ID`	One id per sandbox session
`X-FLLM-SANDBOX-CONFIGURATION-ID`	The configuration from step 3B

Do not send X-USER-IDENTITY. That is the FoundationaLLM-mediated path.

The model-visible tools stay:

create_or_get_sandbox
get_sandbox_status
delete_sandbox
execute_command
execute_python
upload_file
download_file
list_files

Class, policy, image, runtime, capacity, network, and enforcement overrides are not client arguments.

Step 7 — Smoke-test the external path

Call with a bad API key → HTTP 401.
List tools → the eight tools above.
create_or_get_sandbox with the configuration header → Kata pod Running.
Call create again with the same session id → same pod, not a second one.
Run a command, then delete_sandbox.

Files sidecar (workspace download)

Large files leave /workspace through the fllm-files sidecar on port 8081, then Context and Platform download-file. They must not travel as execute_command stdout.

Check	Where
Sidecar image digest	App Config `FoundationaLLM:APIEndpoints:ContextAPI:Configuration:SandboxCodeSessionService:AKSSandboxDefaults` → `FilesSidecarImage`
Ready `fllm-files`	New sandbox pods (image is snapshotted at create — recreate after changing the digest)
Ingress to 8081	NetworkPolicy `allow-context-sandbox-files` in the sandbox namespace. Example: `docs/docs/development/sandbox-files-networkpolicy.yaml`
In-cluster Context	Deployed Context must resolve `*.svc.cluster.local`. Laptop/hybrid Context cannot reach the sidecar
Fallback node pool	`AKSSandboxDefaults.NodePoolName`. Configurations should still set `node_pool_name`

After changing the sidecar image or NetworkPolicy, restart Context and recreate sandboxes.

Troubleshooting

Symptom	Meaning
403, user name blank, `userId` = MCP object id	Platform build still rejects empty UPN, or roles were assigned only on a local API
403 on the configuration id	Missing Reader (or Admin) on that configuration, or Authorization API is too old to know `sandboxConfigurations/read`
500 about PBAC policy on `sandboxes/__new__`	Step 4 was skipped for the MCP identity
500 `Bad Request` from Platform or Context	Context API is an older build that still requires an agent name
Create works but `/etc` is writable	Policy on that configuration is not enforcing a read-only root. This is not an auth failure
Configuration header is ignored	The MCP sent `X-USER-IDENTITY` or has a fallback user identity set

Table of Contents