History of breaking changes

Note

This section is for changes that are not yet released but will affect future releases.

Starting from 1.0.1

Configuration changes

Realtime speech uses the Azure OpenAI GA Realtime API. The Core API connects to wss://{resource}.openai.azure.com/openai/v1/realtime?model={deployment}. The preview protocol (/openai/realtime?api-version={version}&deployment={deployment}) is no longer supported, and there is no backwards compatibility with it.

Resource	Required change
FoundationaLLM.Configuration/apiEndpointConfigurations used by realtime speech AI models	Remove the api_version value. The GA Realtime WebSocket does not accept api-version, and the Core API no longer appends it for realtime connections. If the endpoint is shared with models that still need api_version (for example chat completions through AzureOpenAIDirect), create a dedicated API endpoint configuration for realtime speech without an API version and point the realtime speech AI model at it. The url must be the resource URL only (for example https://{resource}.openai.azure.com); path segments such as /openai/realtime or /openai/v1 are stripped with a warning.
FoundationaLLM.AIModel/aiModels of type realtime-speech	deployment_name must be an Azure OpenAI deployment of a GA realtime model (for example gpt-realtime). Preview deployments such as gpt-4o-realtime-preview are not supported. deployment_name is now required; the Core API rejects the connection when it is empty. version is the model version (for example 2025-08-28), not an API version. input_audio_format and output_audio_format must be pcm16, g711_ulaw, or g711_alaw; any other value is rejected. input_audio_transcription_enabled is now honored: when false, no input transcription is requested, so user speech transcripts and transcript-based conversation naming are unavailable. User messages are still created when show_transcriptions is true (with the fixed text Voice input (transcription disabled).), and agent transcription is unaffected.

Starting from 1.0.0-rc03

Security changes

A new built-in role definition Security Principals Reader (6fbf9db9-8e0e-4337-be05-58a3d6f2eda3) has been added. It now holds the only FoundationaLLM.Authorization/securityPrincipals/read action. That action has been removed from Agents Contributor (3f28aa77-a854-4aa7-ae11-ffda238275c9).

The role grants the following authorizable action:

• FoundationaLLM.Authorization/securityPrincipals/read

Assign Security Principals Reader to the FLLM-Users group so existing users can resolve security principal display names. Without that assignment, Manage Agents shows owner ids as GUIDs for users who are not Agents Contributors.

Re-run Authorization API role-definition initialization (or otherwise refresh built-in role definitions) on existing deployments so Security Principals Reader is available and Agents Contributor no longer includes FoundationaLLM.Authorization/securityPrincipals/read.

Configuration changes

Tool escalation is off unless the agent workflow property enable_tool_escalation is set. Only FoundationaLLMFunctionCallingWorkflow reads it. A missing value, false, or no keeps the previous behavior: the workflow binds every agent tool. Boolean true and the strings 1, true, yes, and on turn it on. Set it at workflow.properties.enable_tool_escalation. In the Management Portal agent editor, use the Enable tool escalation checkbox on Basic Settings. Checked stores boolean true and unchecked stores boolean false, including when the saved value was missing or a truthy string. The Workflow properties list does not show this key. The basic agent template does not set the property.

When the property is on, the first pass uses the platform prompt Platform-Agent-Workflow-Escalation from the platform prompts package. Deploy that package before enabling escalation. Orchestration loads the prompt at /instances/{instanceId}/providers/FoundationaLLM.Prompt/prompts/Platform-Agent-Workflow-Escalation and sends the object id as Workflow.ToolEscalation.EscalationPromptObjectId. A workflow resource with object role escalation_prompt overrides that default. The basic agent template adds that role and still leaves escalation off. If the prompt is missing or has no prefix, the pass binds every tool.

The first pass does not run when the agent has no tools, the turn includes files, or the user command contains ROUTER.

FoundationaLLMFunctionCallingWorkflow includes the workflow router prompt only for bound tools whose resource name is listed in App Configuration. The resource name is the last segment of tool_object_id (for example Platform-Tool-Code), not the configurable tool display name. The check runs on every function-calling completion, including when tool escalation is off. After an escalation pass it is evaluated again against the tools that remain bound. When the setting is missing or empty, no tool requires the router prompt, so the router prompt is omitted.

Name	Default value	Description
FoundationaLLM:APIEndpoints:LangChainAPI:Configuration:WorkflowToolEscalation:ToolsRequiringRouterPrompt	(empty)	A comma-separated list of tool resource names. Orchestration copies it onto the LangChain completion request as Workflow.ToolEscalation.ToolsRequiringRouterPrompt.
FoundationaLLM:APIEndpoints:LangChainAPI:Configuration:WorkflowToolEscalation:StatusMessages	Deciding whether tools are needed...	A pipe-separated list of interim result messages shown while the escalation pass decides whether tools are needed. The workflow randomly selects one. Messages support the {current_round} and {max_rounds} placeholders. When the setting is missing, empty, or contains no non-empty entries, Deciding whether tools are needed... is used.

New deployments receive both keys from the App Configuration template. Existing stores can add them. Leave ToolsRequiringRouterPrompt empty to omit the router prompt. Leave StatusMessages unset to keep the default interim message. The operation status text stays Running agent workflow.

Starting from 1.0.0-rc200

Configuration changes

FoundationaLLM:APIEndpoints:CoreAPI:Configuration:BypassGatekeeper is no longer read. Completions always go to Orchestration. Prompt and response shielding run in Core. Current agents select plugins with shielding_settings. At this version, shielding was still selected by agent.gatekeeper_settings.options.

Previous behavior	Behavior starting in 1.0.0-rc200
BypassGatekeeper=true skipped the Gatekeeper API when the agent deferred to the system setting	The key is ignored. Shielding follows agent options.
Empty or missing gatekeeper_settings.options with use_system_setting=true used instance-wide Gatekeeper feature flags	Empty or missing options disable completion shielding
gatekeeper_settings.use_system_setting selected the system bypass vs agent options	use_system_setting is ignored. Keep it on agent JSON so older payloads still deserialize.

Existing App Configuration stores can keep FoundationaLLM:APIEndpoints:CoreAPI:Configuration:BypassGatekeeper. New deployments no longer create the key, and they do not deploy a Gatekeeper API. To disable completion shielding, omit shielding_settings or leave both plugin lists empty. To enable plugins, set shielding_settings.pre_completion_plugins and post_completion_plugins on each agent. Agents that still store only gatekeeper_settings.options keep working until they are saved with shielding_settings.

CoreServiceSettings.BypassGatekeeper has been removed. Extra keys in the FoundationaLLM:APIEndpoints:CoreAPI:Configuration section are ignored by options binding.

Create a new plugin named Dotnet-FoundationaLLMShieldingPlugins with runtime Dotnet.

API changes

ICompletionService no longer accepts CompletionOptions. FoundationaLLM.Common.Models.Orchestration.Request.CompletionOptions has been removed. Callers that passed BypassGatekeeper can drop the argument; shielding is not controlled from that type.

Starting from 1.0.0-rc131

Configuration changes

The following new App Configuration setting controls the Context API worker that deallocates sandboxes that exceed idle timeout or maximum lifetime (from the bound Sandbox Configuration):

Name	Content type	Default value	Description
FoundationaLLM:APIEndpoints:ContextAPI:Configuration:SandboxLifecycleReclaim	application/json	{"Enabled":true,"PollIntervalSeconds":30}	JSON object bound to SandboxLifecycleReclaimSettings. Enabled turns the worker on or off. PollIntervalSeconds is the scan interval (clamped at runtime to 5–3600 seconds). When the key is missing, the worker uses the same defaults.

Existing deployments must add this key to the App Configuration store. Context API already selects APIEndpoints:*, so no extra key filter is required after the value exists.

Example value:

{
  "Enabled": true,
  "PollIntervalSeconds": 30
}

FoundationaLLM:APIEndpoints:ContextAPI:Configuration:SandboxCodeSessionService:AKSSandboxDefaults (introduced in 1.0.0-rc102) is still required. The JSON schema is expanded. Merge new fields into the existing object; replacing the value drops cluster, gateway, and storage settings.

The Management API now loads this key (used to derive ACR login servers for the sandbox image catalog). Restart Management API after updating it.

New and previously undocumented properties:

Property	Required	Description
FilesSidecarImage	Yes	Image for the in-pod HTTP files sidecar that mounts /workspace. Creates fail when this is empty.
FilesSidecarPort	No (default 8081)	Cluster-internal port exposed by the files sidecar.
WorkspaceQuotaImage	Yes for ephemeral and shared-SSD	Image that enforces workspace_bytes with an in-pod FUSE quota.
ContainerRegistryLoginServer	No	Extra ACR login server for the Management Portal image catalog. The registry from ContainerImage is always queried as well.
Tolerations	No (default [])	Kubernetes taints tolerated by sandbox pods. Each entry uses key, operator (Equal or Exists), optional value, and effect (NoSchedule, PreferNoSchedule, or NoExecute).
WorkspaceStorageClassName	For standard-SMB persistence	Storage class for conversation-scoped Standard Azure Files workspaces.
WorkspaceSsdStorageClassName	For dedicated Premium Files SSD	Storage class for dedicated Premium Azure Files SSD workspaces.
WorkspaceSharedSsdStorageClassName	For shared Premium Files SSD	Storage class for shared Premium Azure Files SSD folders.
WorkspaceDiskStorageClassName	For dedicated Premium SSD v1	Storage class for dedicated Premium SSD (v1) managed disks (non-zonal pools).
WorkspacePremiumV2DiskStorageClassName	For dedicated Premium SSD v2	Storage class for dedicated Premium SSD v2 managed disks (zonal node pools).
WorkspaceUltraDiskStorageClassName	For dedicated Ultra Disk	Storage class for dedicated Ultra managed disks.
WorkspaceServiceAccountName	For Azure Files mounts	Pre-provisioned service account federated to the Azure Files mount identity. Do not automount the token on the user container.

See Upgrade a v1-only sandbox environment for merge steps, image digests, and restart requirements.

A CoreAPI APIEndpointConfiguration resource is now required. Create it in the Management Portal under API Endpoints (or as CoreAPI.json in the FoundationaLLM.Configuration resource provider folder) if it does not already exist. Set url to the public Core API URL — the same browser-facing value as FoundationaLLM:APIEndpoints:CoreAPI:Essentials:APIUrl. Do not use the in-cluster service address. Core API substitutes this URL into {{fllm_base_url}} deep links in conversation content (file and image URLs). In local development, IHttpClientFactoryService already honors FOUNDATIONALLM_URL_OVERRIDE_COREAPI through URL exception resolution on that endpoint.

Example resource:

{
  "type": "api-endpoint",
  "name": "CoreAPI",
  "object_id": "/instances/<instance_id>/providers/FoundationaLLM.Configuration/apiEndpointConfigurations/CoreAPI",
  "display_name": null,
  "description": null,
  "cost_center": null,
  "category": "General",
  "subcategory": null,
  "authentication_type": "AzureIdentity",
  "url": "<public_core_api_url>",
  "status_endpoint": "/instances/<instance_id>/status",
  "url_exceptions": [],
  "authentication_parameters": {},
  "timeout_seconds": 2400,
  "retry_strategy_name": "ExponentialBackoff",
  "provider": "foundationallm",
  "api_version": null,
  "operation_type": null
}

If the resource is stored as a file, add a matching entry to _resource-references.json in the FoundationaLLM.Configuration resource provider folder.

API changes

FoundationaLLM.Platform/managedEndpointDeployments no longer accepts execute-command. Command execution must go through FoundationaLLM.Platform/sandboxes so the server can bind an immutable assignment and enforcement bundle. restart and set-properties remain supported on managed endpoint deployments.

Security changes

The following authorizable actions have been added on FoundationaLLM.Platform. They are not included in Sandboxes Contributor. Operators who manage sandbox infrastructure need Sandboxes Admin (or equivalent custom assignments).

Action	Description
FoundationaLLM.Platform/sandboxNodePools/read	Read sandbox node pools and their capacity.
FoundationaLLM.Platform/sandboxNodePools/write	Create or update sandbox node pools and their settings.
FoundationaLLM.Platform/sandboxClasses/read	Read sandbox classes.
FoundationaLLM.Platform/sandboxClasses/write	Create sandbox class revisions.
FoundationaLLM.Platform/sandboxClasses/delete	Delete unreferenced sandbox class revisions.
FoundationaLLM.Platform/sandboxPolicies/read	Read sandbox policies.
FoundationaLLM.Platform/sandboxPolicies/write	Create sandbox policy revisions.
FoundationaLLM.Platform/sandboxPolicies/delete	Delete unreferenced sandbox policy revisions.
FoundationaLLM.Platform/sandboxConfigurations/read	Read sandbox configurations.
FoundationaLLM.Platform/sandboxConfigurations/write	Create or update sandbox configurations.
FoundationaLLM.Platform/sandboxConfigurations/delete	Delete sandbox configurations.
FoundationaLLM.Platform/sandboxContainerImages/read	List sandbox container images from the configured Azure Container Registry.

The built-in Sandboxes Admin role (b9e4f3a2-5d6c-7b8e-9f0a-1b2c3d4e5f6a) now includes the actions above in addition to sandboxes/read, sandboxes/write, and sandboxes/delete. Sandboxes Contributor remains limited to the three sandboxes/* actions (UPN-owned sessions).

Listing or reading /providers/FoundationaLLM.Platform/sandboxClasses (Management Portal Sandboxes → Classes) requires FoundationaLLM.Platform/sandboxClasses/read. Assign Sandboxes Admin at instance scope. Sandboxes Contributor is not sufficient and returns an authorization error (The read access to the resource path .../sandboxClasses is not authorized). Reader, Contributor, Owner, and User Access Administrator also include this action.

The FoundationaLLM.Platform resource provider requires PBAC policy assignments to load /instances/<instance_id>/providers/FoundationaLLM.Platform/sandboxNodePools. Without them, listing node pools fails with The FoundationaLLM.Platform resource provider requires PBAC policy assignments to load the .../sandboxNodePools resource path. Add policy assignments for sandboxNodePools at instance scope.

The Context API managed identity must also have Azure control-plane access to agent pools on each AKS cluster in AKSSandboxDefaults.AKSClusterResourceId. The rc102 roles (Azure Kubernetes Service Cluster User Role and Azure Kubernetes Service RBAC Cluster Admin) remain required for kubeconfig and Kubernetes data-plane access. They do not include Microsoft.ContainerService/managedClusters/agentPools/read or write. Without the roles below, Management Portal Sandboxes → Node pools shows No node pools found on the associated cluster (the list returns []) and get/create/update fail with an ARM authorization error.

Azure role	Required for
Reader	List and get sandbox node pools and live capacity.
Azure Kubernetes Service Contributor	Create pools and update count, autoscaler, labels, or taints.

Grant both on the cluster or its resource group. Internal deployments assign Reader on the Platform resource group to ContextAPIManagedIdentity (deploy/internal/FoundationaLLM.Deployment.Internal/private/00-shared/data/azure-role-assignments.json). Existing deployments that only have the two AKS roles from rc102 must add Reader. Add Azure Kubernetes Service Contributor if operators will create or edit pools from the portal.

Persistent workspace SKUs that use Azure Files (standard-smb, shared-ssd, dedicated-ssd) also need Kubernetes StorageClasses named in AKSSandboxDefaults and Azure RBAC on the FileStorage account those classes reference. App Config names alone are not enough. If the StorageClass is missing, sandbox create fails with Persistent workspace claim …-provisioning did not bind within 60 seconds (the PVC event is storageclass.storage.k8s.io "…" not found).

shared-ssd is one pre-created Premium share plus a folder per conversation (shareName and folderName: ${pvc.metadata.name} on the StorageClass). Do not point WorkspaceSharedSsdStorageClassName at a stock azurefile-csi-premium class; that provisions a new share per PVC.

Workload-identity token mounts (mountWithWorkloadIdentityToken: "true" on the StorageClass) require AKS 1.35 or later (Azure Files CSI v1.35+) and SMB OAuth enabled on the storage account. On AKS 1.34 the CSI driver rejects that parameter (invalid parameter "mountWithWorkloadIdentityToken").

Create a user-assigned workspace files identity, federate it to system:serviceaccount:<sandbox-namespace>:<WorkspaceServiceAccountName>, set that identity as the StorageClass clientID, and apply the StorageClass before publishing shared-ssd policies.

Identity	Azure role	Scope	Required for
Workspace files MI (StorageClass clientID)	Storage File Data SMB Share Contributor	FileStorage account	SMB mount as the sandbox workspace identity
Workspace files MI	Storage File Data SMB MI Admin	FileStorage account	Identity-based SMB / token mount
Workspace files MI	Storage Account Contributor	FileStorage account	CSI folder create when the driver retrieves account keys
Context API managed identity	Storage File Data Privileged Contributor	File share (or account)	REST CreateIfNotExists of the conversation folder
Context API managed identity	Storage Account Key Operator Service Role	FileStorage account	Key fallback if OAuth folder create fails
AKS cluster system-assigned identity	Storage Account Contributor	Resource group that contains the FileStorage account	CSI controller Microsoft.Storage/storageAccounts/fileServices/shares/read (and folder provisioning). Account-scope Contributor alone is not sufficient for the AKS ARM proxy; create fails with AuthorizationFailed on that action.
AKS kubelet / agentpool identity	Storage Account Contributor	FileStorage account	Node-side Azure Files CSI operations

Inbound managed identities and service principals that act as themselves (no X-USER-IDENTITY substitution) no longer store the Entra object id as UPN. Call context mints a reserved UPN from the Graph service-principal display name:

{slug(displayName)}@foundationallm.internal

Example: the LemonDesert AKS Sandbox MCP identity id-akssandbox-mcp-server-lemondesert becomes id-akssandbox-mcp-server-lemondesert@foundationallm.internal. Platform service processes use platform-service-{servicename}@foundationallm.internal for ServiceContext.ServiceIdentity.

Existing sandboxes, conversation files, and other Cosmos rows created by an inbound MI/SP with the raw object id as UPN will not match the new owner key. Get, command, file, and delete calls from that same identity return 404 unless the caller has Sandboxes Admin (owner bypass). Recreate those sandboxes (and any MI-owned conversation artifacts) after upgrade. User-owned sandboxes created through the User Portal or a successful identity substitution are unchanged.

Re-run Authorization API role-definition initialization (or otherwise refresh built-in role definitions) on existing deployments so Sandboxes Admin assignments pick up the new actions.

The Management API managed identity must be able to list sandbox image catalogs. Grant both of the following Azure roles on each Azure Container Registry used for sandbox images (the environment ACR and the registry from AKSSandboxDefaults.ContainerImage):

• AcrPull
• Container Registry Repository Reader

AcrPull alone is not always enough for repository catalog listing. These ARM assignments do not unlock ACR data-plane token exchange across Entra tenants; product ACRs that enable anonymous pull are listed anonymously in that case.

Internal deployments apply the catalog roles from deploy/internal/FoundationaLLM.Deployment.Internal/private/00-shared/scripts/azure-role-assignments.ps1 (Ensure-AcrCatalogReadRoleAssignments) when initializing the Management API.

Deployment changes

Internal AKS deployments of Authorization API and Management API now request 0.5 CPU and 1Gi memory (previously 1 CPU and 2Gi). Existing deployments are not resized automatically; re-run the management-plane deploy scripts (or patch the deployments) if you want the new requests.

Starting from 1.0.0-rc115

Configuration changes

The following new App Configuration settings are required for the Core API to serve public branding assets (logos, favicons, and similar files) from Azure Storage:

Name	Default value	Description
FoundationaLLM:APIEndpoints:CoreAPI:Configuration:PublicAssets:Storage:AccountName	${env:AZURE_STORAGE_ACCOUNT_NAME}	The name of the Azure Storage account that holds Core API public branding assets.
FoundationaLLM:APIEndpoints:CoreAPI:Configuration:PublicAssets:Storage:AuthenticationType	AzureIdentity	The type of authentication used to connect to the Azure Storage account. Can be one of: AzureIdentity, AccountKey, or ConnectionString.

Existing deployments must add these keys to the App Configuration store. The Core API reads assets from the {instanceId}/public-assets/{fileName} blob path.

Starting from 1.0.0-rc102

Configuration changes

A new feature flag named FoundationaLLM.AgentPortal.SelfService has been added. This flag enables or disables the self-service capabilities in the FoundationaLLM Agent Portal. The default value is false. The feature flag must be created in Azure App Configuration.

The following optional App Configuration settings customize the progress messages emitted by LangChain API agent workflows:

Name	Default value	Description
FoundationaLLM:APIEndpoints:LangChainAPI:Configuration:WorkflowMainStatusMessages	Planning my next moves...	A pipe-separated list of status messages. The workflow randomly selects one whenever it determines whether tools are needed. Messages support the {current_round} and {max_rounds} placeholders. When the setting is missing, empty, or contains no non-empty entries, the existing Checking if I need to use tools... message is used, including its round number for multi-round workflows.
FoundationaLLM:APIEndpoints:LangChainAPI:Configuration:WorkflowToolStatusMessages	Running tools that help me produce the answer...	A pipe-separated list of status messages. The workflow randomly selects one before executing tools. Messages support the {current_round} and {max_rounds} placeholders. When the setting is missing, empty, or contains no non-empty entries, the existing I am running the tools that I need to produce the answer... message is used.

For example, Reviewing round {current_round} of {max_rounds}|Deciding which tools to use provides two possible main workflow messages. Whitespace around each pipe-separated entry is ignored. Unknown placeholders are preserved literally.

The following new App Configuration settings are required for the Management API:

Name	Default value	Description
FoundationaLLM:APIEndpoints:ManagementAPI:Configuration:Entra:RequireScopes	true	Indicates whether a delegated scope claim (scp) or an application permission claim (roles) is required for authorization. Set to false to allow authentication from an external proxy API.
FoundationaLLM:APIEndpoints:ManagementAPI:Configuration:Entra:AllowACLAuthorization	false	Indicates whether tokens that do not have either of the "scp" or "roles" claims are accepted (True means they are accepted). Set to true to allow authentication from an external proxy API.

With the default values, the Management API enforces the same Entra ID scope and app-role authorization model as the Core API. Callers (including the Management Portal and managed identities) must present tokens that include the configured delegated scopes or at least one application permission for the Management API audience.

Update the following App Configuration setting to accept both member and B2B guest user principal names for identity substitution:

Name	Description
FoundationaLLM:Instance:IdentitySubstitutionUserPrincipalNamePattern	Regular expression that validates substituted UPNs in the X-USER-IDENTITY header.

Required value:

^(?:[A-Za-z0-9._-]+@foundationallm\.ai|[A-Za-z0-9._-]+#ext#@foundationallm\.onmicrosoft\.com)$

This pattern matches standard member UPNs (user@foundationaLLM.ai) and Entra ID guest UPNs (user#EXT#@foundationallm.onmicrosoft.com). Deployments that use identity substitution (for example, load testing or delegated impersonation) must update this value so guest accounts are not rejected when substituting identity via the X-USER-IDENTITY header.

The following App Configuration settings have been removed. All platform components resolve their Azure Cosmos DB configuration exclusively through the well-known Platform-Database data source (managed by the FoundationaLLM.DataSource resource provider); these settings are no longer read by any component and can be safely deleted from existing App Configuration stores:

Name
FoundationaLLM:APIEndpoints:CoreAPI:Configuration:CosmosDB:Endpoint
FoundationaLLM:APIEndpoints:CoreAPI:Configuration:CosmosDB:Database
FoundationaLLM:APIEndpoints:CoreAPI:Configuration:CosmosDB:Containers
FoundationaLLM:APIEndpoints:CoreAPI:Configuration:CosmosDB:MonitoredContainers
FoundationaLLM:APIEndpoints:CoreAPI:Configuration:CosmosDB:ChangeFeedLeaseContainer
FoundationaLLM:APIEndpoints:StateAPI:Configuration:CosmosDB:Endpoint
FoundationaLLM:APIEndpoints:StateAPI:Configuration:CosmosDB:Database
FoundationaLLM:APIEndpoints:StateAPI:Configuration:CosmosDB:Containers
FoundationaLLM:APIEndpoints:ContextAPI:Configuration:FileService:CosmosDB:Endpoint
FoundationaLLM:APIEndpoints:ContextAPI:Configuration:FileService:CosmosDB:Database
FoundationaLLM:APIEndpoints:ContextAPI:Configuration:FileService:CosmosDB:Containers
FoundationaLLM:DataPipeline:State:CosmosDB:Endpoint
FoundationaLLM:DataPipeline:State:CosmosDB:Database
FoundationaLLM:DataPipeline:State:CosmosDB:Containers

New deployments no longer create these settings. Ensure the Platform-Database data source resource exists and points to the correct Azure Cosmos DB account and database before removing the settings from existing environments.

The following new App Configuration setting identifies the platform database data source:

Name	Default value	Description
FoundationaLLM:Instance:PlatformDataSource	Platform-Database	The name of the data source that provides the Azure Cosmos DB database used by all platform Cosmos DB features to initialize. When the setting is missing, components fall back to the well-known Platform-Database data source.

The Azure Cosmos DB data source resource supports a new optional partitioning_strategy property: a JSON dictionary where keys are container names and values are comma-separated lists of the document properties included in the partition key (in hierarchy order, for example instanceId,sessionId). The strategy is honored in two places:

• At runtime, the centralized partition key helper shapes all partition keys according to the strategy defined on the platform database data source (containers not covered by the strategy keep the default hierarchical key: instance identifier followed by the container-specific key). Besides the instance identifier, at most one property can participate in a container's partition key.
• During migration, the fllm-cdb db migrate utility honors the FoundationaLLM:Instance:PlatformDataSource setting when --app-config is provided: the partitioning strategy defined on the designated data source resource overrides the built-in partition key definitions for the newly created containers.

The Azure Cosmos DB data source resource also supports a new optional use_instance_id_in_partition_key boolean property (default true) that controls whether the FoundationaLLM instance identifier participates in partition keys. When set to false:

• The instanceId/instance_id entries in partitioning_strategy are ignored, and the instance identifier is not added to any partition key (including the default hierarchical keys of containers not covered by the strategy).
• Instance-scoped partition key prefixes are no longer possible, so instance-wide queries run cross-partition.
• The fllm-cdb db migrate utility excludes the instanceId/instance_id properties from the partition keys of the newly created containers.

Greenfield deployments and use_instance_id_in_partition_key: false

The internal deployment script deploy/internal/FoundationaLLM.Deployment.Internal/private/00-shared/scripts/common-storage.ps1 provisions Cosmos DB container partition keys from the -UseInstanceIdInPartitionKey parameter (default true). Set it to false to omit the FoundationaLLM instance identifier from every container partition key, matching the Platform-Database data source use_instance_id_in_partition_key setting.

Configure it in either place:

• master.yaml: foundationallm.use_instance_id_in_partition_key: false
• Deploy script override: Deploy-ManagementPlane.ps1 -UseInstanceIdInPartitionKey:$false

The value must stay aligned with the Platform-Database data source resource. When the flag is false, UPN-partitioned containers are created with /upn only (for example UserProfiles, UserSessions) instead of /instanceId, /upn.

Existing containers are not altered if they already exist; partition keys are immutable. Changing the flag after initial provisioning requires a new database (for example via fllm-cdb db migrate --app-config ...) or manual container recreation.

Running the database migration utility

The migration utility is the fllm-cdb executable built from the AzureCosmosDBUtility project in utils/azure-cosmosdb. It authenticates with the Azure CLI credential, so run az login first with an identity that has data-plane access to the Azure Cosmos DB account and, when --app-config is used, read access to the App Configuration store and the storage account holding the resource provider artifacts.

fllm-cdb db migrate --help

The copy reads the source through the change feed, in parallel across the source feed ranges (physical partitions), and writes through the Azure Cosmos DB bulk mode with --parallelism concurrent operations (default 500). Its speed is bounded by the provisioned throughput of the containers: the utility temporarily raises the source and target containers to 10,000 RU/s for the duration of each container copy, and throttling (429) is expected and absorbed by the SDK. Progress (documents processed and the current rate) is reported every five seconds; the per-feed-range change feed positions are checkpointed at the same cadence, so an interrupted run loses at most a few seconds of work.

The typical migration flow is a dry run, followed by the big initial copy, followed by a final quick delta run immediately before switching the application to the new database:

# 1. Dry run - reports what would be done, no writes.
fllm-cdb db migrate `
    --account "https://<cosmos-account>.documents.azure.com:443/" `
    --target-database database-v2 `
    --instance-id "<instance-guid>" `
    --app-config "https://<appconfig-name>.azconfig.io" `
    --report-folder "C:\migration-reports"

# 2. Initial copy.
fllm-cdb db migrate `
    --account "https://<cosmos-account>.documents.azure.com:443/" `
    --target-database database-v2 `
    --instance-id "<instance-guid>" `
    --app-config "https://<appconfig-name>.azconfig.io" `
    --report-folder "C:\migration-reports" `
    --apply

# 3. Just before switching the application to the new database: delta run
#    (same command - the migration progress report makes it copy only what changed).
fllm-cdb db migrate `
    --account "https://<cosmos-account>.documents.azure.com:443/" `
    --target-database database-v2 `
    --instance-id "<instance-guid>" `
    --app-config "https://<appconfig-name>.azconfig.io" `
    --report-folder "C:\migration-reports" `
    --apply

The command supports the following options:

Option	Required	Description
--account	Yes	The Azure Cosmos DB account endpoint URI (accessed with the Azure CLI credential).
--target-database	Yes	The name of the target database (created if missing). Must differ from the source database.
--instance-id	Yes	The FoundationaLLM instance identifier used to backfill instanceId/instance_id on documents that do not carry it.
--app-config	No	The App Configuration endpoint URI or connection string. When provided, the FoundationaLLM:Instance:PlatformDataSource setting is honored: the designated data source's partitioning_strategy overrides the built-in partition key definitions, and use_instance_id_in_partition_key: false excludes the instance identifier from all partition keys. Without it, the built-in defaults apply.
--source-database	No	The name of the source database. Defaults to database.
--containers	No	A space-separated subset of containers to migrate (for example, --containers Sessions UserSessions). Defaults to all FoundationaLLM containers.
--apply	No	Applies the changes. Without this flag the command runs in dry-run mode.
--full	No	Ignores the saved change feed positions and watermarks and copies everything again (idempotent).
--parallelism	No	The maximum number of documents written concurrently. Writes use the bulk mode, which only fills its batches with hundreds of operations in flight; small values leave every batch waiting on the dispatch timer. Defaults to 500.
--report-folder	No	The folder receiving the reports written by the command (created when missing). Defaults to reports in the current directory.
--direct-connection	No	Uses direct connection mode instead of gateway mode.
--no-progress	No	Suppresses progress output on stderr.

All commands write their reports into --report-folder under fixed names, so a report is always at a predictable path and re-running a command overwrites its own reports (migration issue logs are the exception: they merge by source document id). Use different report folders to keep runs side by side. The migration and verification commands produce:

Report	Contents
db-migrate-state.json	Migration progress: the per-container change feed positions (one continuation token per source feed range) and delta watermarks. Also an input - this is what makes re-runs resumable and incremental. State files written by earlier versions of the utility (_ts checkpoints only) are still honored: the copy resumes from that point in time.
db-migrate-actions-{container}.jsonl	Current copy issues for that container (upsert failures). Merged across runs by source document id: a new error replaces the previous record for the same id, and a successful copy removes the id.
db-migrate-summary.json	The migration run summary (also printed to stdout).
db-verify-actions.jsonl	One record per verification finding (missing, extra, stale, and content-mismatched documents).
db-verify-summary.json	The verification run summary (also printed to stdout).

Two operational notes:

• db-migrate-state.json is what makes re-runs incremental. Pass the same --report-folder to the apply and delta steps so the final run only copies the delta.
• The migration summary includes platform_data_source and use_instance_id_in_partition_key, so you can confirm the partitioning configuration that was actually applied.

Verifying the migration

The fllm-cdb db verify command is read-only and confirms that the target database matches the state expected after migration. It replays the migration transformation (instance identifier backfill, UPN normalization, identity recomputation, duplicate collapsing) over the source to compute the expected target state, then compares the target in both directions and reports missing documents, extra documents, stale documents (the source was modified after the copy, so a delta run is required), and content mismatches (canonical SHA-256 hashes with system properties excluded). Target container partition key definitions are validated as well.

fllm-cdb db verify `
    --account "https://<cosmos-account>.documents.azure.com:443/" `
    --target-database database-v2 `
    --instance-id "<instance-guid>" `
    --app-config "https://<appconfig-name>.azconfig.io" `
    --report-folder "C:\migration-reports"

The command exits with a non-zero code when any container fails verification. Run it after the initial copy to gauge progress, and again after the final delta run - with traffic to the source stopped - as the authoritative cutover gate. Verification against a live source is only a snapshot: documents modified between the source and target scans surface as stale or mismatched, and TTL-expired source documents surface as extra target documents.

Security changes

The UPN policy must be assigned to FoundationaLLM.User/userProfiles. The UPN policy must be assigned to FoundationaLLM.Platform/sandboxes. The UPN policy must be assignrf to FoundationaLLM.Context/memories.

Two new built-in role definitions have been added for sandbox resources:

• Sandboxes Contributor (a8f3e2d1-4c5b-6a7e-8f9d-0e1f2a3b4c5d) - read, write, and delete sandboxes owned by the assigned principal (enforced by the UPN ownership policy on FoundationaLLM.Platform/sandboxes).
• Sandboxes Admin (b9e4f3a2-5d6c-7b8e-9f0a-1b2c3d4e5f6a) - read, write, and delete all sandboxes regardless of owner.

Both roles grant the following authorizable actions:

• FoundationaLLM.Platform/sandboxes/read
• FoundationaLLM.Platform/sandboxes/write
• FoundationaLLM.Platform/sandboxes/delete

Assign Sandboxes Contributor to users who should manage their own sandbox code sessions. Assign Sandboxes Admin to operators who must access or manage sandboxes for all users.

The Core API and Management API Entra ID app registrations must expose a new application permission (app role) for MCP server access:

App registration	Display name	Value	Description
Core API	MCP Invoker	MCP.Invoke	Allows MCP servers to invoke the Core API. RBAC is managed inside FoundationaLLM.
Management API	MCP Invoker	MCP.Invoke	Allows MCP servers to invoke the Management API. RBAC is managed inside FoundationaLLM.

Each role is assigned to Applications (not users). Grant the appropriate MCP.Invoke app role to each MCP server managed identity (or other app-only caller) on the corresponding API app registration.

With RequireScopes enabled (the default for both APIs), app-only tokens must include the MCP.Invoke app role for the target API audience. Re-run Entra app registration initialization on existing deployments to create the roles, then assign them to MCP server identities before enabling scope enforcement.

The FoundationaLLM-Core-API app registration must include the upn optional claim on access tokens, with Externally authenticated enabled (include_externally_authenticated_upn). This ensures B2B guest users receive a UPN in access tokens presented to the Core API. Without this claim, guest accounts may omit upn, which breaks UPN-based ownership policies, quotas, and partitioning.

Re-run Entra app registration initialization on existing deployments, or manually add the claim under Token configuration on the Core API app registration. Internal deployments apply this automatically via foundationallm-core-api.template.json.

Example manifest entry:

"optionalClaims": {
  "accessToken": [
    {
      "name": "upn",
      "essential": false,
      "additionalProperties": [
        "include_externally_authenticated_upn"
      ]
    }
  ]
}

Configuration changes

The Context API supports sandbox code sessions (environment_type: Sandbox) backed by pod-only, Kata-isolated managed deployments on AKS. Deployments must configure the following App Configuration value:

Name	Content type	Description
FoundationaLLM:APIEndpoints:ContextAPI:Configuration:SandboxCodeSessionService:AKSSandboxDefaults	application/json	Default settings for AKS sandbox code session managed deployments provisioned by the Context API.

Example value:

{
  "ContainerImage": "myregistry.azurecr.io/sandbox-runtime:1.0.0",
  "ContainerPort": 8080,
  "CPU": "500m",
  "Memory": "512Mi",
  "ReadinessPollIntervalSeconds": 2,
  "ReadinessTimeoutSeconds": 120,
  "AKSClusterResourceId": "/subscriptions/{subscription-id}/resourceGroups/{resource-group}/providers/Microsoft.ContainerService/managedClusters/{cluster-name}",
  "Namespace": "fllm-sandbox",
  "NodePoolName": "sandboxpool1",
  "TLSCertificateSecretName": "sandbox-tls",
  "Hostname": "sandbox.example.com",
  "GatewayName": "shared-gateway",
  "RequireExternalIPAddress": true,
  "RequirePrivateIPAddress": false,
  "PathPrefix": "/sandbox-custom",
  "PathRewritePrefix": "/api"
}

When RequireExternalIPAddress is true, Hostname, GatewayName, and TLSCertificateSecretName are required. By default, each sandbox session uses the session identifier as the HTTPRoute path prefix with a rewrite to / (for example, https://sandbox.example.com/{session-id}). Optionally set PathPrefix and PathRewritePrefix together to override the path routing applied to every sandbox deployment. Set RequirePrivateIPAddress to true only when the gateway load balancer should use a private IP address (requires RequireExternalIPAddress to be true).

Security changes

The managed identity of the Context API must have the following roles to provision sandbox sessions:

• FoundationaLLM.Platform/managedEndpointDeployments/write on the FoundationaLLM instance.
• Azure Kubernetes Service Cluster User Role on each AKS cluster referenced in AKSSandboxDefaults.AKSClusterResourceId.
• Azure Kubernetes Service RBAC Cluster Admin on each AKS cluster referenced in AKSSandboxDefaults.AKSClusterResourceId.

A new role definition has been added:

• Managed Deployments Admin (52193588-ce65-4415-a89d-38399ed481e6) - required for sandbox session reading and provisioning.

Agent Portal access

Agent Portal access is now controlled by Reader role assignments on the following app configuration set scope:

/instances/<instance_id>/providers/FoundationaLLM.Configuration/appConfigurationSets/AgentPortal

Before deploying the updated Agent Portal, grant the Reader role on this scope to every user or Entra security group that requires portal access. In the Management Portal, open Security > Role Assignments, select Portal Access as the scope, select Agent Portal, and create a Reader role assignment for each allowed principal.

Users without this role assignment, or users for whom the AgentPortal app configuration set cannot be read, are denied access to the entire Agent Portal after sign-in.

Agent Portal environment variables (brand wordmarks)

Deployments that override the Agent Portal brand logo with a wordmark image (an image of text) must set accessible-name environment variables on the Agent Portal container / server/.env. Without them, screen readers announce a generic or incorrect product name (WCAG 1.1.1).

Name	Required when	Description
FLLM_LOGO_ALT	FLLM_LOGO_IMAGE is set to a wordmark	Accessible name for the sidebar brand logo. Must match the text rendered in the image (for example Clarity Platform).
FLLM_LOGIN_LOGO_ALT	FLLM_LOGIN_LOGO_IMAGE is set to a wordmark	Accessible name for the unauthenticated sign-in wordmark. Falls back to FLLM_LOGO_ALT, then Agent Portal, if unset.

When FLLM_LOGO_IMAGE is used, prefer setting FLLM_LOGO_ALT even if the theme document's sidebar.brandLogo.alt is already present — the env logo replaces the theme image, and a stale theme alt (for example FoundationaLLM) will otherwise be announced. Also keep sidebar.brandLogo.alt in the active theme document aligned with the visible wordmark for deployments that do not use the env logo override.

Artifact changes

A new artifact package, FoundationaLLM Platform Themes (FoundationaLLM-Platform-Themes), distributes the default Agent Portal theme catalog. Package sources live under packages/foundationallm-platform-themes. Releases are published by pushing a package-platform-themes-<version> tag (for example package-platform-themes-1.0.0-rc102), which builds the .fllm package and uploads it to the platform package store.

Existing and new deployments that use the Agent Portal must deploy this package from the Management Portal (Packages → Artifacts). Loading the package upserts the following App Configuration keys in the AgentPortal app configuration set:

Key	Description
FoundationaLLM:AgentPortal:Themes:Manifest	Theme catalog manifest (schemaVersion 1.0). Lists themeIds, defaultThemeId (default interwoven), and allowedThemeIds.
FoundationaLLM:AgentPortal:Themes:Interwoven	Default Interwoven theme document (theme-interwoven.json).
FoundationaLLM:AgentPortal:Themes:Star	Star theme document (theme-star.json).
FoundationaLLM:AgentPortal:Themes:Sunburst	Sunburst theme document (theme-sunburst.json).

Theme document values in appConfigurations.json may reference sibling JSON artifact files (for example "value": "theme-star.json"); the artifact package processor resolves those files when the package is loaded. Package load also stamps artifact_package_name / artifact_package_version on the managed theme documents and Manifest.

When the package is loaded into an instance that already has a Manifest (for example after custom themes were added in the Management Portal), packaged theme IDs are merged into the existing Manifest. The packaged Manifest does not replace an existing Manifest, so custom themes and an existing defaultThemeId are preserved while the platform themes (interwoven, star, sunburst) are added if missing.

After deployment, review the catalog in the Management Portal (Platform → Themes). Core API exposes the catalog through the existing AgentPortal app configuration set; there is no separate theme endpoint.

Starting from 1.0.0-beta126

A new app configuration value named FoundationaLLM:Configuration:AppConfigurationConnectionString must be created pointing to the Azure Key Vault secret named foundationallm-appconfiguration-connectionstring.

The Azure AI Search indexes agent-private-store-files and conversation-files muse get a semantic configuration added to the index. The semantic configuration must be named default and must have a content field named Text.

Enable workload identity and OIDC issuer on the AKS cluster (see enable-workload-identity.ps1).

Starting from 1.0.0-beta125

MCP Client Tool: opt-in system HTTP headers

The MCP Client Tool no longer implicitly sends X-USER-IDENTITY on every HTTP MCP request. User identity and conversation session context must be configured explicitly as HTTP headers using runtime tokens:

Token	Header
context://user_identity	X-USER-IDENTITY
context://conversation_id	X-FLLM-SESSION-ID

Configure these in the Management Portal on MCP Server endpoints or inline MCP tool HTTP headers (System / runtime context source). Internal MCP servers (e.g., Genesis) should add both headers on the MCP server endpoint resource; external MCP servers should omit them unless required.

Use the sample resource in docs/tools/genesis-mcp-endpoint-template.json when creating or updating Genesis-MCP-Production (or equivalent) via the Management API. Replace the placeholder URL with your Genesis MCP server base URL.

The deprecated agent tool property forward_user_identity is no longer surfaced in the Management Portal and is not honored at runtime. When an agent tool with forward_user_identity: true is opened in the portal, the legacy flag is migrated in the editor to X-USER-IDENTITY: context://user_identity under inline transport headers. Re-save affected agent tools and ensure endpoint resources include the system headers above before deploying this change.

Starting from 1.0.0-beta110

The managed identity of the Management API must have the following roles assigned:

• Azure Kubernetes Service Cluster User Role on each AKS cluster referenced by FoundationaLLM.Platform/managedEndpointDeployments resources through cluster_resource_id.
• Azure Kubernetes Service RBAC Cluster Admin on each AKS cluster referenced by FoundationaLLM.Platform/managedEndpointDeployments resources through cluster_resource_id.
• Contributor on each resource group containing managed identities referenced by FoundationaLLM.Platform/managedEndpointDeployments resources through managed_identity_resource_id.

Starting from 0.9.9-rc170

The managed identity of the Core Worker must have Reader on the FoundationaLLM instance.

New Users Cosmos DB container must be created with a hierarchical partition key of /instance_id, /upn and autoscale throughput with a maximum of 4000 RU/s.

New AzureLogicAppWorkflow.json API endpoint configuration:

{
  "type": "api-endpoint",
  "name": "AzureLogicAppWorkflow",
  "display_name": null,
  "description": null,
  "cost_center": null,
  "category": "General",
  "subcategory": null,
  "authentication_type": "AzureIdentity",
  "url": "...",
  "status_endpoint": null,
  "url_exceptions": [],
  "authentication_parameters": {
    "scope": "https://management.azure.com/.default"
  },
  "timeout_seconds": 60,
  "retry_strategy_name": "ExponentialBackoff",
  "provider": "microsoft",
  "api_version": "2016-10-01",
}

New Azure Logic App created. Authentication configured for the managed identity of Core API.

FoundationaLLM:Events:Profiles:CoreWorker configuration entry added with the following value: {"EventProcessingCycleSeconds":60,"Topics":[]}. FoundationaLLM:Events:Profiles:StateAPI configuration entry added with the following value: {"EventProcessingCycleSeconds":60,"Topics":[]}.

New Platform-Database Data Source (pointing to the Azure Cosmos DB account).

State API has Reader role on the FoundationaLLM instance.

Starting from 0.9.9-rc102

The managed identity of the Management API must have the Log Analytics Reader role assigned on the Log Analytics workspace used for the FoundationaLLM instance.

Starting from 0.9.9-beta170

Security changes

Only members of the Configuration Contributor role are now allowed to change configuration settings in the Management Portal.

Starting from 0.9.9-beta140

Security changes

Add the UPN policy to FoundationaLLM.Agent/agentTasks.

Starting from 0.9.9-beta130

Configuration changes

A new FoundationaLLM:Analytics configuration namespace has been added to support the FoundationaLLM Analytics service and abuse detection features.

The following App Configuration values have been added:

Name	Default value	Description
FoundationaLLM:Analytics:AbuseDetection:HighRequestRateThreshold	100	Number of requests within the monitored interval considered a high request rate and subject to mitigation.
FoundationaLLM:Analytics:AbuseDetection:ExtremeRequestRateThreshold	500	Number of requests within the monitored interval considered an extreme request rate for aggressive throttling or blocking.
FoundationaLLM:Analytics:AbuseDetection:RapidFireThreshold	20	Count of rapid consecutive requests from the same source that, when exceeded within the RapidFireWindowMinutes, indicate rapid-fire abuse.
FoundationaLLM:Analytics:AbuseDetection:RapidFireWindowMinutes	1	Time window in minutes used to evaluate rapid-fire request behavior for the RapidFireThreshold.
FoundationaLLM:Analytics:AbuseDetection:ContinuousUsageHours	20	Number of continuous hours of activity from a single user or client considered suspicious and subject to further checks.
FoundationaLLM:Analytics:AbuseDetection:MaxInactivityGapMinutes	30	Maximum gap in minutes between activities that is still considered continuous usage. If a user is inactive for longer than this, the continuous usage timer resets.
FoundationaLLM:Analytics:AbuseDetection:AgentHoppingThreshold	10	Number of distinct agents accessed within the AgentHoppingWindowMinutes that indicates agent-hopping abuse.
FoundationaLLM:Analytics:AbuseDetection:AgentHoppingWindowMinutes	60	Time window in minutes used to evaluate agent-hopping behavior for the AgentHoppingThreshold.
FoundationaLLM:Analytics:AbuseDetection:HighErrorRateThreshold	30	Percentage (0-100) of error responses within the monitored interval considered a high error rate that may indicate abuse or system issues.

The following App Configuration values have been added to support the new FoundationaLLM.Analytics resource provider:

Name	Default value	Description
FoundationaLLM:ResourceProviders:Analytics:Storage:AuthenticationType	AzureIdentity	The type of authentication used to connect to the Azure Blob Storage account used by the FoundationaLLM.Analytics resource provider. Can be one of: AzureIdentity, AccountKey, or ConnectionString.
FoundationaLLM:ResourceProviders:Analytics:Storage:AccountName	<storage_account_name>	The name of the Azure Blob Storage account used by the FoundationaLLM.Analytics resource provider.

where <storage_account_name> is the name of the Azure Blob Storage account used by the FoundationaLLM.Analytics resource provider.

Security changes

A new built-in role definition Analytics Reader (f3ab7cdd-d0c3-43d1-84d7-7409305e0c4a) has been added. Any security principal that needs to query analytics data through the FoundationaLLM.Analytics resource provider must be assigned the Analytics Reader role at the appropriate scope. The role grants the following authorizable action:

• FoundationaLLM.Analytics/analytics/read
• FoundationaLLM.Conversation/conversationAnalytics/read
• FoundationaLLM.Context/contextAnalytics/read

Starting from 0.9.9-beta100

Configuration changes

The following configuration entries have been added:

Name	Default value
FoundationaLLM:ResourceProviders:Quota:Storage:AccountName	<storage_account_name>
FoundationaLLM:ResourceProviders:Quota:Storage:AuthenticationType	AzureIdentity
FoundationaLLM:Code:DedicatedCustomContainers

where <storage_account_name> is the name of the Azure Blob Storage account used by the FoundationaLLM Quota resource provider.

The DedicatedCustomContainers has an application/json content type with the following structure:

{
	"Environments": {
		"Python": {
			"UserEnvironments": {
				"user@foundationallm.ai": [
					{
						"Endpoint": "",
						"Enabled": true
					}
				]
			}
		}
	}
}

The FoundationaLLM.Preview feature flag must be created and set to false.

Security changes

The managed identity of the Core API and Management API services must be assigned the Quotas Contributor role on the FoundationaLLM instance.

Schema changes

The file quota/quot-store.json from the main storage account must be translated into Quota Definition resources and then deleted together with the quota container. Supported quota types are RateLimit, UsageLimit, and ResourceLimit.

For each plugin package, package_platform must be renamed to package_runtime. Reinstall new versions of all plugin packages.

Starting from 0.9.8-rc100

Security changes

The UPN policy must be assigned to FoundationaLLM.Context/files.

Configuration changes

A new API Endpoint Configuration object named LangSmithAPI must be created with the following properties:

{
  "type": "api-endpoint",
  "name": "LangSmithAPI",
  "category": "General",
  "subcategory": null,
  "authentication_type": "APIKey",
  "url": "https://api.smith.langchain.com",
  "status_endpoint": null,
  "url_exceptions": [],
  "authentication_parameters": {
    "api_key_configuration_name": "FoundationaLLM:APIEndpoints:LangSmithAPI:Essentials:APIKey"
  },
  "timeout_seconds": 60,
  "retry_strategy_name": "ExponentialBackoff",
  "provider": "langchain"
}

Important

The LangSmithAPI resource is used exclusively for development and testing purposes. In production environments, the URL should be set to an invalid value e http:/.

Starting from 0.9.8-beta121

Configuration changes

The following configuration entries were added:

Name	Default value
FoundationaLLM:APIEndpoints:GatewayAPI:Configuration:EmbeddingRequestMaxSizeTokens	6400
FoundationaLLM:APIEndpoints:GatewayAPI:Configuration:CompletionRequestMaxSizeTokens	20000

Starting from 0.9.8-beta118

Schema changes

The DataPipelineDefinitionSnapshot object schema in the FoundationaLLM.DataPipeline resource provider has the following breaking changes:

• DataPipelineDefinition renamed to data_pipeline_definition.

For all data pipeline definitions, the most recent data pipeline snapshot object must be updated accordingly.

Configuration changes

A new API endpoint object was created with the following parameters:

Property Name	Value
Name	AzureLogAnalytics
Provider	Microsoft
Category	General
Subcategory	None
Auth Type	Azure Identity
Endpoint URL	4a0da8f9-e406-40e1-801f-d5c5441bdca2 - the Log Analytics workspace id
Timeout	300

Permissions changes

The managed identity of the Data Pipeline API must have the Log Analytics Reader role assigned on the Log Analytics workspace used for the FoundationaLLM instance.

Starting from 0.9.8-beta116

Configuration changes

TheFoundationaLLM:APIEndpoints:CoreAPI:Configuration:AllowedUploadFileExtensions configuration entry has been reintroduced and has the following default value: c, cpp, cs, css, csv, doc, docx, gif, html, java, jpeg, jpg, js, json, md, pdf, php, png, pptx, py, rb, sh, tar, tex, ts, txt, xlsx, xml, zip, wav.

Security changes

The UPN policy must be assigned to FoundationaLLM.Context/fileOperations.

Starting from 0.9.7

Configuration changes

TheFoundationaLLM:APIEndpoints:CoreAPI:Configuration:AllowedUploadFileExtensions configuration entry has been removed and was replaced by the FoundationaLLM:APIEndpoints:ContextAPI:Configuration:FileService:AllowedFileExtensions configuration entry.

The following configuration entries have changed their default values:

Name	New value
FoundationaLLM:APIEndpoints:ContextAPI:Configuration:FileService:AllowedFileExtensions	c, cpp, cs, css, html, java, js, json, jsonl, md, php, py, rb, sh, tex, ts, txt, xml, yaml, yml, gif, jpeg, jpg, png, wav, pdf, docx, pptx, xlsx, csv, zip, tar, doc, ppt, xls
FoundationaLLM:APIEndpoints:ContextAPI:Configuration:FileService:KnowledgeSearchContextFileExtensions	c, cpp, cs, css, html, java, js, json, jsonl, md, php, py, rb, sh, tex, ts, txt, xml, yaml, yml, gif, jpeg, jpg, png, wav
FoundationaLLM:APIEndpoints:ContextAPI:Configuration:FileService:KnowledgeSearchContextFileMaxSizeBytes	{"c, cpp, cs, css, html, java, js, json, jsonl, md, php, py, rb, sh, tex, ts, txt, xml, yaml, yml": 1048576, "gif, jpeg, jpg, png, wav":20971520}
FoundationaLLM:APIEndpoints:ContextAPI:Configuration:FileService:KnowledgeSearchFileExtensions	c, cpp, cs, css, html, java, js, json, jsonl, md, php, py, rb, sh, tex, ts, txt, xml, yaml, yml, gif, jpeg, jpg, png, wav, pdf, docx, pptx

Starting from 0.9.7-rc487

Artifact changes

The new data pipeline ShieldedFileContent is available.

Role assignment changes

The following role assignments must be added to the Context API's Managed Identity:

Name	Type	Target
Data Pipelines Execution Manager	FoundationaLLM	FoundationaLLM instance.

Starting from 0.9.7-rc477

Artifact changes

A new workflow resource must be added to support the LangChainAgentWorkflow. Ensure the reference is added to _resource-references.json as well.

{
    "type": "langchain-agent-workflow",
    "name": "LangChainAgentWorkflow",
    "object_id": "/instances/8ac6074c-bdde-43cb-a140-ec0002d96d2b/providers/FoundationaLLM.Agent/workflows/LangChainAgentWorkflow",
    "display_name": "LangChainAgentWorkflow",
    "description": "LangChain Agent workflow",
    "cost_center": null,
    "properties": null,
    "created_on": "2025-11-16T00:00:00+00:00",
    "updated_on": "0001-01-01T00:00:00+00:00",
    "created_by": "SYSTEM",
    "updated_by": null,
    "deleted": false,
    "expiration_date": null
}

Starting from 0.9.7-rc472

Configuration changes

A new feature flag named FoundationaLLM.Agent.SelfService has been added. This flag enables or disables the self-service capabilities for agents in the FoundationaLLM User Portal and the Core API. The default value is true.

Starting from 0.9.7-rc470

Schema changes

The FoundationaLLM.Agent resource provider schema is updated to version 2. The schema changes are as follows:

• The agent type knowledge-management is replaced with generic-agent.
• The agent workflow type external-agent-workflow is replaced with generic-agent-workflow.

Artifact changes

A new workflow resource must be added to support the GenericAgentWorkflow. Ensure the reference is added to _resource-references.json as well.

{
    "type": "generic-agent-workflow",
    "name": "GenericAgentWorkflow",
    "object_id": "/instances/{instanceId}/providers/FoundationaLLM.Agent/workflows/GenericAgentWorkflow",
    "display_name": "Generic Agent Workflow",
    "description": "Generic Agent Workflow",
    "cost_center": null,
    "properties": null,
    "created_on": "2025-11-01T12:00:00+00:00",
    "updated_on": "0001-01-01T00:00:00+00:00",
    "created_by": "SYSTEM",
    "updated_by": null,
    "deleted": false,
    "expiration_date": null
}

Starting from 0.9.7-rc422

The managed identity of Core API must have the Data Pipelines Execution Manager role assigned on the FoundationaLLM instance.

Starting from 0.9.7-rc388

The external-modules-python container from the main storage account is obsolete and must be deleted.

Starting from 0.9.7-rc382

Python packages must be built using the build version.

Python packages must be deployed via the FoundationaLLM.Core PowerShell module.

Starting from 0.9.7-rc376

Configuration changes

The following App Configuration value have been added:

Name	Default value	Description
FoundationaLLM:UserPortal:Configuration:AgentManagementPermissionRequestUrl	N/A	The URL to request agent management permissions.
FoundationaLLM:UserPortal:Configuration:FeaturedAgentNames	N/A	The comma-separated list of featured agent names.

The following configuration values are obsolete and must be removed:

• FoundationaLLM:APIEndpoints:LangChainAPI:Configuration:ExternalModules:Modules
• FoundationaLLM:APIEndpoints:LangChainAPI:Configuration:ExternalModules:RootStorageContainer

Permissions changes

All User Portal users must have read permission on the /instances/<instance_id_>/providers/FoundationaLLM.Configuration/appConfigurationSets/UserPortal scope.

AllAgentsVirtualSecurityGroup (5bb493a2-5909-4771-93ba-d83b7b5a1de9) must have read permission on the /instances/<instance_id_>/providers/FoundationaLLM.Configuration/appConfigurationSets/UserPortal scope.

All Management Portal users must have read permissions on the /instances/<instance_id_>/providers/FoundationaLLM.Configuration/appConfigurationSets/ManagementPortal scope.

Starting from 0.9.7-rc365

Configuration changes

The following App Configuration value have been added:

Name	Default value	Description
FoundationaLLM:IdentityManagement:MicrosoftGraph:RetrieveOnPremisesAccountName	false	Indicates whether on-premises account names are retrieved by the Microsoft Graph API.

Starting from 0.9.7 (up to rc364)

Configuration changes

The value of the FoundationaLLM:Events:Profiles:DataPipelineAPI configuration entry is changed to:

{
	"EventProcessingCycleSeconds": 5,
	"Topics": [
		{
			"Name": "resource-providers",
			"SubscriptionPrefix": "rp-dp"
		}
	]
}

The FoundationaLLM:APIEndpoints:CoreAPI:Configuration:CompletionResponsePollingIntervalSeconds configuration entry is replaced by the FoundationaLLM:APIEndpoints:CoreAPI:Configuration:CompletionResponsePollingIntervalMilliseconds configuration entry. The default value is now 100 milliseconds instead of 1 second. This change is made to improve the responsiveness of the completion response polling mechanism.

The values of the FoundationaLLM:APIEndpoints:ContextAPI:Configuration:KnowledgeGraphService:Embedding configuration value (listed below as a newly added value) must be configured with a content type of application/json and must have the following structure:

{
    "EmbeddingAPIEndpointConfigurationObjectId": "/instances/<instance_id>/providers/FoundationaLLM.Configuration/apiEndpointConfigurations/AzureOpenAI",
    "ModelDeployments": {
        "text-embedding-3-large": "embeddings-3-large"
    }
}

where:

• <instance_id> is the identifier of the FoundationaLLM instance.
• It is assumed that the AzureOpenAI API endpoint configuration is already created. If not, AzureOpenAI must be replaced with the name of the API endpoint configuration that is used for embeddings.
• It is assumed that a model deployment named embeddings-3-large is already created in the Azure OpenAI service for the text-embedding-3-large embedding model. If not, it must be replaced with the name of the model deployment that is used for embeddings with the text-embedding-3-large embedding model.

The following App Configuration value have been added:

Name	Default value	Description
FoundationaLLM:ResourceProviders:Context:Storage:AccountName	<storage_account_name>	The name of the storage account used by the FoundationaLLM Context resource provider.
FoundationaLLM:ResourceProviders:Context:Storage:AuthenticationType	AzureIdentity	The type of authentication used by the FoundationaLLM Context resource provider to connect to the storage account.
FoundationaLLM:APIEndpoints:ContextAPI:Configuration:KnowledgeService:Embedding	{"EmbeddingAPIEndpointConfigurationObjectId": null, "ModelDeployments": {}}	The embedding configuration used by the FoundationaLLM Context API Knowledge service. This configuration value must have a content type of application/json.
FoundationaLLM:APIEndpoints:ContextAPI:Configuration:KnowledgeService:Storage:AccountName	<storage_account_name>	The name of the dedicated storage account used by the FoundationaLLM Context API Knowledge service. This must be the same storage account as the one used by the FoundationaLLM Context API.
FoundationaLLM:APIEndpoints:ContextAPI:Configuration:KnowledgeService:Storage:AuthenticationType	AzureIdentity	The type of authentication used by the FoundationaLLM Context API Knowledge service to connect to the dedicated storage account.
FoundationaLLM:APIEndpoints:GatewayAPI:Configuration:TokenRateLimitMultiplier	0.8	The multiplier used to calculate the token rate limit for the Gateway API. This value is used to ensure that the rate limit is not exceeded and accounts for the possible differences in tokenization between the Gateway API and the deployed model(s).
FoundationaLLM:APIEndpoints:ContextAPI:Configuration:FileService:KnowledgeSearchFileExtensions	c, cpp, cs, css, doc, docx, html, java, js, json, md, pdf, php, pptx, py, rb, sh, tex, ts, txt, gif, jpeg, jpg, png	The comma-separated list file extensions that are processed as sources for knowledge search.
FoundationaLLM:APIEndpoints:ContextAPI:Configuration:FileService:KnowledgeSearchContextFileExtensions	c, cs, cpp, css, html, java, js, json, jsonl, md, php, py, rb, sh, ts, tex, txt, gif, jpeg, jpg, png	The comma-separated list of file extensions that indicate files that can be directly used in the context of a completion request.
FoundationaLLM:APIEndpoints:ContextAPI:Configuration:FileService:KnowledgeSearchContextFileMaxSizeBytes	{"c,cs,cpp,html,java,js,json,jsonl,md,php,py,rb,sh,ts,tex,txt": 25000, "gif,jpeg,jpg,png":500000}	The dictionary of lists of extensions of files that can be directly used in the context of a completion request and their associated maximum file sizes.
FoundationaLLM:DataPipeline:State:CosmosDB:Containers	DataPipelines	The names of the Azure Cosmos DB containers used by the Data Pipeline State service.
FoundationaLLM:DataPipeline:State:CosmosDB:Database	database	The Azure Cosmos DB database name used by the Data Pipeline State service.
FoundationaLLM:DataPipeline:State:CosmosDB:Endpoint	<cosmos_db_endpoint>	The endpoint URL of the Azure Cosmos DB used by the Data Pipeline State service.
FoundationaLLM:DataPipeline:State:Storage:AccountName	<storage_account_name>	The name of the dedicated storage account used by the FoundationaLLM Data Pipeline State service. This must be the same storage account as the one used by the FoundationaLLM Context API.
FoundationaLLM:DataPipeline:State:Storage:AuthenticationType	AzureIdentity	The type of authentication used by the FoundationaLLM Data Pipeline State service to connect to the dedicated storage account.
FoundationaLLM:APIEndpoints:DataPipelineAPI:Configuration:Storage:AccountName	<storage_account_name>	The name of the dedicated storage account used by the FoundationaLLM Data Pipeline API.
FoundationaLLM:APIEndpoints:DataPipelineAPI:Configuration:Storage:AuthenticationType	AzureIdentity	The type of authentication used by the FoundationaLLM Data Pipeline API to connect to the dedicated storage account.
FoundationaLLM:APIEndpoints:DataPipelineAPI:Configuration:FrontendWorkerQueue	frontend-worker	The queue used to submit data pipeline work items for the Data Pipeline Frontend Worker service.
FoundationaLLM:APIEndpoints:DataPipelineAPI:Configuration:BackendWorkerQueue	backend-worker	The queue used to submit data pipeline work items for the Data Pipeline Backend Worker service.
FoundationaLLM:APIEndpoints:DataPipelineFrontendWorker:Configuration:Storage:AccountName	<storage_account_name>	The name of the dedicated storage account used by the FoundationaLLM Data Pipeline Frontend Worker service.
FoundationaLLM:APIEndpoints:DataPipelineFrontendWorker:Configuration:Storage:AuthenticationType	AzureIdentity	The type of authentication used by the FoundationaLLM Data Pipeline Frontend Worker service to connect to the dedicated storage account.
FoundationaLLM:APIEndpoints:DataPipelineFrontendWorker:Configuration:Queue	frontend-worker	The queue used to process data pipeline work items by the Data Pipeline Frontend Worker service.
FoundationaLLM:APIEndpoints:DataPipelineFrontendWorker:Configuration:ParallelProcessorsCount	10	The number of parallel processors of data pipeline run work items used by the Data Pipeline Frontend Worker service.
FoundationaLLM:APIEndpoints:DataPipelineBackendWorker:Configuration:Storage:AccountName	<storage_account_name>	The name of the dedicated storage account used by the FoundationaLLM Data Pipeline Backend Worker service.
FoundationaLLM:APIEndpoints:DataPipelineBackendWorker:Configuration:Storage:AuthenticationType	AzureIdentity	The type of authentication used by the FoundationaLLM Data Pipeline Backend Worker service to connect to the dedicated storage account.
FoundationaLLM:APIEndpoints:DataPipelineBackendWorker:Configuration:Queue	frontend-worker	The queue used to process data pipeline work items by the Data Pipeline Backend Worker service.
FoundationaLLM:APIEndpoints:DataPipelineBackendWorker:Configuration:ParallelProcessorsCount	10	The number of parallel processors of data pipeline run work items used by the Data Pipeline Backend Worker service.
FoundationaLLM:Events:Profiles:ContextAPI	{"EventProcessingCycleSeconds":60,"Topics":[{"Name": "resource-providers", "SubscriptionPrefix": "rp-context"}]}	The event processing settings for the Context API.
FoundationaLLM:Events:Profiles:DataPipelineAPI	{"EventProcessingCycleSeconds":60,"Topics":[]}	The event processing settings for the Data Pipeline API.
FoundationaLLM:Events:Profiles:DataPipelineFrontendWorker	{"EventProcessingCycleSeconds":60,"Topics":[]}	The event processing settings for the Data Pipeline Frontend Worker service.
FoundationaLLM:Events:Profiles:DataPipelineBackendWorker	{"EventProcessingCycleSeconds":60,"Topics":[]}	The event processing settings for the Data Pipeline Backend Worker service.
FoundationaLLM:ResourceProviders:Vector:Storage:AccountName	<storage_account_name>	The name of the storage account used by the FoundationaLLM Vector resource provider.
FoundationaLLM:ResourceProviders:Vector:Storage:AuthenticationType	AzureIdentity	The type of authentication used by the FoundationaLLM Vector resource provider to connect to the storage account.
FoundationaLLM:ResourceProviders:Plugin:Storage:AccountName	<storage_account_name>	The name of the storage account used by the FoundationaLLM Plugin resource provider.
FoundationaLLM:ResourceProviders:Plugin:Storage:AuthenticationType	AzureIdentity	The type of authentication used by the FoundationaLLM Plugin resource provider to connect to the storage account.

Important

A new Azure Cosmos DB container named DataPipelines must be created with a parition key of /run_id and an autoscale transactional throughput of maximum 4000 RU/s.

Important

A new APIEndpointConfiguration artifact named DataPipelineAPI must be created and saved to a file named DataPipelineAPI.json. The file must be created in the FoundationaLLM.Configuration resource provider folder of the FoundationaLLM storage account. The file must contain the following content:

{
  "type": "api-endpoint",
  "name": "DataPipelineAPI",
  "object_id": "/instances/<instance_id>/providers/FoundationaLLM.Configuration/apiEndpointConfigurations/DataPipelineAPI",
  "display_name": null,
  "description": null,
  "cost_center": null,
  "category": "General",
  "subcategory": null,
  "authentication_type": "APIKey",
  "url": "<deployed_api_endpoint>",
  "status_endpoint": "/instances/<instance_id>/status",
  "url_exceptions": [],
  "authentication_parameters": {
    "api_key_configuration_name": "FoundationaLLM:APIEndpoints:DataPipelineAPI:Essentials:APIKey",
    "api_key_header_name": "X-API-KEY"
  },
  "timeout_seconds": 2400,
  "retry_strategy_name": "ExponentialBackoff",
  "provider": null,
  "api_version": null,
  "operation_type": null,
  "properties": null,
  "created_on": "0001-01-01T00:00:00-05:00",
  "updated_on": "2025-02-25T11:10:55.3458874-05:00",
  "created_by": null,
  "updated_by": "ciprian@foundationaLLM.ai",
  "deleted": false,
  "expiration_date": null
}

where <deployed_api_endpoint> is the endpoint of the Data Pipeline API.

Also, a new entry must be added to the _resource-references.json file from the FoundationaLLM.Configuration resource provider folder with the following content:

{
    "Name": "DataPipelineAPI",
    "Filename": "/FoundationaLLM.Configuration/DataPipelineAPI.json",
    "Type": "api-endpoint",
    "Deleted": false
}

Important

Two new storage queues must be created in the FoundationaLLM storage account:

• frontend-worker - used by the Data Pipeline Frontend Worker to process requests.
• backend-worker - used by the Data Pipeline Backend Worker to process requests.

The previous queues extract, partition, embed, and index are now deprecated. For each existing FoundationaLLM deployment, they should be removed as soon as it is confirmed that the previous version of vectorization process is no longer used.

Important

A new container with the name of the FoundationaLLM instance identifier must be created in the FoundationaLLM main storage account. The container will be used by the Data Pipeline API and the Data Pipeline Frontend/Backend Worker to store data pipeline work items.

Role assignment changes

The following role assignments must be added to the Management API's Managed Identity:

Name	Type	Target
Data Pipelines Execution Manager	FoundationaLLM	FoundationaLLM instance.

The following role assignments must be added to the Context API's Managed Identity:

Name	Type	Target
Reader	FoundationaLLM	FoundationaLLM instance.
App Configuration Data Reader	Azure	Azure App Configuration service.
Key Vault Secrets User	Azure	Azure Key Vault service.
Storage Blob Data Contributor	Azure	Storage account used by the FoundationaLLM Context API.
Storage Blob Data Contributor	Azure	Main FoundationaLLM storage account.
Azure ContainerApps Session Executor	Azure	Azure Container Apps Dynamic Sessions used by the FoundationaLLM Context API.
Azure ContainerApps Session Executor	Azure	Azure Container Apps Custom Container used by the FoundationaLLM Context API.
EventGrid Contributor	Azure	Azure EventGrid Namespace used by the FoundationaLLM Data Pipeline API.
Search Index Data Reader	Azure	Azure AI Search service used by the FoundationaLLM instance.

The following role assignments must be added to the Orchestrations API's Managed Identity:

Name	Type	Target
Data Pipelines Execution Manager	FoundationaLLM	FoundationaLLM instance.

The following role assignments must be added to the Data Pipeline API's Managed Identity:

Name	Type	Target
App Configuration Data Reader	Azure	Azure App Configuration service.
Key Vault Secrets User	Azure	Azure Key Vault service.
Key Vault Certificate User	Azure	Azure Key Vault service.
Storage Blob Data Contributor	Azure	Main FoundationaLLM storage account.
Storage Blob Data Contributor	Azure	Storage account used by the FoundationaLLM Context API.
Cosmos DB Built-in Data Contributor	Azure	Azure Cosmos DB account used by the FoundationaLLM Data Pipeline API.
Storage Queue Data Message Sender	Azure	Storage account used by the FoundationaLLM Data Pipeline API.
EventGrid Contributor	Azure	Azure EventGrid Namespace used by the FoundationaLLM Data Pipeline API.
Data Pipelines Execution Manager	FoundationaLLM	FoundationaLLM instance.
Reader	FoundationaLLM	FoundationaLLM instance.

The following role assignments must be added to the Data Pipeline Frontend Worker's Managed Identity:

Name	Type	Target
App Configuration Data Reader	Azure	Azure App Configuration service.
Key Vault Secrets User	Azure	Azure Key Vault service.
Key Vault Certificate User	Azure	Azure Key Vault service.
Storage Blob Data Contributor	Azure	Main FoundationaLLM storage account.
Storage Blob Data Contributor	Azure	Storage account used by the FoundationaLLM Context API.
Cosmos DB Built-in Data Contributor	Azure	Azure Cosmos DB account used by the FoundationaLLM Data Pipeline Frontend Worker.
Storage Queue Data Contributor	Azure	Storage account used by the FoundationaLLM Data Pipeline Frontend Worker.
EventGrid Contributor	Azure	Azure EventGrid Namespace used by the FoundationaLLM Data Pipeline Frontend Worker.
Search Service Contributor	Azure	Azure AI Search service used by the FoundationaLLM instance.
Search Index Data Contributor	Azure	Azure AI Search service used by the FoundationaLLM instance.
Data Pipelines Execution Manager	FoundationaLLM	FoundationaLLM instance.
Reader	FoundationaLLM	FoundationaLLM instance.

The following role assignments must be added to the Data Pipeline Backend Worker's Managed Identity:

Name	Type	Target
App Configuration Data Reader	Azure	Azure App Configuration service.
Key Vault Secrets User	Azure	Azure Key Vault service.
Key Vault Certificate User	Azure	Azure Key Vault service.
Storage Blob Data Contributor	Azure	Main FoundationaLLM storage account.
Storage Blob Data Contributor	Azure	Storage account used by the FoundationaLLM Context API.
Cosmos DB Built-in Data Contributor	Azure	Azure Cosmos DB account used by the FoundationaLLM Data Pipeline Backend Worker.
Storage Queue Data Contributor	Azure	Storage account used by the FoundationaLLM Data Pipeline Backend Worker.
EventGrid Contributor	Azure	Azure EventGrid Namespace used by the FoundationaLLM Data Pipeline Backend Worker.
Search Service Contributor	Azure	Azure AI Search service used by the FoundationaLLM instance.
Search Index Data Contributor	Azure	Azure AI Search service used by the FoundationaLLM instance.
Data Pipelines Execution Manager	FoundationaLLM	FoundationaLLM instance.
Reader	FoundationaLLM	FoundationaLLM instance.

New resource providers

FoundationaLLM.Vector

The FoundationaLLM.Vector resource provider has been added. A new folder named FoundationaLLM.Vector must be created in the FoundationaLLM storage account under the resource-provider container. The folder must contain the following files:

• _resource-references.json - the resource references file.
• ConversationFiles.json - the resource file for the ConversationFiles vector database resource.

The content of the resource-references.json file is as follows:

{
	"ResourceReferences": [
		{
			"Name": "ConversationFiles",
			"Filename": "/FoundationaLLM.Vector/ConversationFiles.json",
			"Type": "vector-database",
			"Deleted": false
		}
	],
	"DefaultResourceName": null
}

The content of the ConversationFiles.json file is as follows:

{
  "type": "vector-database",
  "name": "ConversationFiles",
  "object_id": "/instances/{instanceId}/providers/FoundationaLLM.Vector/vectorDatabases/ConversationFiles",
  "display_name": "Conversation files vectorization",
  "description": "The vector database that used by default when processing files that are uploaded in conversations.",
  "cost_center": null,
  "database_type": "AzureAISearch",
  "database_name": "conversation-files",
  "embedding_property_name": "Embedding",
  "content_property_name": "Text",
  "vector_store_id_property_name": "VectorStoreId",
  "metadata_property_name": "Metadata",
  "api_endpoint_configuration_object_id": "/instances/{instanceId}/providers/FoundationaLLM.Configuration/apiEndpointConfigurations/AzureAISearch",
  "properties": null,
  "created_on": "2025-05-20T13:57:58.9770592+00:00",
  "updated_on": "0001-01-01T00:00:00+00:00",
  "created_by": "SYSTEM",
  "updated_by": null,
  "deleted": false,
  "expiration_date": null
}

Important

The configuration entries for the FoundationaLLM.Vector resource provider are specified above, in the Configuration changes section.

FoundationaLLM.Plugin

The FoundationaLLM.Plugin resource provider has been added. A new folder named FoundationaLLM.Plugin must be created in the FoundationaLLM storage account under the resource-provider container.

Important

The configuration entries for the FoundationaLLM.Plugin resource provider are specified above, in the Configuration changes section.

Starting from 0.9.7-beta147

The following App Configuration value has been added:

Name	Default value	Description
FoundationaLLM:ResourceProviders:AzureAI:Storage:AccountName	The name of the deployment storage account.	The name of the Azure Blob Storage account used by the FoundationaLLM.AzureAI resource provider.
FoundationaLLM:ResourceProviders:AzureAI:Storage:AuthenticationType	AzureIdentity	The authentication type used by the FoundationaLLM.AzureAI resource provider to connect to the storage account.

Policy assignment changes

The FLLM-Users will need an entry in the policy-assignments file for agentConversationMappings and agentFileMappings resources in the FoundationaLLM.AzureAI resource provider.

{
    "name": "87e6fd6e-ddd3-4054-b382-d7f0fa190aa9",
    "type": "FoundationaLLM.Authorization/policyAssignments",
    "object_id": "/providers/FoundationaLLM.Authorization/policyAssignments/87e6fd6e-ddd3-4054-b382-d7f0fa190aa9",
    "description": "Ownership on agent conversation mapping resources managed by the FoundationaLLM.AzureAI resource provider.",
    "policy_definition_id": "/providers/FoundationaLLM.Authorization/policyDefinitions/00000000-0000-0000-0001-000000000001",
    "principal_id": "c54871ba-1fa1-439a-9e86-30d74dfe4a4a",
    "principal_type": "Group",
    "scope": "/instances/{instanceId}/providers/FoundationaLLM.AzureAI/agentConversationMappings",
    "created_on": "2025-01-14T17:58:19.5954014Z",
    "updated_on": "2025-01-14T17:58:19.5954014Z",
    "created_by": "SYSTEM",
    "updated_by": "SYSTEM"
},
{
    "name": "0e7684d2-e8b9-40ba-88b4-3f358f93afa3",
    "type": "FoundationaLLM.Authorization/policyAssignments",
    "object_id": "/providers/FoundationaLLM.Authorization/policyAssignments/0e7684d2-e8b9-40ba-88b4-3f358f93afa3",
    "description": "Ownership on agent file mapping resources managed by the FoundationaLLM.AzureAI resource provider.",
    "policy_definition_id": "/providers/FoundationaLLM.Authorization/policyDefinitions/00000000-0000-0000-0001-000000000001",
    "principal_id": "c54871ba-1fa1-439a-9e86-30d74dfe4a4a",
    "principal_type": "Group",
    "scope": "/instances/{instanceId}/providers/FoundationaLLM.AzureAI/agentFileMappings",
    "created_on": "2025-01-14T17:58:19.5954014Z",
    "updated_on": "2025-01-14T17:58:19.5954014Z",
    "created_by": "SYSTEM",
    "updated_by": "SYSTEM"
}

Starting from 0.9.7-beta139

Configuration changes

The following App Configuration value have been added:

Name	Default value	Description
FoundationaLLM:APIEndpoints:GatewayAPI:Configuration:AzureAIAgentServiceMaxVectorizationTimeSeconds	120	The maximum time in seconds allowed for an Azure AI Agent Service vectorization process to complete.
FoundationaLLM:APIEndpoints:CoreAPI:Configuration:AzureAIAgentsFileSearchExensions	c, cpp, cs, css, doc, docx, html, java, js, json, md, pdf, php, pptx, py, rb, sh, tex, ts, txt	The comma-separated list file extensions that are supported by the Azure AI Agent Service file search tool.

The following App Configuration value have been removed as they are no longer needed:

1. ResourceProviders:AzureOpenAI:Storage:AuthenticationType
2. ResourceProviders:AzureOpenAI:Storage:AccountName

A new workflow resource must be added to support the AzureAIAgentServiceWorkflow. Ensure the reference is added to _resource-references.json as well.

{
    "type": "azure-ai-agent-service-workflow",
    "name": "AzureAIAgentServiceWorkflow",
    "object_id": "/instances/{instanceId}/providers/FoundationaLLM.Agent/workflows/AzureAIAgentServiceWorkflow",
    "display_name": "AzureAIAgentServiceWorkflow",
    "description": "Azure AI Agent Service Workflow",
    "cost_center": null,
    "properties": null,
    "created_on": "2024-11-08T10:08:27.1953263+00:00",
    "updated_on": "0001-01-01T00:00:00+00:00",
    "created_by": "SYSTEM",
    "updated_by": null,
    "deleted": false,
    "expiration_date": null
}

Starting from 0.9.7-beta138

Configuration changes

The FoundationaLLM:Code:CodeExecution:AzureContainerAppsDynamicSessions configuration setting has been replaced by the following configuration settings:

Name	Content Type	Description	Default value
FoundationaLLM:Code:CodeExecution:AzureContainerAppsDynamicSessions:CodeInterpreter	application/json	The settings for the Azure Container Apps Dynamic Sessions code interpreter.	{ "Endpoints": { "Python": [], "CSharp": []} }
FoundationaLLM:Code:CodeExecution:AzureContainerAppsDynamicSessions:CustomContainer	application/json	The settings for the Azure Container Apps Dynamic Sessions custom container.	{ "Endpoints": { "Python": [], "CSharp": []} }

FoundationaLLM now supports Azure AI Inference APIs accessible to the agent using the LangChain workflow host. This functionality expects an Azure AI Service resource. Authentication is accomplished support is via Azure Identity (Entra) or API Key. If using Azure Identity, ensure the LangChain managed identity has Cognitive Services User role on the Azure AI Service resource (may be inherited). Within the project defined in AI Foundry, ensure the LangChain managed identity has the Azure AI Developer role. If using API Key, ensure the API Key is securely stored in the key vault and exposed via an Application Configuration value. When defining the API Endpoint Configuration, add the authentication parameter api_key_configuration_name with the name of the Application Configuration key.

Code sessions

The tool property foundationallm_aca_code_execution_enabled has been renamed to code_session_required. The property indicates whether the tool requires a code session during its execution. If the property is set to true, the following additional properties must be set:

• code_session_endpoint_provider: Supported values are AzureContainerAppsCodeInterpreter (indicates that the code session is provided by the Azure Container Apps Dynamic Sessions code interpreter) and AzureContainerAppsCustomContainer (indicates that the code session is provided by the Azure Container Apps Dynamic Sessions custom container).
• code_session_language: Supported values are Python and CSharp.

Tools that require a code session should expect the following properties to be injected:

• code_session_endpoint (instead of the previous foundationallm_aca_code_execution_session_id)
• code_session_id (instead of the previous foundationallm_aca_code_execution_session_id)

Starting from 0.9.7-beta128

Configuration changes

Agent Workflow configuration now has a class_name field. This is the underlying implementation class of the workflow. This is not a breaking change, as in previous versions the name field contained the class name and if the class_name field is empty, it will default to the name field. However, it is recommended to set the class_name field to the implementation class name.

Starting from 0.9.7-beta112

Configuration changes

Added the following App Configuration value:

Name	Default value	Description
FoundationaLLM:APIEndpoints:DataPipelineAPI:Essentials:APIKey	Points to foundationallm-apiendpoints-datapipelineapi-apikey	KeyVault reference value pointing to the specified secret.
FoundationaLLM:APIEndpoints:DataPipelineAPI:Essentials:AppInsightsConnectionString	Points to foundationallm-appinsights-connectionstring	KeyVault reference value pointing to the specified secret.
FoundationaLLM:APIEndpoints:DataPipelineFrontendWorker:Essentials:APIKey	Points to foundationallm-apiendpoints-datapipelinefrontendworker-apikey	KeyVault reference value pointing to the specified secret.
FoundationaLLM:APIEndpoints:DataPipelineFrontendWorker:Essentials:AppInsightsConnectionString	Points to foundationallm-appinsights-connectionstring	KeyVault reference value pointing to the specified secret.
FoundationaLLM:APIEndpoints:DataPipelineBackendWorker:Essentials:APIKey	Points to foundationallm-apiendpoints-datapipelinebackendworker-apikey	KeyVault reference value pointing to the specified secret.
FoundationaLLM:APIEndpoints:DataPipelineBackendWorker:Essentials:AppInsightsConnectionString	Points to foundationallm-appinsights-connectionstring	KeyVault reference value pointing to the specified secret.
FoundationaLLM:APIEndpoints:ContextAPI:Essentials:APIKey	Points to foundationallm-apiendpoints-contextapi-apikey	KeyVault reference value pointing to the specified secret.
FoundationaLLM:APIEndpoints:ContextAPI:Essentials:AppInsightsConnectionString	Points to foundationallm-appinsights-connectionstring	KeyVault reference value pointing to the specified secret.
FoundationaLLM:APIEndpoints:ContextAPI:Configuration:FileService:Storage:AccountName	<context_api_file_storage_account_name>	The name of the dedicated storage account used by the FoundationaLLM Context API file service.
FoundationaLLM:APIEndpoints:ContextAPI:Configuration:FileService:Storage:AuthenticationType	AzureIdentity	The type of authentication used by the FoundationaLLM Context API file service to connect to the dedicated storage account.
FoundationaLLM:APIEndpoints:ContextAPI:Configuration:FileService:CosmosDB:Endpoint	<cosmos_db_endpoint>	The endpoint URL of the Azure Cosmos DB.
FoundationaLLM:APIEndpoints:ContextAPI:Configuration:FileService:CosmosDB:Database	database	The Azure Cosmos DB database name.
FoundationaLLM:APIEndpoints:ContextAPI:Configuration:FileService:CosmosDB:Containers	Context	The list of Azure CosmosDB containers used by the Context API file service.

Important

A dedicated storage account for the FoundationaLLM Context API must be created. The account should be isolated in a dedicated resource group and permissions should only be granted to the FoundationaLLM Context API managed identity. The account should be configured the same way the main FoundationaLLM storage account is. A new storage container with a name equal to the FoundationaLLM instance identifier must be created in the storage account.

Important

A new Azure Cosmos DB container named Context must be created with a parition key of /upn and an autoscale transactional throughput of maximum 4000 RU/s.

Important

A new Azure Container Apps Session Pool must be created with in the same resource groups as the Context API storage account. Network egress should be disabled. The FoundationaLLM:Code:CodeExecution:AzureContainerAppsDynamicSessions must be set to the following value:

{"DynamicSessionsEndpoints": [
   "<session_pool_endpoint>"
]}

where <session_pool_endpoint> is the endpoint of the Azure Container Apps Session Pool.

Role assignment changes

The following role assignments must be added to the Context API's Managed Identity:

Name	Target
App Configuration Data Reader	Azure App Configuration service.
Key Vault Secrets User	Azure Key Vault service.
Storage Blob Data Contributor	Dedicated storage account used by the FoundationaLLM Context API.
Cosmos DB Built-in Data Contributor	Azure Cosmos DB account used by the FoundationaLLM Context API.
Azure ContainerApps Session Executor	Azure Container Apps Session Pool.

The following role assignments must be added to the LangChain API's Managed Identity:

Name	Target
Azure ContainerApps Session Executor	Azure Container Apps Session Pool.

The following role assignments must be added to the Semantic Kernel API's Managed Identity:

Name	Target
Azure ContainerApps Session Executor	Azure Container Apps Session Pool.

Starting from 0.9.7-beta107

For External Agent Workflows, the workflow invokation now includes conversation file history. The complete file history is passed regardless of conversation history settings.

The signature is now as follows:

async def invoke_async(self,
                       operation_id: str,
                       user_prompt: str,
                       user_prompt_rewrite: Optional[str],
                       message_history: List[MessageHistoryItem],
                       file_history: List[FileHistoryItem])-> CompletionResponse:

Starting from 0.9.7-beta106

For External Agent Workflows, the invocation of the workflow now sends the completion request MessageHistoryItem list vs the translated LangChain BaseMessage construct. This way workflows have more control on how to handle the message history of a conversation.

Starting from 0.9.7-beta105

Configuration changes

The FoundationaLLM.Common.Models.ResourceProviders.Agent.AgentTool definition now contains a ClassName property. This should be set to the implementation class name of the tool. The Name property of the tool is what will be used for uniqueness in the case multipe tools of the same type/class are used in the same agent.

Note: This is not a breaking change as if the ClassName property is not set, the Name property will be used as the class name which is the existing pattern. However, it is recommended to set the ClassName property to adhere to the new pattern.

Agent configuration tools property is impacted as follows:

Old pattern sample:

{
      "name": "DALLEImageGeneration",
      "description": "Generates an image based on a prompt.",
      "package_name": "FoundationaLLM",
      "resource_object_ids": {
        "/instances/8ac6074c-bdde-43cb-a140-ec0002d96d2b/providers/FoundationaLLM.AIModel/aiModels/DALLE3Model": {
          "object_id": "/instances/8ac6074c-bdde-43cb-a140-ec0002d96d2b/providers/FoundationaLLM.AIModel/aiModels/DALLE3Model",
          "properties": {
            "object_role": "main_model",
            "model_parameters": {}
          }
        }
      },
      "properties": {}
    }

New pattern:

{
      "name": "DALLETool1",
      "description": "Generates an image based on a prompt.",
      "package_name": "FoundationaLLM",
      "class_name": "DALLEImageGenerationTool",
      "resource_object_ids": {
        "/instances/8ac6074c-bdde-43cb-a140-ec0002d96d2b/providers/FoundationaLLM.AIModel/aiModels/DALLE3Model": {
          "object_id": "/instances/8ac6074c-bdde-43cb-a140-ec0002d96d2b/providers/FoundationaLLM.AIModel/aiModels/DALLE3Model",
          "properties": {
            "object_role": "main_model",
            "model_parameters": {}
          }
        }
      },
      "properties": {}
    }

Starting from 0.9.7-beta103

Configuration changes

Added the following App Configuration value:

Name	Default value	Description
FoundationaLLM:Quota:Storage:AccountName	<storage_account_name>	Provides the storage account used by the FoundationaLLM quota management service.
FoundationaLLM:Quota:Storage:AuthenticationType	AzureIdentity	Indicates the authentication type used by the FoundationaLLM quota management service to connect to the storage account.

A new container named quota must be created in the default storage account.

The quota definitions are stored in the quota container in a file named quota-store.json. If the file does not exist, the file is automatically created. The file contains a list of quota definitions with the following structure:

{
	"name": "TestAPI01CompletionsUPNRawRequestRateLimit",
	"description": "Defines a per UPN raw request rate limit on the TestAPI01 Completions controller.",
	"context": "TestAPI01:Completions",
	"type": "RawRequestRateLimit",
	"metric_partition": "UserPrincipalName",
	"metric_limit": 120,
	"metric_window_seconds": 60,
	"lockout_duration_seconds": 60,
	"distributed_enforcement": false
}

The following table provides details about the quota definition properties:

Name	Description	Notes
name	The name of the quota definition.
description	A description of the quota definition.
context	The context of the quota definition.	The format of the context is <service_name>:<controller_name> or <service_name>:<controller_name>:<agent_name>. Currently the following contexts can be used: CoreAPI:Completions, CoreAPI:Completions:<agent_name> where <agent_name> must be a valid agent name.
type	The type of the quota enforcement applied.	The following types are supported: RawRequestRateLimit and AgentRequestRateLimit. RawRequestRateLimt defines the quota metric to be raw API requests and requires a context of <service_name>:<controller_name>. AgentRequestRateLimit defines the quota metric to be agent completion requests and requires a context of <service_name>:<controller_name>:<agent_name>.
metric_partition	The metric partition used to enforce the quota.	The following partitions are supported: None (the metric is not partitioned) UserPrincipalName (the metric is partitioned by user principal name) and UserIdentifier (the metric is partitioned by user identifier).
metric_limit	The limit of the metric.	The limit is enforced over the metric_window_seconds. In the example above, a maximum number of 120 raw API requests are allowed per user principal name in a 60-second window.
metric_window_seconds	The time window in seconds over which the limit is enforced.	In the example above, a maximum number of 120 raw API requests are allowed per user principal name in a 60-second window.
lockout_duration_seconds	The duration in seconds for which the caller is locked out after exceeding the quota.	The lockout duration is applied after the user exceeds the quota limit. The user is locked out for the specified duration before the quota is reset.
distributed_enforcement	Indicates whether the quota is enforced across multiple instances of the same API.	If true, the quota is enforced across multiple instances. If false, the quota is enforced on a single instance. Currently, only false is supported.

Starting from 0.9.7-beta101

Configuration changes

Added the following App Configuration value:

Name	Default value	Description
FoundationaLLM:ResourceProviders:DataPipeline:Storage:AccountName	<storage_account_name>	Provides the storage account used by the FoundationaLLM.DataPipeline resource provider.
FoundationaLLM:ResourceProviders:DataPipeline:Storage:AuthenticationType	AzureIdentity	Indicates the authentication type used by the FoundationaLLM.DataPipeline resource provider to connect to the storage account.
FoundationaLLM:ResourceProviders:Plugin:Storage:AccountName	<storage_account_name>	Provides the storage account used by the FoundationaLLM.Plugin resource provider.
FoundationaLLM:ResourceProviders:Plugin:Storage:AuthenticationType	AzureIdentity	Indicates the authentication type used by the FoundationaLLM.Plugin resource provider to connect to the storage account.

Management API

The POST /instances/{instanceId}/providers/{resourceProvider}/{resourcePath} endpoint now supports providing a resource form data key in the request body. This key is used to provide the serialized resource FoundationaLLM resource when a file upload is performed using a form-data request body.

The call to POST /instances/{instanceId}/providers/FoundationaLLM.Plugin/pluginPackages/Dotnet-FoundationaLLMDataPipelinePlugins expects a form-data request body with the following keys:

• file (of type file) - the plugin package file to upload.
• resource (of type text) - the serialized plugin package FoundationaLLM resource.

Starting with 0.9.7-beta100

Configuration changes

App configuration settings

The value of the FoundationaLLM:Events:Profiles:CoreAPI must be updated to include the api-statistics topic. The updated value is as follows:

{
    "EventProcessingCycleSeconds": 5,
    "Topics": [
        {
            "Name": "resource-providers",
            "SubscriptionPrefix": "rp-core"
        },
        {
            "Name": "api-statistics",
            "SubscriptionPrefix": "as-core"
        }
    ]
}

Starting with 0.9.4-rc100

Configuration changes

App configuration settings

Important

The App Config setting FoundationaLLM:Instance:EnableResourceProvidersCache is obsolete and should be removed from the App Config settings.

The following App Config properties make cache settings for the resource providers configurable:

Name	Description	Default Value
FoundationaLLM:ResourceProvidersCache:EnableCache	Indicates whether resource providers should cache resources or not.	true
FoundationaLLM:ResourceProvidersCache:AbsoluteCacheExpirationSeconds	Absolute cache expiration in seconds.	300
FoundationaLLM:ResourceProvidersCache:SlidingCacheExpirationSeconds	Sets how many seconds the cache entry can be inactive (e.g. not accessed) before it will be removed. This will not extend the entry lifetime beyond the absolute expiration (if set).	120
FoundationaLLM:ResourceProvidersCache:CacheSizeLimit	The maximum number of items that can be stored in the cache.	10000
FoundationaLLM:ResourceProvidersCache:CacheExpirationScanFrequencySeconds	Gets or sets the minimum length of time between successive scans for expired items in seconds.	30

Starting with 0.9.3

This version introduces the concept of a well-known virtual security group (AllAgentsVirtualSecurityGroup) that is used by agents using Agent Access Token authentication and have their own virtual security group defined. Assign the following PBAC and RBAC roles to the AllAgentsVirtualSecurityGroup (replace the tokens denoted by {{...}} with the actual values):

PBAC changes

{
    "name": "{{pbacConversationsOwnerGuid}}",
    "type": "FoundationaLLM.Authorization/policyAssignments",
    "object_id": "/providers/FoundationaLLM.Authorization/policyAssignments/{{pbacConversationsOwnerGuid}}",
    "description": "Ownership on conversation resources for AllAgentsVirtualSecurityGroup by the FoundationaLLM.Conversation resource provider.",
    "policy_definition_id": "/providers/FoundationaLLM.Authorization/policyDefinitions/00000000-0000-0000-0001-000000000001",
    "principal_id": "5bb493a2-5909-4771-93ba-d83b7b5a1de9",
    "principal_type": "Group",
    "scope": "/instances/{{instanceId}}/providers/FoundationaLLM.Conversation/conversations",
    "created_on": "{{deployTime}}",
    "updated_on": "{{deployTime}}",
    "created_by": "SYSTEM",
    "updated_by": "SYSTEM"
},
{
    "name": "{{pbacConversationMappingsGuid}}",
    "type": "FoundationaLLM.Authorization/policyAssignments",
    "object_id": "/providers/FoundationaLLM.Authorization/policyAssignments/{{pbacConversationMappingsGuid}}",
    "description": "Ownership on conversation mapping resources for AllAgentsVirtualSecurityGroup managed by the FoundationaLLM.AzureOpenAI resource provider.",
    "policy_definition_id": "/providers/FoundationaLLM.Authorization/policyDefinitions/00000000-0000-0000-0001-000000000001",
    "principal_id": "5bb493a2-5909-4771-93ba-d83b7b5a1de9",
    "principal_type": "Group",
    "scope": "/instances/{{instanceId}}/providers/FoundationaLLM.AzureOpenAI/conversationMappings",
    "created_on": "{{deployTime}}",
    "updated_on": "{{deployTime}}",
    "created_by": "SYSTEM",
    "updated_by": "SYSTEM"
},
{
    "name": "{{pbacAttachmentsOwnerGuid}}",
    "type": "FoundationaLLM.Authorization/policyAssignments",
    "object_id": "/providers/FoundationaLLM.Authorization/policyAssignments/{{pbacAttachmentsOwnerGuid}}",
    "description": "Ownership on attachment resources for AllAgentsVirtualSecurityGroup managed by the FoundationaLLM.Attachment resource provider.",
    "policy_definition_id": "/providers/FoundationaLLM.Authorization/policyDefinitions/00000000-0000-0000-0001-000000000001",
    "principal_id": "5bb493a2-5909-4771-93ba-d83b7b5a1de9",
    "principal_type": "Group",
    "scope": "/instances/{{instanceId}}/providers/FoundationaLLM.Attachment/attachments",
    "created_on": "{{deployTime}}",
    "updated_on": "{{deployTime}}",
    "created_by": "SYSTEM",
    "updated_by": "SYSTEM"
},
{
    "name": "{{pbacFileMappingsGuid}}",
    "type": "FoundationaLLM.Authorization/policyAssignments",
    "object_id": "/providers/FoundationaLLM.Authorization/policyAssignments/{{pbacFileMappingsGuid}}",
    "description": "Ownership on file mapping resources for AllAgentsVirtualSecurityGroup managed by the FoundationaLLM.AzureOpenAI resource provider.",
    "policy_definition_id": "/providers/FoundationaLLM.Authorization/policyDefinitions/00000000-0000-0000-0001-000000000001",
    "principal_id": "5bb493a2-5909-4771-93ba-d83b7b5a1de9",
    "principal_type": "Group",
    "scope": "/instances/{{instanceId}}/providers/FoundationaLLM.AzureOpenAI/fileMappings",
    "created_on": "{{deployTime}}",
    "updated_on": "{{deployTime}}",
    "created_by": "SYSTEM",
    "updated_by": "SYSTEM"
}

RBAC changes

{
    "type": "FoundationaLLM.Authorization/roleAssignments",
    "name": "{{openAiAssistantsReaderGuid}}",
    "object_id": "/providers/FoundationaLLM.Authorization/roleAssignments/{{openAiAssistantsReaderGuid}}",
    "display_name": null,
    "description": "Read Access for OpenAIAssistants for the AllAgentsVirtualSecurityGroup group.",
    "cost_center": null,
    "role_definition_id": "/providers/FoundationaLLM.Authorization/roleDefinitions/00a53e72-f66e-4c03-8f81-7e885fd2eb35",
    "principal_id": "5bb493a2-5909-4771-93ba-d83b7b5a1de9",
    "principal_type": "Group",
    "scope": "/instances/{{instanceId}}/providers/FoundationaLLM.Agent/workflows/OpenAIAssistants",
    "properties": null,
    "created_on": "{{deployTime}}",
    "updated_on": "{{deployTime}}",
    "created_by": null,
    "updated_by": null,
    "deleted": false,
    "expiration_date": null
},
{
    "type": "FoundationaLLM.Authorization/roleAssignments",
    "name": "{{langGraphReactAgentReaderGuid}}",
    "object_id": "/providers/FoundationaLLM.Authorization/roleAssignments/{{langGraphReactAgentReaderGuid}}",
    "display_name": null,
    "description": "Read Access for LangGraphReactAgent for the AllAgentsVirtualSecurityGroup group.",
    "cost_center": null,
    "role_definition_id": "/providers/FoundationaLLM.Authorization/roleDefinitions/00a53e72-f66e-4c03-8f81-7e885fd2eb35",
    "principal_id": "5bb493a2-5909-4771-93ba-d83b7b5a1de9",
    "principal_type": "Group",
    "scope": "/instances/{{instanceId}}/providers/FoundationaLLM.Agent/workflows/LangGraphReactAgent",
    "properties": null,
    "created_on": "{{deployTime}}",
    "updated_on": "{{deployTime}}",
    "created_by": null,
    "updated_by": null,
    "deleted": false,
    "expiration_date": null
},
{
    "type": "FoundationaLLM.Authorization/roleAssignments",
    "name": "{{attachmentContributorGuid2}}",
    "object_id": "/providers/FoundationaLLM.Authorization/roleAssignments/{{attachmentContributorGuid2}}",
    "display_name": null,
    "description": "Attachment contributor role for AllAgentsVirtualSecurityGroup group.",
    "cost_center": null,
    "role_definition_id": "/providers/FoundationaLLM.Authorization/roleDefinitions/8e77fb6a-7a78-43e1-b628-d9e2285fe25a",
    "principal_id": "5bb493a2-5909-4771-93ba-d83b7b5a1de9",
    "principal_type": "Group",
    "scope": "/instances/{{instanceId}}",
    "properties": null,
    "created_on": "{{deployTime}}",
    "updated_on": "{{deployTime}}",
    "created_by": null,
    "updated_by": null,
    "deleted": false,
    "expiration_date": null
},
{
    "type": "FoundationaLLM.Authorization/roleAssignments",
    "name": "{{conversationContributorGuid2}}",
    "object_id": "/providers/FoundationaLLM.Authorization/roleAssignments/{{conversationContributorGuid2}}",
    "display_name": null,
    "description": "Conversation contributor role for AllAgentsVirtualSecurityGroup group.",
    "cost_center": null,
    "role_definition_id": "/providers/FoundationaLLM.Authorization/roleDefinitions/d0d21b90-5317-499a-9208-3a6cb71b84f9",
    "principal_id": "5bb493a2-5909-4771-93ba-d83b7b5a1de9",
    "principal_type": "Group",
    "scope": "/instances/{{instanceId}}",
    "properties": null,
    "created_on": "{{deployTime}}",
    "updated_on": "{{deployTime}}",
    "created_by": null,
    "updated_by": null,
    "deleted": false,
    "expiration_date": null
},
{
    "type": "FoundationaLLM.Authorization/roleAssignments",
    "name": "{{configReadAccessGuid3}}",
    "object_id": "/providers/FoundationaLLM.Authorization/roleAssignments/{{configReadAccessGuid3}}",
    "display_name": null,
    "description": "Read Access for configuration for the AllAgentsVirtualSecurityGroup group.",
    "cost_center": null,
    "role_definition_id": "/providers/FoundationaLLM.Authorization/roleDefinitions/00a53e72-f66e-4c03-8f81-7e885fd2eb35",
    "principal_id": "5bb493a2-5909-4771-93ba-d83b7b5a1de9",
    "principal_type": "Group",
    "scope": "/instances/{{instanceId}}/providers/FoundationaLLM.Configuration/appConfigurations/FoundationaLLM:APIEndpoints:CoreAPI:Configuration:MaxUploadsPerMessage",
    "properties": null,
    "created_on": "{{deployTime}}",
    "updated_on": "{{deployTime}}",
    "created_by": null,
    "updated_by": null,
    "deleted": false,
    "expiration_date": null
},
{
    "type": "FoundationaLLM.Authorization/roleAssignments",
    "name": "{{configReadAccessGuid4}}",
    "object_id": "/providers/FoundationaLLM.Authorization/roleAssignments/{{configReadAccessGuid4}}",
    "display_name": null,
    "description": "Read Access for configuration for the AllAgentsVirtualSecurityGroup group.",
    "cost_center": null,
    "role_definition_id": "/providers/FoundationaLLM.Authorization/roleDefinitions/00a53e72-f66e-4c03-8f81-7e885fd2eb35",
    "principal_id": "5bb493a2-5909-4771-93ba-d83b7b5a1de9",
    "principal_type": "Group",
    "scope": "/instances/{{instanceId}}/providers/FoundationaLLM.Configuration/appConfigurations/FoundationaLLM:APIEndpoints:CoreAPI:Configuration:CompletionResponsePollingIntervalSeconds",
    "properties": null,
    "created_on": "{{deployTime}}",
    "updated_on": "{{deployTime}}",
    "created_by": null,
    "updated_by": null,
    "deleted": false,
    "expiration_date": null
},
{
    "type": "FoundationaLLM.Authorization/roleAssignments",
    "name": "{{externalAgentWorkflowReaderGuid}}",
    "object_id": "/providers/FoundationaLLM.Authorization/roleAssignments/{{externalAgentWorkflowReaderGuid}}",
    "display_name": null,
    "description": "Read Access for ExternalAgentWorkflow for the AllAgentsVirtualSecurityGroup group.",
    "cost_center": null,
    "role_definition_id": "/providers/FoundationaLLM.Authorization/roleDefinitions/00a53e72-f66e-4c03-8f81-7e885fd2eb35",
    "principal_id": "5bb493a2-5909-4771-93ba-d83b7b5a1de9",
    "principal_type": "Group",
    "scope": "/instances/{{instanceId}}/providers/FoundationaLLM.Agent/workflows/ExternalAgentWorkflow",
    "properties": null,
    "created_on": "{{deployTime}}",
    "updated_on": "{{deployTime}}",
    "created_by": null,
    "updated_by": null,
    "deleted": false,
    "expiration_date": null
},
{
    "type": "FoundationaLLM.Authorization/roleAssignments",
    "name": "{{langChainExpressionLanguageReaderGuid}}",
    "object_id": "/providers/FoundationaLLM.Authorization/roleAssignments/{{langChainExpressionLanguageReaderGuid}}",
    "display_name": null,
    "description": "Read Access for LangChainExpressionLanguage for the AllAgentsVirtualSecurityGroup group.",
    "cost_center": null,
    "role_definition_id": "/providers/FoundationaLLM.Authorization/roleDefinitions/00a53e72-f66e-4c03-8f81-7e885fd2eb35",
    "principal_id": "5bb493a2-5909-4771-93ba-d83b7b5a1de9",
    "principal_type": "Group",
    "scope": "/instances/{{instanceId}}/providers/FoundationaLLM.Agent/workflows/LangChainExpressionLanguage",
    "properties": null,
    "created_on": "{{deployTime}}",
    "updated_on": "{{deployTime}}",
    "created_by": null,
    "updated_by": null,
    "deleted": false,
    "expiration_date": null
},
{
    "type": "FoundationaLLM.Authorization/roleAssignments",
    "name": "{{openAIAssistantsFileSearchReaderGuid}}",
    "object_id": "/providers/FoundationaLLM.Authorization/roleAssignments/{{openAIAssistantsFileSearchReaderGuid}}",
    "display_name": null,
    "description": "Read Access for OpenAIAssistantsFileSearch for the AllAgentsVirtualSecurityGroup group.",
    "cost_center": null,
    "role_definition_id": "/providers/FoundationaLLM.Authorization/roleDefinitions/00a53e72-f66e-4c03-8f81-7e885fd2eb35",
    "principal_id": "5bb493a2-5909-4771-93ba-d83b7b5a1de9",
    "principal_type": "Group",
    "scope": "/instances/{{instanceId}}/providers/FoundationaLLM.Agent/tools/OpenAIAssistantsFileSearch",
    "properties": null,
    "created_on": "{{deployTime}}",
    "updated_on": "{{deployTime}}",
    "created_by": null,
    "updated_by": null,
    "deleted": false,
    "expiration_date": null
},
{
    "type": "FoundationaLLM.Authorization/roleAssignments",
    "name": "{{openAIAssistantsCodeInterpreterReaderGuid}}",
    "object_id": "/providers/FoundationaLLM.Authorization/roleAssignments/{{openAIAssistantsCodeInterpreterReaderGuid}}",
    "display_name": null,
    "description": "Read Access for OpenAIAssistantsCodeInterpreter for the AllAgentsVirtualSecurityGroup group.",
    "cost_center": null,
    "role_definition_id": "/providers/FoundationaLLM.Authorization/roleDefinitions/00a53e72-f66e-4c03-8f81-7e885fd2eb35",
    "principal_id": "5bb493a2-5909-4771-93ba-d83b7b5a1de9",
    "principal_type": "Group",
    "scope": "/instances/{{instanceId}}/providers/FoundationaLLM.Agent/tools/OpenAIAssistantsCodeInterpreter",
    "properties": null,
    "created_on": "{{deployTime}}",
    "updated_on": "{{deployTime}}",
    "created_by": null,
    "updated_by": null,
    "deleted": false,
    "expiration_date": null
},
{
    "type": "FoundationaLLM.Authorization/roleAssignments",
    "name": "{{dalleImageGenerationReaderGuid}}",
    "object_id": "/providers/FoundationaLLM.Authorization/roleAssignments/{{dalleImageGenerationReaderGuid}}",
    "display_name": null,
    "description": "Read Access for DALLEImageGeneration for the AllAgentsVirtualSecurityGroup group.",
    "cost_center": null,
    "role_definition_id": "/providers/FoundationaLLM.Authorization/roleDefinitions/00a53e72-f66e-4c03-8f81-7e885fd2eb35",
    "principal_id": "5bb493a2-5909-4771-93ba-d83b7b5a1de9",
    "principal_type": "Group",
    "scope": "/instances/{{instanceId}}/providers/FoundationaLLM.Agent/tools/DALLEImageGeneration",
    "properties": null,
    "created_on": "{{deployTime}}",
    "updated_on": "{{deployTime}}",
    "created_by": null,
    "updated_by": null,
    "deleted": false,
    "expiration_date": null
},
{
    "type": "FoundationaLLM.Authorization/roleAssignments",
    "name": "{{foundationaLLMContentSearchToolReaderGuid}}",
    "object_id": "/providers/FoundationaLLM.Authorization/roleAssignments/{{foundationaLLMContentSearchToolReaderGuid}}",
    "display_name": null,
    "description": "Read Access for FoundationaLLMContentSearchTool for the AllAgentsVirtualSecurityGroup group.",
    "cost_center": null,
    "role_definition_id": "/providers/FoundationaLLM.Authorization/roleDefinitions/00a53e72-f66e-4c03-8f81-7e885fd2eb35",
    "principal_id": "5bb493a2-5909-4771-93ba-d83b7b5a1de9",
    "principal_type": "Group",
    "scope": "/instances/{{instanceId}}/providers/FoundationaLLM.Agent/tools/FoundationaLLMContentSearchTool",
    "properties": null,
    "created_on": "{{deployTime}}",
    "updated_on": "{{deployTime}}",
    "created_by": null,
    "updated_by": null,
    "deleted": false,
    "expiration_date": null
}

Starting with 0.9.3-rc016

Configuration Resource Provider

The APIEnpointConfiguration class has been updated to change the previous StatusUrl property to StatusEndpoint, which is a relative path to the status endpoint. By extension, the related JSON files now have a status_endpoint property that contains the relative path. Here is the OrchestrationAPI JSON template as an example of this change:

{
    "type": "api-endpoint",
    "name": "OrchestrationAPI",
    "object_id": "/instances/{{instanceId}}/providers/FoundationaLLM.Configuration/apiEndpointConfigurations/OrchestrationAPI",
    "display_name": null,
    "description": null,
    "cost_center": null,
    "category": "General",
    "authentication_type": "APIKey",
    "authentication_parameters": {
        "api_key_configuration_name": "FoundationaLLM:APIEndpoints:OrchestrationAPI:Essentials:APIKey",
        "api_key_header_name": "X-API-KEY"
    },
    "url": "http://orchestration-api.{{serviceNamespaceName}}.svc.cluster.local",
    "status_endpoint": "/instances/{{instanceId}}/status",
    "url_exceptions": [],
    "timeout_seconds": 2400,
    "retry_strategy_name": "ExponentialBackoff",
    "created_on": "0001-01-01T00:00:00+00:00",
    "updated_on": "0001-01-01T00:00:00+00:00",
    "created_by": null,
    "updated_by": "SYSTEM",
    "deleted": false
}

The status path is a relative health check. Orchestration services such as LangChain and Semantic Kernel use it to check API status.

Important

All files within the /resource-provider/FoundationaLLM.Configuration directory must be updated to change the name of the status_url field to status_endpoint and change the value to a relative path as needed.

Vectorization resource provider changes

Vectorization indexing and partitioning profile settings dictionary keys are now persisted as snake case (ex. IndexName becomes index_name).

Agent resource provider changes

Deployment notes

1. Ensure the feature flag is enabled for FoundationaLLM.Agent.PrivateStore if using the private store feature on OpenAI Assistants workflow agents is desired.
2. Open existing OpenAIAssistantsWorkflow agents in the Management portal and select Save to populate the global vector store in the OpenAI service for the assistant.

Agent file resources

The agent file references are now stored in a new Cosmos DB container, while the file contents are stored in the storage account. Here are the configuration parameters for the required Cosmos DB container:

Name	Value
Name	Agents
Maximum RU/s	4000
Hierarchical Partition key	/instanceId + /agentName

As a result of the migration, the newly created Agents container will initially contain only ony type of times: AgentFileReference.

This is an example of such item:

{
    "instanceId": "8ac6074c-bdde-43cb-a140-ec0002d96d2b",
    "agentName": "TestAgentFiles1",
    "originalFilename": "curious_cat_story.pdf",
    "contentType": "application/pdf",
    "size": 2433,
    "upn": "andrei@foundationaLLM.ai",
    "id": "af-0285ddb8-a5b8-48b0-8248-bd0ad2f123bf",
    "objectId": "/instances/8ac6074c-bdde-43cb-a140-ec0002d96d2b/providers/FoundationaLLM.Agent/agents/TestAgentFiles1/agentFiles/af-0285ddb8-a5b8-48b0-8248-bd0ad2f123bf",
    "name": "af-0285ddb8-a5b8-48b0-8248-bd0ad2f123bf",
    "filename": "/FoundationaLLM.Agent/8ac6074c-bdde-43cb-a140-ec0002d96d2b/TestAgentFiles1/private-file-store/af-0285ddb8-a5b8-48b0-8248-bd0ad2f123bf.pdf",
    "type": "agent-file",
    "deleted": false,
    "_rid": "ie9IAMu0+b0EAAAAAAAAAA==",
    "_self": "dbs/ie9IAA==/colls/ie9IAMu0+b0=/docs/ie9IAMu0+b0EAAAAAAAAAA==/",
    "_etag": "\"37012abc-0000-0200-0000-67afaa800000\"",
    "_attachments": "attachments/",
    "_ts": 1739565696
}

The agent-file type has been removed and the references are no longer saved in the agent reference store _resource-references.json.

Tools

In the agent resource provider storage folder (resource-provider/FoundationaLLM.Agent), add a tool resource reference entry (_resource_references.json) as well as configuration file for the following tools: OpenAIAssistantsFileSearch, OpenAIAssistantsCodeInterpreter, DALLEImageGeneration, and FoundationaLLMContentSearchTool.

Reference example:

{
    "Name": "OpenAIAssistantsFileSearch",
    "Filename": "/FoundationaLLM.Agent/OpenAIAssistantsFileSearch.json",
    "Type": "tool",
    "Deleted": false
}

File example:

{
  "type": "tool",
  "name": "OpenAIAssistantsFileSearch",
  "object_id": "/instances/{{instanceId}}/providers/FoundationaLLM.Agent/tools/OpenAIAssistantsFileSearch",
  "display_name": "OpenAIAssistantsFileSearch",
  "description": "OpenAIAssistantsFileSearch",
  "cost_center": null,
  "properties": null,
  "created_on": "2025-01-10T08:22:34.2682433+00:00",
  "updated_on": "0001-01-01T00:00:00+00:00",
  "created_by": "dev@foundationaLLM.ai",
  "updated_by": null,
  "deleted": false,
  "expiration_date": null
}

Starting with 0.9.3-rc010

Resource provider cache warm-up

Resource providers now support a cache warm-up mechanism. This mechanism allows the cache to be pre-populated with the resource provider data before the service starts processing requests. This feature is useful when the service is deployed in a cold environment and needs to be warmed up before it can handle requests.

The cache warm-up mechanism is enabled when a file named _cache_warmup.json exists in the blob storage location associated with the resource provider. Here is an example of such a file:

[
	{
		"ServiceName": "OrchestrationAPI",
		"Description": "Resources required by: service principal x, service principal y.",
		"ResourceObjectIds": [
			"/instances/73fad442-.../providers/FoundationaLLM.Configuration/apiEndpointConfigurations/GatewayAPI",
			"/instances/73fad442-.../providers/FoundationaLLM.Configuration/apiEndpointConfigurations/AzureAISearch",
			"/instances/73fad442-.../providers/FoundationaLLM.Configuration/apiEndpointConfigurations/LangChainAPI",
			"/instances/73fad442-.../providers/FoundationaLLM.Configuration/apiEndpointConfigurations/StateAPI"
		],
		"SecurityPrincipalIds": [
			"4150c6b3-...",
			"949195b1-..."
		]
	},
	{
		"ServiceName": "OrchestrationAPI",
		"Description": "Resources required by: service principal x, service principal y, service principal z.",
		"ResourceObjectIds": [
			"/instances/73fad442-.../providers/FoundationaLLM.Configuration/apiEndpointConfigurations/AzureOpenAI"
		],
		"SecurityPrincipalIds": [
			"4150c6b3-...",
			"949195b1-...",
			"d6a6317a-..."
		]
	}
]

The configuration contains an array of objects, each representing a cache warm-up configuration. Each object contains the following properties:

• ServiceName - The name of the service that the cache warm-up configuration is for.

• Description - A description of the cache warm-up configuration.

• ResourceObjectIds - The list of resource object identifiers that will be pre-loaded into the resource provider cache.

  Note

  The resource object identifiers must be specific to the resource provider.

• SecurityPrincipalIds - The list of security principal identifiers that will be used to authenticate the cache warm-up requests.

  Important

  As a result of the cache warm-up process, the client authorization cache will be populated with all combinations of security principal and resource object identifiers that exist in the cache warm-up configuration. Make sure the two lists only contain the necessary values to avoid a long startup time for the resource provider.

Starting with 0.9.3-rc002

App configuration settings

The following App Config properties make cache settings for the AuthorizationServiceClientCacheService configurable:

Name	Description	Default Value
FoundationaLLM:APIEndpoints:AuthorizationAPI:Essentials:EnableCache	Indicates whether calls to the Authorization API should be cached or not.	false
FoundationaLLM:APIEndpoints:AuthorizationAPI:Essentials:AbsoluteCacheExpirationSeconds	Absolute cache expiration in seconds.	300
FoundationaLLM:APIEndpoints:AuthorizationAPI:Essentials:SlidingCacheExpirationSeconds	Sets how many seconds the cache entry can be inactive (e.g. not accessed) before it will be removed. This will not extend the entry lifetime beyond the absolute expiration (if set).	120
FoundationaLLM:APIEndpoints:AuthorizationAPI:Essentials:CacheSizeLimit	The maximum number of items that can be stored in the cache.	10000
FoundationaLLM:APIEndpoints:AuthorizationAPI:Essentials:CacheExpirationScanFrequencySeconds	Gets or sets the minimum length of time between successive scans for expired items in seconds.	30

Starting with 0.9.2-rc005

Agent configuration changes

Starting with this version, all agents MUST transition to the agent workflow configuraiton approach.

The following agent properties are no longer supported and should be deleted as part of upgrading to this version:

• OrchestrationSettings - fully replaced by the agent worflow settings.
• PromptObjectId - replaced by the agent workflow resource object identifier with an object_role of main_prompt.
• AIModelObjectId - replaced by the agent workflow resource object identifier with an object_role of main_model.
• Capabilities - removed. The equivalent of having Azure OpenAI Assistants capabilities is having an agent workflow with the type azure-openai-assistants-workflow.
• Azure.OpenAI.Assistant.Id property in properties - replaced by the assistant_id property of an agent workflow witht the type azure-openai-assistants-workflow.

Important

If the Azure.OpenAI.Assistant.Id property is set in the agent properties, it's value must be copied to the assistant_id property of the agent workflow.

Here is an example of a fully configured worfklow section for an agent:

{
    "type": "azure-openai-assistants-workflow",
		"name": "OpenAIAssistants",
		"package_name": "FoundationaLLM",
    "assistant_id": "asst_...",
    "resource_object_ids": {
        "/instances/.../providers/FoundationaLLM.Agent/workflows/OpenAIAssistants": {
            "object_id": "/instances/.../providers/FoundationaLLM.Agent/workflows/OpenAIAssistants",
            "properties": {}
        },
        "/instances/.../providers/FoundationaLLM.AIModel/aiModels/GPT4oMiniCompletionAIModel" : {
            "object_id": "/instances/.../providers/FoundationaLLM.AIModel/aiModels/GPT4oMiniCompletionAIModel",
            "properties": {
                "object_role": "main_model",
                "model_parameters": {}
            }
        },
        "/instances/.../providers/FoundationaLLM.Prompt/prompts/FoundationaLLM-mini": {
            "object_id": "/instances/.../providers/FoundationaLLM.Prompt/prompts/FoundationaLLM-mini",
            "properties": {
                "object_role": "main_prompt"
            }
        }
    }
}

Starting from 0.9.1

App configuration settings

To support the event grid infrastructure, the following new App Configuration settings are required.

[
	{
		"key": "FoundationaLLM:Events:Profiles:CoreAPI",
		"label": null,
		"value": "{\"EventProcessingCycleSeconds\": 5,\"Topics\": [{\"Name\": \"resource-providers\",\"SubscriptionPrefix\": \"rp-core\"}]}",
		"content_type": "application/json",
		"tags": {}
	},
	{
		"key": "FoundationaLLM:Events:Profiles:GatekeeperAPI",
		"label": null,
		"value": "{\"EventProcessingCycleSeconds\":60,\"Topics\":[]}",
		"content_type": "application/json",
		"tags": {}
	},
	{
		"key": "FoundationaLLM:Events:Profiles:GatewayAPI",
		"label": null,
		"value": "{\"EventProcessingCycleSeconds\": 5,\"Topics\": [{\"Name\": \"resource-providers\",\"SubscriptionPrefix\": \"rp-gateway\"}]}",
		"content_type": "application/json",
		"tags": {}
	},
	{
		"key": "FoundationaLLM:Events:Profiles:ManagementAPI",
		"label": null,
		"value": "{\"EventProcessingCycleSeconds\": 5,\"Topics\": [{\"Name\": \"resource-providers\",\"SubscriptionPrefix\": \"rp-management\"}]}",
		"content_type": "application/json",
		"tags": {}
	},
	{
		"key": "FoundationaLLM:Events:Profiles:OrchestrationAPI",
		"label": null,
		"value": "{\"EventProcessingCycleSeconds\": 5,\"Topics\": [{\"Name\": \"resource-providers\",\"SubscriptionPrefix\": \"rp-orch\"}]}",
		"content_type": "application/json",
		"tags": {}
	},
	{
		"key": "FoundationaLLM:Events:Profiles:VectorizationAPI",
		"label": null,
		"value": "{\"EventProcessingCycleSeconds\":60,\"Topics\":[]}",
		"content_type": "application/json",
		"tags": {}
	},
	{
		"key": "FoundationaLLM:Events:Profiles:VectorizationWorker",
		"label": null,
		"value": "{\"EventProcessingCycleSeconds\":60,\"Topics\":[]}",
		"content_type": "application/json",
		"tags": {}
	}
]

Note: The event grid system topics need to be removed.

The following topic needs to be created in the event grid namespace, must have a resource-providers topic with a publisher type of Custom and an input schema of Cloud Events v1.0.

Configuration changes

Added the following App Configuration value:

Name	Default value	Description
FoundationaLLM:UserPortal:Authentication:Entra:TimeoutInMinutes	60	The timeout in minutes for a user's auth token in the User Portal.
FoundationaLLM:UserPortal:Configuration:ShowFileUpload	true	Global setting to determine if file upload is allowed on chat messages.

Starting with 0.9.1-rc117

Agent configuration changes

"text_rewrite_settings": {
    "user_prompt_rewrite_enabled" : true,
    "user_prompt_rewrite_settings": {
        "user_prompt_rewrite_ai_model_object_id": "/instances/73fad442-f614-4510-811f-414cb3a3d34b/providers/FoundationaLLM.AIModel/aiModels/GPT4oCompletionAIModel",
        "user_prompt_rewrite_prompt_object_id": "/instances/73fad442-f614-4510-811f-414cb3a3d34b/providers/FoundationaLLM.Prompt/prompts/FoundationaLLM-v2-Rewrite",
        "user_prompts_window_size": 1
    }
},
"cache_settings": {
    "semantic_cache_enabled": true,
    "semantic_cache_settings": {
        "embedding_ai_model_object_id": "/instances/73fad442-f614-4510-811f-414cb3a3d34b/providers/FoundationaLLM.AIModel/aiModels/DefaultEmbeddingAIModel",
        "embedding_dimensions": 2048,
        "minimum_similarity_threshold": 0.975
    }
},

Semantic cache

Enable vector search in the Cosmos DB database using the following CLI command:

az cosmosdb update --resource-group <resource-group-name> --name <account-name> --capabilities EnableNoSQLVectorSearch

Create the CompletionsCache container in the Cosmos DB database with the following properties:

• Container id: CompletionsCache
• Partition key: /operationId
• Container Vector Policy: a policy with the following properties:
  • Path: /userPromptEmbedding
  • Data type: float32
  • Distance function: Cosine
  • Dimensions: 2048
  • Index type: diskANN (leave the default values)

After the container is created, set the Time to Live property on the container to 300 seconds.

Starting with 0.9.1-rc105

Configuration changes

The following new App Configuration settings are required:

Name	Default value	Description
FoundationaLLM:PythonSDK:Logging:LogLevel:Azure	Warning	Provides the default level of logging for Azure modules in the Python SDK.

Agent workflow configuration changes

Agent resource configuration files that have a workflow property now requires a name and package_name property. This is to support loading external workflows via plugins. For internal workflows, the package_name should be set to FoundationaLLM. Example below truncated for brevity.

{
    "workflow": {
        "type": "langgraph-react-agent-workflow",
        "name": "LangGraphReactAgent",
        "package_name": "FoundationaLLM",
        "workflow_host": "LangChain",
        "graph_recursion_limit": 10,
        "resource_object_ids": {}
    }
}

A new Workflow resource must be added to the FoundationaLLM.Agent resource provider:

{
  "type": "external-agent-workflow",
  "name": "ExternalAgentWorkflow",
  "object_id": "/instances/<instance_id>/providers/FoundationaLLM.Agent/workflows/ExternalAgentWorkflow",
  "display_name": "ExternalAgentWorkflow",
  "description": "External Agent workflow",
  "cost_center": null,
  "properties": null,
  "created_on": "2024-11-13T18:12:07.0223039+00:00",
  "updated_on": "0001-01-01T00:00:00+00:00",
  "created_by": "dev@foundationaLLM.ai",
  "updated_by": null,
  "deleted": false,
  "expiration_date": null
}

Starting with 0.9.1-rc102

Configuration changes

The following new App Configuration settings are required:

Name	Default value	Description
FoundationaLLM:APIEndpoints:OrchestrationAPI:Configuration:CompletionRequestsStorage:AccountName	<main_storage_account_name>	Provides the storage account used by the Orchestration API to persist completion requests.
FoundationaLLM:APIEndpoints:OrchestrationAPI:Configuration:CompletionRequestsStorage:AuthenticationType	AzureIdentity	Indicates that managed identity authentication should be used to access the storage account.
FoundationaLLM:APIEndpoints:OrchestrationAPI:Configuration:CompletionRequestsStorage:ContainerName	orchestration-completion-requests	Provides the storage container name used by the Orchestration API to persist completion requests. Should always be orchestration-completion-requests

User profile changes

A new flag named persistOrchestrationCompletionRequests is added to the user profile. This flag is used to determine whether the user's completion requests should be persisted in the storage account. The default value is false.

Sample configuration:

"flags": {
        "oneDriveWorkSchoolEnabled": true,
        "persistOrchestrationCompletionRequests": true
    },

Starting with 0.9.1-rc101

Configuration changes

The following new App Configuration settings are required:

Name	Default value	Description
FoundationaLLM:Code:CodeExecution:AzureContainerAppsDynamicSessions	{"DynamicSessionsEndpoints": []}	Provides the configuration for the Azure Container Apps Dynamic Sessions code execution service. DynamicSessionsEnpoints is a list of Dynamic Sessions endpoints that are used to run code execution sessions. Must contain at least one value.

Agent tool configuration changes

Each agent tool should have an entry in the properties dictionary named foundationallm_aca_code_execution_enabled (true or false) to indicate whether the tool requires code execution sessions based on the the Azure Container Apps Dynamic Sessions service.

Prompt definition changes

Prompt prefixes and suffixes support FoundationaLLM variables for dynamic replacement at runtime. The variable format is {{foundationallm:variable_name[:format]}} where

• variable_name is the name of the well-known variable.
• format is the optional formatting applied to the value of the variable.

The following variables are supported:

Name	Value	Example
current_datetime_utc	The current UTC date and time.	The current date is {{foundationallm:current_datetime_utc:dddd, MMMM dd, yyyy}}. This looks great. -> The current date is Sunday, December 15, 2024. This looks great.

Starting with 0.9.0

Configuration changes

The following new App Configuration settings are required:

Name	Default value	Description
FoundationaLLM:PythonSDK:Logging:LogLevel:Default	Information	-
FoundationaLLM:PythonSDK:Logging:EnableConsoleLogging	false	-
FoundationaLLM:APIEndpoints:CoreAPI:Configuration:Entra:RequireScopes	true	Indicates whether a delegated scope claim (scp) or an application permission claim (roles) is required for authorization. Set to false to allow authentication from an external proxy API.
FoundationaLLM:APIEndpoints:CoreAPI:Configuration:Entra:AllowACLAuthorization	false	Indicates whether tokens that do not have either of the "scp" or "roles" claims are accepted (True means they are accepted). Set to true to allow authentication from an external proxy API.
FoundationaLLM:APIEndpoints:LangChainAPI:Configuration:ExternalModules:Storage:AccountName	-	-
FoundationaLLM:APIEndpoints:LangChainAPI:Configuration:ExternalModules:Storage:AuthenticationType	-	-
FoundationaLLM:APIEndpoints:LangChainAPI:Configuration:ExternalModules:RootStorageContainer	-	-
FoundationaLLM:APIEndpoints:LangChainAPI:Configuration:ExternalModules:Modules	-	-
FoundationaLLM:APIEndpoints:LangChainAPI:Configuration:PollingIntervalSeconds	1	The interval in seconds at which the LangChain API will be polled for status.
FoundationaLLM:UserPortal:Configuration:ShowMessageRating	true	If true, rating options on agent messages will appear.
FoundationaLLM:UserPortal:Configuration:ShowLastConversationOnStartup	false	If true, the last conversation will be displayed when the user logs in. Otherwise, a new conversation placeholder appears on page load.
FoundationaLLM:UserPortal:Configuration:ShowMessageTokens	true	If true, the number of consumed tokens on agent and user messages will appear.
FoundationaLLM:UserPortal:Configuration:ShowViewPrompt	true	If true, the "View Prompt" button on agent messages will appear.
FoundationaLLM:Instance:EnableResourceProvidersCache	false	If true, the caching of resource providers will be enabled.
FoundationaLLM:APIEndpoints:AuthorizationAPI:Essentials:EnableCache	false	If true, the caching of authorization call results will be enabled.

Agent Tool configuration changes

Agent tools are now an array of AgentTool objects rather than a dictionary.

When defining tools for an agent, each tool now requires a package_name property. This property is used to identify the package that contains the tool's implementation. If the tool is internal, the package_name should be set to FoundationaLLM, if the tool is external, the package_name should be set to the name of the external package.

Security-related changes

The Authorization API now requires the ability to write to the Key Vault account contained within the auth resource group. Currently, the Authorization APIs managed identity is assigned to the Key Vault Secrets User role on the Key Vault account. This role assignment must be updated to include the Key Vault Secrets Officer role in addition to the user role.

Renamed classes

The following classes have been renamed:

Original Class	New Class
FoundationaLLM.Common.Models.Orchestration.Response.Citation	FoundationaLLM.Common.Models.Orchestration.Response.ContentArtifact

API endpoint changes

Core API

The /instances/{instanceId}/sessions/{sessionId}/message/{id}/rate endpoint has been updated to accept the rating in the message body, rather than as a query parameter. Send the following payload in the request body:

{
  "rating": true,
  "comments": "string"
}

Note

Please note that both properties are nullable. Set them to null to clear out the rating and comments.

Starting with 0.8.4

Configuration changes

The following new App Configuration settings are required:

Name	Default value
FoundationaLLM:APIEndpoints:ManagementAPI:Configuration:AllowedUploadFileExtensions	c, cpp, cs, css, csv, doc, docx, gif, html, java, jpeg, jpg, js, json, md, pdf, php, png, pptx, py, rb, sh, tar, tex, ts, txt, xlsx, xml, zip
FoundationaLLM:Branding:NoAgentsMessage	No agents available. Please check with your system administrator for assistance.
FoundationaLLM:Branding:DefaultAgentWelcomeMessage	Start the conversation using the text box below.

The following new App Configuration feature flags are required:

Name	Default value
FoundationaLLM.Agent.PrivateStore	Not enabled

Assistants API enabled Agent(s)

Important

Any existing agent that has the Assistants API enabled needs to be saved from the Management UI to update itself.

Resource provider changes

FoundationaLLM.Authorization

The following entries need to be added to the policy store file:

{
    "name": "GUID03",
    "type": "FoundationaLLM.Authorization/policyAssignments",
    "object_id": "/providers/FoundationaLLM.Authorization/policyAssignments/GUID03",
    "description": "Ownership on conversation mapping resources managed by the FoundationaLLM.AzureOpenAI resource provider.",
    "policy_definition_id": "/providers/FoundationaLLM.Authorization/policyDefinitions/00000000-0000-0000-0001-000000000001",
    "principal_id": "SECURITY_GROUP_ID",
    "principal_type": "Group",
    "scope": "/instances/FOUNDATIONALLM_INSTANCEID/providers/FoundationaLLM.AzureOpenAI/conversationMappings",
    "created_on": "DEPLOY_TIME",
    "updated_on": "DEPLOY_TIME",
    "created_by": "SYSTEM",
    "updated_by": "SYSTEM"
},
{
    "name": "GUID04",
    "type": "FoundationaLLM.Authorization/policyAssignments",
    "object_id": "/providers/FoundationaLLM.Authorization/policyAssignments/GUID04",
    "description": "Ownership on file mapping resources managed by the FoundationaLLM.AzureOpenAI resource provider.",
    "policy_definition_id": "/providers/FoundationaLLM.Authorization/policyDefinitions/00000000-0000-0000-0001-000000000001",
    "principal_id": "SECURITY_GROUP_ID",
    "principal_type": "Group",
    "scope": "/instances/FOUNDATIONALLM_INSTANCEID/providers/FoundationaLLM.AzureOpenAI/fileMappings",
    "created_on": "DEPLOY_TIME",
    "updated_on": "DEPLOY_TIME",
    "created_by": "SYSTEM",
    "updated_by": "SYSTEM"
}

The following placehoders need to be replaced with the actual values:

• SECURITY_GROUP_ID - the ID of the security group that needs to be assigned to the policy
• FOUNDATIONALLM_INSTANCEID - the ID of the FoundationaLLM instance
• DEPLOY_TIME - the time when the policy was deployed
• GUID03 and GUID04 - unique identifiers for the policy assignments

FoundationaLLM.AzureOpenAI

The assistant and file user context artifacts are now simplified and stored in a new Cosmos DB container. Here are the configuration parameters for the new Cosmos DB container:

Name	Value
Name	ExternalResources
Maximum RU/s	1000
Time to live	Off
Partition key	/partitionKey

Part of the upgrade to this version is to migrate the existing assistant and file user context artifacts to the new Cosmos DB container. Refer to the dedicated upgrade tool for instructions on how to perform this update.

As a result of the migration, the newly created ExternalResources container will contain two types of items: AzureOpenAIConversationMapping and AzureOpenAIFileMapping.

This is an example of an AzureOpenAIConversationMapping item:

{
    "conversationId": "0e56a170-5355-...",
    "openAIAssistantsAssistantId": "asst_kc...",
    "openAIAssistantsThreadId": "thread_73...",
    "openAIAssistantsThreadCreatedOn": "2024-10-14T17:57:10.510345+00:00",
    "openAIVectorStoreId": "vs_X6...",
    "openAIVectorStoreCreatedOn": null,
    "type": "AzureOpenAIConversationMapping",
    "id": "0e56a170-5355-...",
    "partitionKey": "...-73fad442-f614-4510-811f-414cb3a3d34b",
    "upn": "jackthecat@foundationaLLM.ai",
    "instanceId": "73fad442-f614-4510-811f-414cb3a3d34b",
    "openAIEndpoint": "https://openai-....openai.azure.com/",
    "objectId": null,
    "displayName": null,
    "description": null,
    "costCenter": null,
    "properties": null,
    "createdOn": "0001-01-01T00:00:00+00:00",
    "updatedOn": "0001-01-01T00:00:00+00:00",
    "createdBy": null,
    "updatedBy": null,
    "deleted": false,
    "expirationDate": null,
    "name": "0e56a170-5355-...",
    "_rid": "J2BUAKktW41bAAAAAAAAAA==",
    "_self": "dbs/J2BUAA==/colls/J2BUAKktW40=/docs/J2BUAKktW41bAAAAAAAAAA==/",
    "_etag": "\"8702b793-0000-0200-0000-672a60b90000\"",
    "_attachments": "attachments/",
    "_ts": 1730830521
}

This is an example of an AzureOpenAIFileMapping item:

{
    "fileObjectId": "/instances/73fad442-f614-4510-811f-414cb3a3d34b/providers/FoundationaLLM.Attachment/attachments/a-f8...",
    "originalFileName": "some_file.csv",
    "fileContentType": "text/csv",
    "fileRequiresVectorization": false,
    "openAIFileId": "assistant-8G...",
    "openAIFileUploadedOn": "2024-10-14T23:01:02.3075592+00:00",
    "openAIAssistantsFileGeneratedOn": null,
    "openAIVectorStoreId": null,
    "type": "AzureOpenAIFileMapping",
    "id": "assistant-8G...",
    "partitionKey": "...-73fad442-f614-4510-811f-414cb3a3d34b",
    "upn": "jackthecat@foundationaLLM.ai",
    "instanceId": "73fad442-f614-4510-811f-414cb3a3d34b",
    "openAIEndpoint": "https://openai-....openai.azure.com/",
    "objectId": null,
    "displayName": null,
    "description": null,
    "costCenter": null,
    "properties": null,
    "createdOn": "0001-01-01T00:00:00+00:00",
    "updatedOn": "0001-01-01T00:00:00+00:00",
    "createdBy": null,
    "updatedBy": null,
    "deleted": false,
    "expirationDate": null,
    "name": "assistant-8G...",
    "_rid": "J2BUAKktW40yAAAAAAAAAA==",
    "_self": "dbs/J2BUAA==/colls/J2BUAKktW40=/docs/J2BUAKktW40yAAAAAAAAAA==/",
    "_etag": "\"87025e93-0000-0200-0000-672a60b70000\"",
    "_attachments": "attachments/",
    "_ts": 1730830519
}

Cleanup role assignments

As a result of migrated resources from storage account to Cosmos DB, as well as the new policy-assignments mentioned above, the role-assignments store will have obsolete Owner role assignments on those objects. Please refer to the dedicated tool for instructions on how to perform this cleanup.

The dedicated tool will cleanup role assignments for the following resources:

• FoundationaLLM.Attachment/attachments
• FoundationaLLM.AzureOpenAI/fileUserContexts
• FoundationaLLM.AzureOpenAI/assistantUserContexts
• FoundationaLLM.Conversation/conversations

Configuration changes

Resource provider templates

The AzureOpenAI.template.json files within deploy/quick-start/data/resource-provider/FoundationaLLM.Configuration and deploy/standard/data/resource-provider/FoundationaLLM.Configuration have been updated to set the category field to the value LLM. This discriminator allows the Management Portal to filter the list of API endpoints by category and provide options to add AI Models to endpoints with the LLM category.

The existing category property needs to be set to LLM on existing API endpoint configurations in the FoundationaLLM.Configuration resource provider that fit this description, including the AzureOpenAI endpoint configuration.

Starting with 0.8.3

Resource provider changes

If a user/group is not assigned to the instance-level Contributor role, then they will not be able to create new Conversations or upload Attachments. To adjust their permissions, the following changes are required:

FoundationaLLM.Conversation

In addition to assigning users/groups to the policy-assignments/<instance_id>-policy.json file within the FoundationaLLM.Authorization resource provider to assign them to the Conversation policy, we must now add them to the new Conversation contributor role (role_definition_id: d0d21b90-5317-499a-9208-3a6cb71b84f9) within the role-assignments/<instance_id>-role.json file within the FoundationaLLM.Authorization resource provider if the user/group is not assigned to the Contributor role on the FoundationaLLM instance (role_definition_id: a9f0020f-6e3a-49bf-8d1d-35fd53058edf). Here is an example entry:

{
    "type": "FoundationaLLM.Authorization/roleAssignments",
    "name": "a40b15f1-75ce-4a40-a857-1093ac9adf4d",
    "object_id": "/instances/0a1840df-71b6-496d-905a-145d93d827f3/providers/FoundationaLLM.Authorization/roleAssignments/a40b15f1-75ce-4a40-a857-1093ac9adf4d",
    "display_name": null,
    "description": "Conversation contributor role for FLLM Users",
    "cost_center": null,
    "role_definition_id": "/providers/FoundationaLLM.Authorization/roleDefinitions/d0d21b90-5317-499a-9208-3a6cb71b84f9",
    "principal_id": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee",
    "principal_type": "Group",
    "scope": "/instances/0a1840df-71b6-496d-905a-145d93d827f3",
    "properties": null,
    "created_on": "0001-01-01T00:00:00+00:00",
    "updated_on": "0001-01-01T00:00:00+00:00",
    "created_by": null,
    "updated_by": null,
    "deleted": false,
    "expiration_date": null
}

FoundationaLLM.Attachment

In addition to assigning users/groups to the policy-assignments/<instance_id>-policy.json file within the FoundationaLLM.Authorization resource provider to assign them to the Attachment policy, we must now add them to the new Attachment contributor role (role_definition_id: 8e77fb6a-7a78-43e1-b628-d9e2285fe25a) within the role-assignments/<instance_id>-role.json file within the FoundationaLLM.Authorization resource provider if the user/group is not assigned to the Contributor role on the FoundationaLLM instance (role_definition_id: a9f0020f-6e3a-49bf-8d1d-35fd53058edf). Here is an example entry:

{
    "type": "FoundationaLLM.Authorization/roleAssignments",
    "name": "891ca947-e648-46cf-a12a-774b52ded886",
    "object_id": "/instances/0a1840df-71b6-496d-905a-145d93d827f3/providers/FoundationaLLM.Authorization/roleAssignments/891ca947-e648-46cf-a12a-774b52ded886",
    "display_name": null,
    "description": "Attachment contributor role for FLLM Users",
    "cost_center": null,
    "role_definition_id": "/providers/FoundationaLLM.Authorization/roleDefinitions/8e77fb6a-7a78-43e1-b628-d9e2285fe25a",
    "principal_id": "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee",
    "principal_type": "Group",
    "scope": "/instances/0a1840df-71b6-496d-905a-145d93d827f3",
    "properties": null,
    "created_on": "0001-01-01T00:00:00+00:00",
    "updated_on": "0001-01-01T00:00:00+00:00",
    "created_by": null,
    "updated_by": null,
    "deleted": false,
    "expiration_date": null
}

Starting with 0.8.2

Configuration changes

The following settings are required:

Name	Default value
FoundationaLLM:APIEndpoints:CoreAPI:Configuration:AllowedUploadFileExtensions	c, cpp, cs, css, csv, doc, docx, gif, html, java, jpeg, jpg, js, json, md, pdf, php, png, pptx, py, rb, sh, tar, tex, ts, txt, xlsx, xml, zip
FoundationaLLM:APIEndpoints:CoreAPI:Configuration:AzureOpenAIAssistantsFileSearchFileExtensions	c, cpp, cs, css, doc, docx, html, java, js, json, md, pdf, php, pptx, py, rb, sh, tex, ts, txt
FoundationaLLM:APIEndpoints:CoreAPI:Configuration:MaxUploadsPerMessage	{ "value": 10, "value_exceptions": [] }
FoundationaLLM:APIEndpoints:CoreAPI:Configuration:CompletionResponsePollingIntervalSeconds	{ "value": 5, "value_exceptions": [] }
FoundationaLLM:APIEndpoints:GatewayAPI:Configuration:AzureOpenAIAssistantsMaxVectorizationTimeSeconds	120

Note

Here is an example of an override for the MaxUploadsPerMessage setting:

{
  "value": 10,
  "value_exceptions": [
      {
          "user_principal_name": "ciprian@solliance.net",
          "value": 5,
          "enabled": true
      }
  ]
}

Note

Here is an example of an override for the CompletionResponsePollingIntervalSeconds setting:

{
  "value": 5,
  "value_exceptions": [
      {
          "user_principal_name": "ciprian@solliance.net",
          "value": 3,
          "enabled": true
      }
  ]
}

The following settings are optional (they should not be set by default):

Name	Default value
FoundationaLLM:Instance:IdentitySubstitutionSecurityPrincipalId	<security_principal_id>
FoundationaLLM:Instance:IdentitySubstitutionUserPrincipalNamePattern	^fllm_load_test_user_\d{5}_\d{3}@solliance\.net$

Note

The FoundationaLLM:Instance:IdentitySubstitutionSecurityPrincipalId and FoundationaLLM:Instance:IdentitySubstitutionUserPrincipalNamePattern settings are used for load testing purposes only. If set, their values must be replaced with the appropriate values for the specific Entra ID tenant.

Resource provider changes

The following resource provider files must be renamed (if they already exist):

Location	Old name	New name
resource-provider/FoundationaLLM.Agent	_agent-references.json	_resource-references.json
resource-provider/FoundationaLLM.AIModel	_ai-model-references.json	_resource-references.json
resource-provider/FoundationaLLM.Configuration	_api-endpoint-references.json	_resource-references.json
resource-provider/FoundationaLLM.DataSource	_data-source-references.json	_resource-references.json
resource-provider/FoundationaLLM.Prompt	_prompt-references.json	_resource-references.json

Note

Within each of the renamed files, the <entity>References property must be renamed to ResourceReferences.

FoundationaLLM.Agent

A new property can be added to agent definitions:

"tools": {
    "dalle-image-generation": {
        "name": "dalle-image-generation",
        "description": "Generates an image based on a prompt.",
        "ai_model_object_ids": {
            "main_model": "/instances/73fad442-f614-4510-811f-414cb3a3d34b/providers/FoundationaLLM.AIModel/aiModels/DALLE3"
        }
    }
}

FoundationaLLM.Authorization

A new storage container named policy-assignments is required. The FoundationaLLM.Authorization resource provider will use this container to store policy assignments.

Within the container, the <instance_id>-policy.json must be deployed with the default policy assignments. The template for the default policy assignments is available in Common/Constants/Data/DefaultPolicyAssignments.json.

FoundationaLLM.Conversation

When upgrading an existing FoundationaLLM instance, the items in the Sessions collection in Cosmos DB must be updated according to the following rules:

if Object is of type Session of KioskSession:
    If the property DisplayName exists and is set to a non-empty string:
        Don't touch the item
    else:
        Set DisplayName to the value of Name
        Set Name to the value of SessionId
else:
    No action needed

Refer to the dedicated upgrade tool for instruction on how to perform this update.

FoundationaLLM.Configuration

The OneDrive (Work or School) integration requires the following API Endpoint Configuration entry in the storage account:

FoundationaLLM.Configuration/OneDriveFileStoreConnector.json

{
    "type": "api-endpoint",
    "name": "OneDriveFileStoreConnector",
    "object_id": "/instances/{{instance_id}}/providers/FoundationaLLM.Configuration/apiEndpointConfigurations/OneDriveFileStoreConnector",
    "display_name": null,
    "description": null,
    "cost_center": null,
    "category": "FileStoreConnector",
    "subcategory": "OneDriveWorkSchool",
    "authentication_type": "AzureIdentity",
    "authentication_parameters": {
      "scope": "Files.Read.All"
    },
    "url": "{{onedrive_base_url}}",
    "status_url": "",
    "url_exceptions": [],
    "timeout_seconds": 2400,
    "retry_strategy_name": "ExponentialBackoff",
    "created_on": "0001-01-01T00:00:00+00:00",
    "updated_on": "0001-01-01T00:00:00+00:00",
    "created_by": null,
    "updated_by": "SYSTEM",
    "deleted": false
  }

Update FoundationaLLM.Configuration/_resource-references_.json with the reference to the file above.

{
	"Name": "OneDriveFileStoreConnector",
	"Filename": "/FoundationaLLM.Configuration/OneDriveFileStoreConnector.json",
	"Type": "api-endpoint",
	"Deleted": false
}

FoundationaLLM.Attachment

The Attachment resource provider now saves the attachment references to Cosmos DB, instead of Data Lake storage. A new Cosmos DB container must be created, named Attachments, with the following partition key: /upn.

The following MSIs require a Cosmos DB role assigned:

1. Gateway API
2. Orchestration API
3. Management API

Long-Running Operations

The context for a long-running operation is now stored in Cosmos DB. A new Cosmos DB container must be created, named Operations, with a partition key /id.

Starting with 0.8.0

Core API changes:

1. All Core API endpoints have been moved to the /instances/{instanceId} path. For example, the /status endpoint is now /instances/{instanceId}/status.
2. The /orchestration/* endpoints have been moved to /instances/{instanceId}/completions/*.
   1. The previous /orchestration/completions endpoint is now /instances/{instanceId}/completions.
3. The /sessions/{sessionId}/completion endpoint has been moved to /instances/{instanceId}/completions. Instead of having the sessionId as a path parameter, it is now in the request body as part of the CompletionRequest payload.
4. /sessions/{sessionId}/summarize-name has been removed. In the future, the /completions endpoint will be used to generate summaries.
5. OrchestrationRequest and CompletionRequest have combined into a single CompletionRequest object.
6. DirectionCompletionRequest has been removed. Use CompletionRequest instead.
7. Status controllers \status action in the .NET API projects return value has renamed the Instance property to InstanceName.
8. The CompletionController.cs file under dotnet/CoreApi/controllers has introduced the Async-Completions endpoint to handle asynchronous completions.
9. With the introduction of Async-Completions, long running operations can now report on completion status based on Pending, InProgress, Completed and Failed states.
10. Vectorization Embedding Profile introduces a required key in the Settings property named model_name. Every embedding request now flows through the Gateway API.
11. Vectorization Indexing Profile introduces a required key api_endpoint_configuration_object_id in the Settings property.
12. Retirement of SemanticKernel embedding type. All embedding requests now flow through the Gateway API.

Gatekeeper API changes:

1. All Gatekeeper API endpoints have been moved to the /instances/{instanceId} path. For example, the /status endpoint is now /instances/{instanceId}/status.
2. The /orchestration/* endpoints have been moved to /instances/{instanceId}/completions/*.

Orchestration API changes:

1. All Gatekeeper API endpoints have been moved to the /instances/{instanceId} path. For example, the /status endpoint is now /instances/{instanceId}/status.
2. The /orchestration/* endpoints have been moved to /instances/{instanceId}/completions/*. =======

New APIs

Gateway Adapter API - requires the following configuration settings:

• FoundationaLLM:APIs:GatewayAdapterAPI:APIUrl
• FoundationaLLM:APIs:GatewayAdapterAPI:APIKey (mapped to the foundationallm-apis-gatewayadapterapi-apikey secret)
• FoundationaLLM:APIs:GatewayAdapterAPI:APIAppInsightsConnectionString (mapped to the foundationallm-app-insights-connection-string secret)

State API - requires the following configuration settings:

• FoundationaLLM:APIs:StateAPI:APIUrl
• FoundationaLLM:APIs:StateAPI:APIKey (mapped to the foundationallm-apis-stateapi-apikey secret)
• FoundationaLLM:APIs:StateAPI:APIAppInsightsConnectionString (mapped to the foundationallm-app-insights-connection-string secret)

Note

These new APIs will be converted to use the new APIEndpoint artifacts.

Changes in app registration names

API Name	Entra ID app registration name	Application ID URI	Scope name
Core API	FoundationaLLM-Core-API	api://FoundationaLLM-Core	Data.Read
Management API	FoundationaLLM-Management-API	api://FoundationaLLM-Management	Data.Manage
Authorization API	FoundationaLLM-Authorization-API	api://FoundationaLLM-Authorization	Authorization.Manage
User Portal	FoundationaLLM-Core-Portal	api://FoundationaLLM-Core-Portal	N/A
Management Portal	FoundationaLLM-Management-Portal	api://FoundationaLLM-Management-Portal	N/A

Changes in app configuration settings

The FoundationaLLM:APIs and FoundationaLLM:ExternalAPIs configuration namespaces have been replaced with the FoundationaLLM:APIEndpoints configuration namespace.

Important

All existing API registrations need to be updated to reflect these changes. The only two settings that will exist under FoundationaLLM:APIEndpoints are APIKey (for those API enpoints which use API key authentication) and AppInsightsConnectionString, all the other settings are now part of the APIEndpoint artifact managed by the FoundationaLLM.Configuration resource provider. This is an example for CoreAPI:

• FoundationaLLM:APIEndpoints:CoreAPI:APIKey
• FoundationaLLM:APIEndpoints:CoreAPI:AppInsightsConnectionString

The FoundationaLLM:AzureAIStudio configuration namespace expects an APIEndpointConfigurationName property instead of BaseUrl.

A new configuration setting named FoundationaLLM:Instance:SecurityGroupRetrievalStrategy with a value of IdentityManagementService must exist in the app configuration. It will be added by default in new deployments.

Two new configuration settings required by the new FoundationaLLM.AzureOpenAI resource provider:

• FoundationaLLM:ResourceProviders:AzureOpenAI:Storage:AuthenticationType
• FoundationaLLM:ResourceProviders:AzureOpenAI:Storage:AccountName

Pre-0.8.0

1. Vectorization resource stores use a unique collection name, Resources. They also add a new top-level property named DefaultResourceName.

2. The items in the index_references collection have a property incorrectly named type which was renamed to index_entry_id.

3. New gateway API, requires the following app configurations:

   • FoundationaLLM:APIs:GatewayAPI:APIUrl
   • FoundationaLLM:APIs:GatewayAPI:APIKey (with secret foundationallm-apis-gatewayapi-apikey)
   • FoundationaLLM:APIs:GatewayAPI:AppInsightsConnectionString (with secret foundationallm-app-insights-connection-string)
   • FoundationaLLM:Gateway:AzureOpenAIAccounts

4. The AgentFactory and AgentFactoryAPI classes have been renamed to Orchestration and OrchestrationAPI, respectively. The following App Config settings need to be replaced in existing environments:

   • FoundationaLLM:APIs:AgentFactoryAPI:APIKey -> FoundationaLLM:APIs:OrchestrationAPI:APIKey
   • FoundationaLLM:APIs:AgentFactoryAPI:APIUrl -> FoundationaLLM:APIs:OrchestrationAPI:APIUrl
   • FoundationaLLM:APIs:AgentFactoryAPI:AppInsightsConnectionString -> FoundationaLLM:APIs:OrchestrationAPI:AppInsightsConnectionString
   • FoundationaLLM:Events:AzureEventGridEventService:Profiles:AgentFactoryAPI -> FoundationaLLM:Events:AzureEventGridEventService:Profiles:OrchestrationAPI
   • FoundationaLLM:APIs:AgentFactoryAPI:ForceHttpsRedirection -? FoundationaLLM:APIs:OrchestrationAPI:ForceHttpsRedirection

5. The following Key Vault secrets need to be replaced in existing environments:

   • foundationallm-apis-agentfactoryapi-apikey -> foundationallm-apis-orchestrationapi-apikey

   There is an upgrade script available that migrates these settings and secrets to their new names.

6. The following App Config settings are no longer needed:

   • FoundationaLLM:Vectorization:Queues:Embed:ConnectionString
   • FoundationaLLM:Vectorization:Queues:Extract:ConnectionString
   • FoundationaLLM:Vectorization:Queues:Index:ConnectionString
   • FoundationaLLM:Vectorization:Queues:Partition:ConnectionString

7. The following Key Vault secret is no longer needed:

   • foundationallm-vectorization-queues-connectionstring

8. The following App Config settings need to be added as key-values:

   • FoundationaLLM:Vectorization:Queues:Embed:AccountName (set to the name of the storage account that contains the vectorization queues - e.g., stejahszxcubrpi)
   • FoundationaLLM:Vectorization:Queues:Extract:AccountName (set to the name of the storage account that contains the vectorization queues - e.g., stejahszxcubrpi)
   • FoundationaLLM:Vectorization:Queues:Index:AccountName (set to the name of the storage account that contains the vectorization queues - e.g., stejahszxcubrpi)
   • FoundationaLLM:Vectorization:Queues:Partition:AccountName (set to the name of the storage account that contains the vectorization queues - e.g., stejahszxcubrpi)

9. The value for the App Config setting FoundationaLLM:Events:AzureEventGridEventService:Profiles:OrchestrationAPI should be set in the following format:

   {
       "EventProcessingCycleSeconds": 20,
       "Topics": [
           {
               "Name": "storage",
               "SubscriptionPrefix": "orch",
               "EventTypeProfiles": [
                   {
                       "EventType": "Microsoft.Storage.BlobCreated",
                       "EventSets": [
                           {
                               "Namespace": "ResourceProvider.FoundationaLLM.Agent",
                               "Source": "/subscriptions/0a03d4f9-c6e4-4ee1-87fb-e2005d2c213d/resourceGroups/rg-fllm-aca-050/providers/Microsoft.Storage/storageAccounts/stejahszxcubrpi",
                               "SubjectPrefix": "/blobServices/default/containers/resource-provider/blobs/FoundationaLLM.Agent"
                           },
                           {
                               "Namespace": "ResourceProvider.FoundationaLLM.Vectorization",
                               "Source": "/subscriptions/0a03d4f9-c6e4-4ee1-87fb-e2005d2c213d/resourceGroups/rg-fllm-aca-050/providers/Microsoft.Storage/storageAccounts/stejahszxcubrpi",
                               "SubjectPrefix": "/blobServices/default/containers/resource-provider/blobs/FoundationaLLM.Vectorization"
                           },
                           {
                               "Namespace": "ResourceProvider.FoundationaLLM.Prompt",
                               "Source": "/subscriptions/0a03d4f9-c6e4-4ee1-87fb-e2005d2c213d/resourceGroups/rg-fllm-aca-050/providers/Microsoft.Storage/storageAccounts/stejahszxcubrpi",
                               "SubjectPrefix": "/blobServices/default/containers/resource-provider/blobs/FoundationaLLM.Prompt"
                           }
                       ]
                   }
               ]
           }
       ]
   }
   

10. Vectorization text embedding profiles require only two items in the configuration_references section: DeploymentName and Endpoint. Optionally, a deployment_name entry can be specified in the settings section to override the default value in configuration_references.Endpoint. Here is an example of the updated format for a text embedding profile:

    {
        "type": "text-embedding-profile",
        "name": "AzureOpenAI_Embedding_BaselineGlobalMacro",
        "object_id": "/instances/a6221c30-0bf2-4003-adb8-d3086bb2ad49/providers/FoundationaLLM.Vectorization/textEmbeddingProfiles/AzureOpenAI_Embedding_BaselineGlobalMacro",
        "display_name": null,
        "description": null,
        "text_embedding": "SemanticKernelTextEmbedding",
        "settings": {
            "deployment_name": "embeddings-3-large"
        },
        "configuration_references": {
            "DeploymentName": "FoundationaLLM:Vectorization:SemanticKernelTextEmbeddingService:DeploymentName",
            "Endpoint": "FoundationaLLM:Vectorization:SemanticKernelTextEmbeddingService:Endpoint"
        },
        "created_on": "0001-01-01T00:00:00+00:00",
        "updated_on": "0001-01-01T00:00:00+00:00",
        "created_by": null,
        "updated_by": null,
        "deleted": false
    }
    

11. External orchestration APIs must be configured using the FoundationaLLM:ExternalAPIs configuration namespace. For example, the BaselineTradingGlobalMacro external API has the following configurations:

    • FoundationaLLM:ExternalAPIs:BaselineTradingGlobalMacro:APIUrl
    • FoundationaLLM:ExternalAPIs:BaselineTradingGlobalMacro:APIKey

Note

These entries do not need to be created as part of the deployment process.

12. App Config key namespace that was previously FoundationaLLM:Vectorization:ContentSources:* has been moved to FoundationaLLM:DataSources:*. All existing keys need to be moved to the new namespace.

13. New app config entries required:

    • FoundationaLLM:Attachment:ResourceProviderService:Storage:AuthenticationType
    • FoundationaLLM:Attachment:ResourceProviderService:Storage:AccountName

14. App Config key namespace that was previously FoundationaLLM:Vectorization:ContentSources:* has been moved to FoundationaLLM:DataSources:*. All existing keys need to be moved to the new namespace.

15. The following App Config setting needs to be added/updated as key-values:

• Add FoundationaLLM:APIs:GatekeeperAPI:Configuration:EnableAzureContentSafetyPromptShield
• Add FoundationaLLM:APIs:GatekeeperAPI:Configuration:EnableLakeraGuard
• Add FoundationaLLM:APIs:GatekeeperAPI:Configuration:EnableEnkryptGuardrails
• Rename FoundationaLLM:AzureContentSafety:APIKey in FoundationaLLM:APIs:Gatekeeper:AzureContentSafety:APIKey
• Rename FoundationaLLM:AzureContentSafety:APIUrl in FoundationaLLM:APIs:Gatekeeper:AzureContentSafety:APIUrl
• Rename FoundationaLLM:AzureContentSafety:HateSeverity in FoundationaLLM:APIs:Gatekeeper:AzureContentSafety:HateSeverity
• Rename FoundationaLLM:AzureContentSafety:SelfHarmSeverity in FoundationaLLM:APIs:Gatekeeper:AzureContentSafety:SelfHarmSeverity
• Rename FoundationaLLM:AzureContentSafety:SexualSeverity in FoundationaLLM:APIs:Gatekeeper:AzureContentSafety:SexualSeverity
• Rename FoundationaLLM:AzureContentSafety:ViolenceSeverity in FoundationaLLM:APIs:Gatekeeper:AzureContentSafety:ViolenceSeverity
• Add FoundationaLLM:APIs:Gatekeeper:LakeraGuard:APIKey
• Add FoundationaLLM:APIs:Gatekeeper:LakeraGuard:APIUrl
• Add FoundationaLLM:APIs:Gatekeeper:EnkryptGuardrails:APIKey
• Add FoundationaLLM:APIs:Gatekeeper:EnkryptGuardrails:APIUrl

16. The following Key Vault secret is needed:

    • lakera-guard-api-key
    • enkrypt-guardrails-apikey