FoundationaLLM Concepts

This page introduces the core FoundationaLLM concepts that you work with in the Management Portal and the Management API.

Overview

The following mindmap provides a high-level overview of the core FoundationaLLM concepts.

mindmap
    root((FoundationaLLM))
        (Authorization)
            Secret Key
            Role Based Access Control
            Policy Based Access Control
            Security Principal
                Virtual Security Principal
                    Virtual User
                    Virtual Security Group
        (Agent)
            Agent Workflow
            Agent Tool
            Agent Shielding
            Agent Access Token
            Agent Virtual Security Group
        (Chargeback)
        (Data Pipeline)
        (Plugin)
            Plugin Package
            Plugin
        (Prompt)
        (Quota)
            Rate Limit Quota
            Usage Limit Quota
            Resource Limit Quota
        (Resource Provider)
        (Tool)
        (Workflow)
            Workflow Host

• Authorization
  • Secret Key
  • Role Based Access Control
  • Policy Based Access Control
  • Security Principal
• Agent
  • Agent Workflow
  • Agent Tool
  • Agent Shielding
  • Agent Access Token
  • Agent Virtual Security Group
• Chargeback
• Data Pipeline
• Plugin
• Prompt
• Quota
• Usage Analytics
• Resource Provider
• Tool
• Workflow

Authorization

Secret Key

A secret key is a platform-generated credential that the Authorization API stores and validates on behalf of another resource. Agent access tokens are the main example: each token is a secret key bound to one agent. The full key value is shown only once, when it is created.

Role Based Access Control

Role-based access control (RBAC) restricts access to resources through role assignments. A role assignment grants a role definition (for example, Reader or Contributor) to a security principal on a scope (the whole instance, a resource type, or a single resource). Role assignments are resources of the FoundationaLLM.Authorization resource provider. In the Management Portal, manage them under Security > Access Control, or from the Access Control button on an individual resource.

Policy Based Access Control

Policy-based access control (PBAC) complements RBAC. A policy assignment binds a security principal to a policy definition on a resource type (not on individual resources). The built-in User Principal Name (UPN) match policy lets covered principals work with the resources of that type that belong to their own UPN, such as their conversations, attachments, and user profile. In the Management Portal, manage policy assignments under Security > Policies.

Security Principal

A security principal is an entity that can be authenticated and authorized to access resources. The Management Portal supports the User, Group, and ServicePrincipal principal types in role assignments.

Virtual Security Principal

A virtual security principal is created by the FoundationaLLM platform rather than by Microsoft Entra ID. Virtual principals represent callers that do not sign in with Entra ID.

• A virtual user is the identity assigned to a caller that presents a valid agent access token. Its user principal name has the form aat_{agentName}_{tokenId}@foundationallm.internal_.
• A virtual security group is a platform-generated group identifier. Every agent has one, and every virtual user created from that agent's access tokens is a member of it.

Agent

An agent is the resource that defines a conversational AI experience: its main model and prompt, workflow, tools, shielding, conversation settings, and User Portal experience settings. For the full structure, see Agents and Workflows.

Agent Workflow

An agent workflow drives how the agent calls its main model and its tools to produce a response. The FoundationaLLM function-calling workflow (FoundationaLLMFunctionCallingWorkflow in the foundationallm_agent_plugins package) is the workflow used by the basic agent template and supports multi-round tool use (Max rounds) and tool escalation. See Workflow.

Agent Tool

An agent tool is a tool from the platform tool catalog that has been added to an agent, together with the models, prompts, resources, and property values that the agent uses for that tool. See Tools.

Agent Shielding

Agent shielding screens content with shielding plugins. Pre-completion plugins screen the user prompt before the model is called, and post-completion plugins screen the model response before it is returned. Plugins run in the order in which they are listed. Shielding replaces the earlier Gatekeeper options. Agents that still store the legacy gatekeeper_settings.options continue to run, because the platform maps those options to the equivalent shielding plugins.

Agent Access Token

An agent access token lets a client use an agent without signing in with Entra ID. The client sends the token in the X-AGENT-ACCESS-TOKEN header to the Core API, and the platform runs the request as a virtual user. See Agent Access Tokens.

Agent Virtual Security Group

The agent virtual security group is the virtual security group that contains all virtual users created from the agent's access tokens. The platform generates its identifier when the agent is first saved and grants it the Reader role on the agent at that time.

Chargeback

Chargeback tracks resource usage so that costs can be attributed to cost centers. Agents, prompts, and other resources have an optional cost_center property. In the agent editor, set it on the Agent Configuration tab.

Data Pipeline

A FoundationaLLM data pipeline extracts data from a data source, transforms it for LLM-based workloads (for example, text extraction, text partitioning, entity and relationship extraction, embedding, and knowledge graph construction), and loads it into a target system such as a vector store or knowledge store. Data pipelines are managed in the Management Portal under Data > Data Pipelines or programmatically with the Management API.

FoundationaLLM data pipelines address the following requirements for enterprise-grade generative AI workloads:

• Scalability - handle anything from a few documents to millions of documents or records, and scale horizontally.
• Flexibility - use a modular structure that supports complex processing, parallel work, and reusable components.
• Local and global processing - support per-item processing (for example, text extraction and partitioning) and global processing (for example, assembling knowledge graphs).
• Parameterization - accept parameters at run time, so a single pipeline definition can serve many scenarios (for example, all user file uploads in the User Portal).
• Extensibility - implement every stage as a plugin, so you can add new stages or replace existing ones.

For more details, see Data Pipelines.

Plugin

A plugin is a software component that extends the FoundationaLLM platform. Plugins are delivered in plugin packages, which are managed in the Management Portal under Platform > Packages or with the Management API (FoundationaLLM.Platform resource provider, pluginPackages and plugins resource types).

Plugin Package

A plugin package is the unit of versioning and deployment for one or more plugins.

Platform	Package type	Description
Python	Python wheel	Contains agent workflow and agent tool plugins. The package includes a foundationallm_manifest.json file that lists its plugin managers, agent workflows, and agent tools. Python packages appear as options for Workflow package name in the agent editor.
.NET	NuGet package	Contains data source, data pipeline stage, content text extraction, content text partitioning, and shielding plugins.

Plugin Categories

Platform	Category	Description
Python	Agent workflow	Implements an agent workflow that runs in the LangChain API. For example, foundationallm_agent_plugins provides FoundationaLLMFunctionCallingWorkflow.
Python	Agent tool	Implements a tool that an agent can call from a workflow that runs in the LangChain API.
.NET	Data Source	Retrieves data from an external system and makes it available to data pipelines.
.NET	Data Pipeline Stage	Implements a stage of a data pipeline.
.NET	Content Text Extraction	Extracts text from binary content (for example, PDF, DOCX, and images).
.NET	Content Text Partitioning	Splits text into segments using a strategy such as token-based or semantic partitioning.
.NET	Shielding	Screens content for agent shielding (pre-completion and post-completion) and for tool output shielding.

For more details, see Plugins and Plugin Packages.

Prompt

A FoundationaLLM prompt is a text template used to build completion requests. Agents use prompts in two main contexts:

• Agent workflow - The main workflow prompt defines the agent's persona and instructions. Additional workflow prompts (router, files, escalation, and final prompts) help the workflow select tools, use files, and build the final answer.
• Agent tool - Tool prompts describe the tool to the workflow (router prompts) or instruct the tool's internal logic (main prompts).

Prompts support variables, which are resolved when the completion request is built. For more details, see Prompts and Prompt Variables.

Quota

A FoundationaLLM quota defines a limit on how clients use the platform. Quotas are resources of the FoundationaLLM.Quota resource provider and are managed in the Management Portal under Quotas > Quotas. The following quota types are supported:

Quota type	What it limits	Example
Rate limit	Requests to a Core API controller within a time window, optionally scoped to one agent.	100 requests per minute per user to the Completions controller.
Usage limit	Token usage per month, for all agents or for one agent.	1,000,000 tokens per month per user for a given agent.
Resource limit	The number of agents or sandboxes that a user can own.	At most 5 agents per user.

Quotas can be shared (one limit for everyone) or individual (a separate limit per user identifier or per user principal name), and can optionally apply only to a specific user, group, service principal, or agent access token.

For more details, see Quotas and API and Token Limits.

Usage Analytics

FoundationaLLM usage analytics provides insights into platform usage, agent performance, and token consumption through aggregated metrics. In the Management Portal, the dashboards are under Analytics. For more details, see the Analytics Reference.

Resource Provider

A resource provider is the platform component that manages a family of resource types. All FoundationaLLM resources (agents, prompts, AI models, data pipelines, plugins, quotas, and so on) are created, read, updated, and deleted through resource providers, both by the Management Portal and by the Management API. For the list of resource providers and resource types, see Resource Management.

Tool

A tool is a software component that an agent workflow can invoke to perform a specific task, such as searching knowledge, running code, generating images, or calling an MCP server. Tools are resources of the FoundationaLLM.Agent resource provider (tools, grouped by toolFamilies and toolSets). Manage the tool catalog under Agents > Tools and MCP servers under Agents > MCP Servers. See Tools and MCP Servers.

Workflow

A workflow is the logic that an agent follows to answer a request: calling the main model, selecting and running tools, and composing the final response. Workflow resources (FoundationaLLM.Agent/workflows) define the workflow types that you can select on the agent editor's Workflow tab. Python workflow plugins implement the logic. For details on workflow types, max rounds, and tool escalation, see Agents and Workflows.

Workflow Host

A workflow host is the service that runs agent workflows and the tools they call. The Management Portal offers LangChain (the LangChain API, a Python service that loads workflow and tool plugins from Python plugin packages) as the workflow host. LangChain is also used when a workflow does not specify a host.

Related Topics

• Agents and Workflows
• Agent Access Tokens
• Prompts
• Prompt Variables
• Resource Management
• API and Token Limits
• Configuration Reference