Analytics Reference

Reference for the FoundationaLLM analytics API: the query endpoint, request types, response models, metric definitions, and abuse detection settings.

Overview

All analytics queries go through a single action on the FoundationaLLM.Analytics resource provider. The Management Portal dashboards use this action, and you can call it directly through the Management API.

The FoundationaLLM.Analytics resource provider composes results from two domain resource providers:

Domain	Resource provider and type	Data
Conversations	`FoundationaLLM.Conversation/conversationAnalytics`	Conversation sessions and messages in Azure Cosmos DB, and token usage records
Context files	`FoundationaLLM.Context/contextAnalytics`	File record metadata from the Context API (count, size, and MIME type, never file content)

The token usage dashboard (request type 17) is computed directly from token usage records.

Privacy

Analytics never include message content (prompts, completions, or file content). Results contain aggregate counts, token totals, agent names, model deployment names, user principal names (UPNs), timestamps, and conversation and operation identifiers.

Permissions

Analytics queries require the Analytics Reader role. The role is mandatory: other roles, including Owner, do not grant access to analytics on their own.

Analytics Reader grants these actions:

FoundationaLLM.Analytics/analytics/read
FoundationaLLM.Conversation/conversationAnalytics/read
FoundationaLLM.Context/contextAnalytics/read

The user identity is passed through to each domain resource provider, which checks its own action.

Query Endpoint

POST {managementApiUrl}/instances/{instanceId}/providers/FoundationaLLM.Analytics/analytics/query?api-version=2024-02-16
Authorization: Bearer {token}
Content-Type: application/json

Request Body (`AnalyticsRequest`)

{
  "request_type": 0,
  "filters": {
    "start_date": "2026-07-01T00:00:00Z",
    "end_date": "2026-09-30T23:59:59Z",
    "agent_name": "my-agent"
  },
  "options": {
    "include_abuse_detection": false
  }
}

Property	Type	Description
`request_type`	int	The request type. See Request Types.
`filters.start_date`	ISO 8601 datetime	Start of the period. Defaults to 30 days before now.
`filters.end_date`	ISO 8601 datetime	End of the period. Defaults to now.
`filters.agent_name`	string	Agent filter. Optional for `Overview`; required for `AgentSummary`, `AgentToolCombinations`, and `AgentFiles`.
`filters.username`	string	User principal name. Required for `UserSummary`, `UserActivityTimeline`, `UserAbuseRiskScore`, and `UserAbuseIndicators`.
`filters.top_count`	int	Number of users for `TopUsers`. Default 10.
`filters.sort_by`	string	Sort order for `TopUsers`: `Requests` (default), `Tokens`, `Sessions`, or `RiskScore`.
`options.include_abuse_detection`	bool	For `TopUsers`, calculates the abuse risk score and indicators for each returned user.
`options.include_anomaly_detection`	bool	Accepted but not currently used.

A missing required filter returns HTTP 400.

Response

The action returns a resource provider action result whose resource is an AnalyticsResponse:

Property	Description
`request_type`	The request type that was processed.
`generated_at_utc`	When the response was generated.
`data`	The result. Its shape depends on the request type.

Request Types

Value	Name	`data`	Used by the portal
0	`Overview`	`AnalyticsOverview`	Analytics Overview cards
1	`AllAgents`	List of `AnalyticsAgentSummary`	Agent Analytics table
2	`AgentSummary`	`AnalyticsAgentSummary`	No
3	`AgentToolCombinations`	Map of tool combination to count	No
4	`AgentFiles`	`AnalyticsFileSummary`	No
5	`Tools`	List of `AnalyticsToolSummary`	No
6	`Models`	List of `AnalyticsModelSummary`	No
7	`TopUsers`	List of `AnalyticsTopUserSummary`	No
8	`UserSummary`	`AnalyticsUserSummary`	No
9	`UserActivityTimeline`	`AnalyticsUserActivityTimeline`	No
10	`DailyMessageCountsPerAgent`	List of `AnalyticsDailyMessageCount`	Daily Usage by Agent charts
11	`DailyUserCountsPerAgent`	List of `AnalyticsDailyUserCount`	Number of Users per Day by Agent chart
12	`AllUsers`	List of `AnalyticsUserSummarySimple`	User Analytics table
13	`UserAbuseRiskScore`	Risk score and per-factor scores	No
14	`UserAbuseIndicators`	`AnalyticsUserAnalyticsAbuseIndicators`	No
15	`UserAnomalies`	List of `AnalyticsUserAnomaly`	No
16	`DailyActiveUsers`	List of `AnalyticsDailyActiveUserCount`	Active Users per Day chart
17	`TokenUsageDashboard`	`AnalyticsTokenUsageDashboard`	Token Usage Analytics page

Note: AgentSummary, AgentToolCombinations, and Tools are not implemented yet. They return an empty summary (only agent_name is set), an empty map, and an empty list.

Data Sources

Source	Used for
Session documents (`Session` items in the Cosmos DB sessions container)	Conversation counts and active users on the overview, per-user session counts
Message documents (`Message` items)	Agent activity (`sender` = `Agent`, agent name from `senderDisplayName`), user requests (`sender` = `User`), daily counts, tool executions (content artifacts of type tool execution)
Token usage records (`token-usage-record` items in the user data store)	All token totals, operations, model deployments, and the token usage dashboard

Days are calendar days in UTC. Daily series include every day in the range, with zero for days without activity.

Response Models

AnalyticsOverview

Field	Description
`total_conversations`	Sessions with activity in the period. With `agent_name`, distinct sessions that contain a response from that agent.
`total_tokens`	Sum of tokens in token usage records for the period (and agent, if filtered).
`active_users`	Distinct UPNs on sessions with activity. With `agent_name`, distinct UPNs on that agent's responses.
`total_agents`	Distinct agent names on agent responses. With `agent_name`, `1` if the agent responded in the period, otherwise `0`.
`avg_response_time_ms`, `total_tools`, `total_models`	Not populated (always `0`).

AnalyticsAgentSummary

Returned by AllAgents, one item per agent, sorted by agent name. Agents that appear only in token usage records are included.

Field	Description
`agent_name`	The agent name.
`unique_users`	Distinct UPNs on the agent's responses.
`total_conversations`	Distinct sessions that contain a response from the agent.
`total_tokens`	Sum of the agent's tokens in token usage records.
`avg_tokens_per_session`	`total_tokens` divided by the number of conversations in the agent's token usage records (or by `total_conversations` if there are none).
`avg_tools_used`	Tool executions in the agent's responses divided by the number of responses.
`avg_response_time_ms`, `p95_response_time_ms`, `error_rate`, `most_used_tools`, `avg_files_per_conversation`, `avg_rounds_per_conversation`	Not populated.

AnalyticsDailyMessageCount and AnalyticsDailyUserCount

Field	Description
`date`	The UTC day.
`agent_counts`	Map of agent name to the number of agent responses that day (`DailyMessageCountsPerAgent`), or to the number of distinct users who received a response from the agent that day (`DailyUserCountsPerAgent`).

AnalyticsDailyActiveUserCount

Field	Description
`date`	The UTC day.
`count`	Distinct UPNs on messages that day.

AnalyticsUserSummarySimple

Returned by AllUsers, one item per user, sorted by username.

Field	Description
`username`	The UPN.
`total_conversations`	Distinct sessions with messages for the user (or, if none, distinct conversations in the user's token usage records).
`total_messages`	Messages for the user in the period. Not shown in the portal.
`operation_count`	Distinct operation IDs in the user's token usage records.
`total_tokens`	Sum of the user's tokens in token usage records.

AnalyticsTokenUsageDashboard

Field	Description
`total_tokens`	Sum of tokens in all token usage records for the period.
`total_users`, `total_agents`, `total_models`	Distinct users, agents, and model deployments in the records.
`agents`	Per agent: `agent_name`, `total_tokens`, `unique_users`, `total_conversations`.
`users`	Per user: `username`, `total_tokens`, `total_conversations`, `operation_count`.
`models`	Per model deployment: `model_deployment_name` (`Unknown` when the record has none), `total_tokens`, `agent_count`.

Each list is sorted by total_tokens, highest first. The portal resolves model_deployment_name to an AI model name by matching the AI model's deployment name.

AnalyticsModelSummary

Returned by Models, one item per model deployment in token usage records.

Field	Description
`model_name`, `deployment_name`	The model deployment name.
`total_tokens`	Tokens consumed on the deployment.
`agent_count`, `usage_count`	Distinct agents with token usage on the deployment.
`avg_response_time_ms`, `avg_tokens_per_completion`, `cost_per_request`, `error_rate`	Not populated.

AnalyticsTopUserSummary

Field	Description
`username`, `upn`	The UPN.
`total_requests`	User messages in the period.
`total_tokens`	Sum of the user's tokens in token usage records.
`active_sessions`	Sessions with activity for the user.
`last_activity`	The latest session creation time.
`abuse_risk_score`, `abuse_indicators`	Populated only when `include_abuse_detection` is `true`: the risk score and the list of indicator types.
`avg_response_time_ms`, `error_rate`, `agents_used`	Not populated.

AnalyticsUserSummary

Field	Description
`username`	The UPN.
`total_requests`	User messages in the period.
`total_tokens`	Sum of the user's tokens in token usage records.
`total_sessions`	Sessions with activity for the user.
`active_days`	Distinct days with messages for the user.
`avg_session_length`	Average number of messages per session.
`avg_response_time_ms`, `error_rate`, `agents_used`, `peak_usage_hour`, `peak_usage_day`	Not populated.

AnalyticsUserActivityTimeline

Field	Description
`username`	The UPN.
`entries`	One entry per operation in the user's token usage records, oldest first: `timestamp`, `session_id`, `operation_id`, `tokens`, `agent_name`.

AnalyticsFileSummary

Returned by AgentFiles for one agent, from Context API file records.

Field	Description
`total_files`	Files uploaded.
`total_storage_bytes`	Total size of the files.
`avg_file_size_bytes`, `median_file_size_bytes`	Average and median file size.
`avg_files_per_conversation`	Average files per conversation.
`file_type_distribution`	Map of MIME type to count.

Abuse Indicators and Anomalies

UserAbuseIndicators returns username, risk_score, and four lists of indicators: volume_indicators, temporal_indicators, behavioral_indicators, and resource_indicators. Each indicator has type, description, severity, detected_at, and metrics.

UserAnomalies returns a list of anomalies with username, anomaly_type, description, severity, detected_at, and anomaly_data.

Abuse Detection

Abuse detection evaluates one user over the requested period.

Indicators

Category	Type	Condition	Severity
Volume	`ExtremeRequestRate`	Average user requests per hour over the period above `ExtremeRequestRateThreshold`	High
Volume	`HighRequestRate`	Average user requests per hour above `HighRequestRateThreshold`	Medium
Volume	`ExtremeTokenConsumption`	Average tokens per hour above 100,000	High
Volume	`HighTokenConsumption`	Average tokens per hour above 50,000	Medium
Temporal	`RapidFireRequests`	More than `RapidFireThreshold` operations within `RapidFireWindowMinutes`	High
Temporal	`ContinuousUsage`	Activity without a gap longer than `MaxInactivityGapMinutes` for more than `ContinuousUsageHours`	Medium
Behavioral	`HighErrorRate`	Error rate above `HighErrorRateThreshold`	Medium
Behavioral	`AgentHopping`	More than `AgentHoppingThreshold` agents per `AgentHoppingWindowMinutes`	Medium

The user error rate and agent count are not populated yet, so HighErrorRate and AgentHopping are not raised today. No resource indicators are produced.

Risk Score

Each category gets a score from the severities of its indicators (High = 20, Medium = 10, capped at 100). The risk score (0 to 100) is the weighted sum:

Factor	Weight
Volume (request rate indicators)	25%
Token consumption	20%
Error rate	15%
Temporal anomalies	15%
Behavioral anomalies	15%
Resource exhaustion	10%

Anomalies

UserAnomalies evaluates the top 100 users by requests and reports:

HighRiskUser when the risk score is 61 or higher (severity Critical at 81 or higher, otherwise High)
VolumeAnomaly when the user has High or Critical volume indicators

Settings

The thresholds are read from Azure App Configuration:

Key	Default
`FoundationaLLM:Analytics:AbuseDetection:HighRequestRateThreshold`	100
`FoundationaLLM:Analytics:AbuseDetection:ExtremeRequestRateThreshold`	500
`FoundationaLLM:Analytics:AbuseDetection:RapidFireThreshold`	20
`FoundationaLLM:Analytics:AbuseDetection:RapidFireWindowMinutes`	1
`FoundationaLLM:Analytics:AbuseDetection:ContinuousUsageHours`	20
`FoundationaLLM:Analytics:AbuseDetection:MaxInactivityGapMinutes`	30
`FoundationaLLM:Analytics:AbuseDetection:AgentHoppingThreshold`	10
`FoundationaLLM:Analytics:AbuseDetection:AgentHoppingWindowMinutes`	60
`FoundationaLLM:Analytics:AbuseDetection:HighErrorRateThreshold`	30

Table of Contents