Copilot Usage Report APIs Available

Tony Redmond — Fri, 13 Sep 2024 07:00:00 +0000

Copilot Usage Reports Weak on Detail

Announced in message center notification MC877369 (29 August 2024, Microsoft 365 roadmap item 396562), the Microsoft Graph beta usage reports API now includes support for Copilot for Microsoft 365 tenant usage data. All tenants that use Copilot for Microsoft 365 should now have access to the usage data.

Microsoft says that the availability of this information will “facilitate the creation of customized reporting and analytics,” but the fact is that the data exposed by the API is bare-bones. On the upside, the data matches what’s available in the report section of the Microsoft 365 admin center (Figure 1).

Tenant-level summary of Copilot-enabled (licensed) users and active users.
Adoption trend (tenant summary) over time.
Last activity date for Copilot interaction in different apps for each user.

Figure 1: Copilot usage reports in the Microsoft 365 admin center

Accounts accessing the Graph data must have a Copilot for Microsoft 365 license.

User Count Summary

The user count summary report returns a count of the user accounts licensed for Copilot for Microsoft 365 (enabled users) and a count of the users with an active interaction with Copilot in each app during the reporting period (7, 30, 90, or 180 days). Unsurprisingly, when someone is enabled for Copilot in one app, they’re usually enabled for all:

Teams
Outlook (classic, new Outlook for Windows, OWA).
Excel.
PowerPoint.
Copilot Graph-grounded chat (aka Copilot Chat).
OneNote.
Loop.

$Uri = "https://graph.microsoft.com/beta/reports/getMicrosoft365CopilotUserCountSummary(period='D90')"
$Data = Invoke-GraphRequest -Uri $Uri -Method Get
$Data.value.adoptionByProduct

Name                           Value
----                           -----
loopEnabledUsers               100
reportPeriod                   90
oneNoteActiveUsers             3
wordEnabledUsers               100
powerPointEnabledUsers         100
microsoftTeamsActiveUsers      97
oneNoteEnabledUsers            100
excelActiveUsers               43
loopActiveUsers                2
copilotChatEnabledUsers        100
outlookEnabledUsers            100
anyAppEnabledUsers             100
anyAppActiveUsers              97
microsoftTeamsEnabledUsers     100
excelEnabledUsers              100
wordActiveUsers                61
powerPointActiveUsers          12
copilotChatActiveUsers         73
outlookActiveUsers             18

User Activity Detail

This report is the most interesting because it details the last activity date for Copilot interaction by users with each of the various Copilot-enabled apps. In addition, the last activity date for any Copilot interaction with any of the supported apps is published (lastActivityDate). An array (value) holds a separate usage report for each Copilot-enabled account.

The user principal name and display name is obfuscated if the tenant data privacy control is enabled. In the following extract, we see that the user has never used Copilot for Loop and OneNote and hasn’t used Copilot with PowerPoint since April 11, 2024:

$Uri = https://graph.microsoft.com/beta/reports/getMicrosoft365CopilotUsageUserDetail(period='D90')
$Data = Invoke-GraphRequest -Uri $Uri -Method Get
$Data.value[0]

Name                           Value
----                           -----
copilotActivityUserDetailsByP… {System.Collections.Hashtable}
microsoftTeamsCopilotLastActi… 2024-09-05
outlookCopilotLastActivityDate 2024-08-29
lastActivityDate               2024-09-05
reportRefreshDate              2024-09-08
excelCopilotLastActivityDate   2024-09-05
loopCopilotLastActivityDate
oneNoteCopilotLastActivityDate
copilotChatLastActivityDate    2024-09-05
powerPointCopilotLastActivity… 2024-04-11
userPrincipalName              Tony.Redmond@office365itpros.com
displayName                    Tony Redmond
wordCopilotLastActivityDate    2024-09-05

Adoption Trend over Time

This report returns an array called adoptionByDate with entries for each day during the reporting period (7, 30, 90, or 180 days). The purpose of the report is to track progress in Copilot adoption over time and to note if any specific action had an effect. For instance, you might run an education campaign to teach users how to generate effective results using Copilot in Excel. Over the weeks following the campaign, you’d expect to see the number of users who use Copilot in Excel to grow.

$Uri = "https://graph.microsoft.com/beta/reports/getMicrosoft365CopilotUserCountTrend(period='D90')"
$Data = Invoke-GraphRequest -Uri $Uri -Method Get
$Data.Value.copilotActivityUserDetailsByPeriod

reportDate                     2024-06-17
excelEnabledUsers              100
wordActiveUsers                51
powerPointActiveUsers          11
copilotChatActiveUsers         66
outlookActiveUsers             15
loopEnabledUsers               100
oneNoteActiveUsers             1
wordEnabledUsers               100
powerPointEnabledUsers         100
microsoftTeamsActiveUsers      86
oneNoteEnabledUsers            1
excelActiveUsers               21
loopActiveUsers                1
copilotChatEnabledUsers        100
outlookEnabledUsers            100
anyAppEnabledUsers             100
anyAppActiveUsers              86
microsoftTeamsEnabledUsers     100

Track Copilot Activity Using Audit Records instead of Copilot Usage Reports

Although it’s nice to have Copilot usage reports included in the Graph API, the information exposed isn’t very informative in terms of how people use Copilot. The data tells you that someone used Copilot in an app during a day. At least, they clicked a Copilot button. The information doesn’t reveal any more insight than that. Any enterprise who invests large sums of money in expensive Copilot for Microsoft 365 licenses will find a dearth of detail here in terms of understanding whether the investment is justified. In many cases, you will be better off analyzing the audit records captured for Copilot interactions to figure out what’s really going on.