Copilot Usage Reports Weak on Detail

Announced in message center notification MC877369 (29 August 2024, Microsoft 365 roadmap item 396562), the Microsoft Graph beta usage reports API now includes support for Copilot for Microsoft 365 tenant usage data. All tenants that use Copilot for Microsoft 365 should now have access to the usage data.

Microsoft says that the availability of this information will “facilitate the creation of customized reporting and analytics,” but the fact is that the data exposed by the API is bare-bones. On the upside, the data matches what’s available in the report section of the Microsoft 365 admin center (Figure 1).

• Tenant-level summary of Copilot-enabled (licensed) users and active users.
• Adoption trend (tenant summary) over time.
• Last activity date for Copilot interaction in different apps for each user.

Accounts accessing the Graph data must have a Copilot for Microsoft 365 license.

User Count Summary

The user count summary report returns a count of the user accounts licensed for Copilot for Microsoft 365 (enabled users) and a count of the users with an active interaction with Copilot in each app during the reporting period (7, 30, 90, or 180 days). Unsurprisingly, when someone is enabled for Copilot in one app, they’re usually enabled for all:

• Teams
• Outlook (classic, new Outlook for Windows, OWA).
• Excel.
• PowerPoint.
• Copilot Graph-grounded chat (aka Copilot Chat).
• OneNote.
• Loop.

$Uri = "https://graph.microsoft.com/beta/reports/getMicrosoft365CopilotUserCountSummary(period='D90')"
$Data = Invoke-GraphRequest -Uri $Uri -Method Get
$Data.value.adoptionByProduct

Name                           Value
----                           -----
loopEnabledUsers               100
reportPeriod                   90
oneNoteActiveUsers             3
wordEnabledUsers               100
powerPointEnabledUsers         100
microsoftTeamsActiveUsers      97
oneNoteEnabledUsers            100
excelActiveUsers               43
loopActiveUsers                2
copilotChatEnabledUsers        100
outlookEnabledUsers            100
anyAppEnabledUsers             100
anyAppActiveUsers              97
microsoftTeamsEnabledUsers     100
excelEnabledUsers              100
wordActiveUsers                61
powerPointActiveUsers          12
copilotChatActiveUsers         73
outlookActiveUsers             18

User Activity Detail

This report is the most interesting because it details the last activity date for Copilot interaction by users with each of the various Copilot-enabled apps. In addition, the last activity date for any Copilot interaction with any of the supported apps is published (lastActivityDate). An array (value) holds a separate usage report for each Copilot-enabled account.

The user principal name and display name is obfuscated if the tenant data privacy control is enabled. In the following extract, we see that the user has never used Copilot for Loop and OneNote and hasn’t used Copilot with PowerPoint since April 11, 2024:

$Uri = https://graph.microsoft.com/beta/reports/getMicrosoft365CopilotUsageUserDetail(period='D90')
$Data = Invoke-GraphRequest -Uri $Uri -Method Get
$Data.value[0]

Name                           Value
----                           -----
copilotActivityUserDetailsByP… {System.Collections.Hashtable}
microsoftTeamsCopilotLastActi… 2024-09-05
outlookCopilotLastActivityDate 2024-08-29
lastActivityDate               2024-09-05
reportRefreshDate              2024-09-08
excelCopilotLastActivityDate   2024-09-05
loopCopilotLastActivityDate
oneNoteCopilotLastActivityDate
copilotChatLastActivityDate    2024-09-05
powerPointCopilotLastActivity… 2024-04-11
userPrincipalName              Tony.Redmond@office365itpros.com
displayName                    Tony Redmond
wordCopilotLastActivityDate    2024-09-05

Adoption Trend over Time

This report returns an array called adoptionByDate with entries for each day during the reporting period (7, 30, 90, or 180 days). The purpose of the report is to track progress in Copilot adoption over time and to note if any specific action had an effect. For instance, you might run an education campaign to teach users how to generate effective results using Copilot in Excel. Over the weeks following the campaign, you’d expect to see the number of users who use Copilot in Excel to grow.

$Uri = "https://graph.microsoft.com/beta/reports/getMicrosoft365CopilotUserCountTrend(period='D90')"
$Data = Invoke-GraphRequest -Uri $Uri -Method Get
$Data.Value.copilotActivityUserDetailsByPeriod

reportDate                     2024-06-17
excelEnabledUsers              100
wordActiveUsers                51
powerPointActiveUsers          11
copilotChatActiveUsers         66
outlookActiveUsers             15
loopEnabledUsers               100
oneNoteActiveUsers             1
wordEnabledUsers               100
powerPointEnabledUsers         100
microsoftTeamsActiveUsers      86
oneNoteEnabledUsers            1
excelActiveUsers               21
loopActiveUsers                1
copilotChatEnabledUsers        100
outlookEnabledUsers            100
anyAppEnabledUsers             100
anyAppActiveUsers              86
microsoftTeamsEnabledUsers     100

Track Copilot Activity Using Audit Records instead of Copilot Usage Reports

Although it’s nice to have Copilot usage reports included in the Graph API, the information exposed isn’t very informative in terms of how people use Copilot. The data tells you that someone used Copilot in an app during a day. At least, they clicked a Copilot button. The information doesn’t reveal any more insight than that. Any enterprise who invests large sums of money in expensive Copilot for Microsoft 365 licenses will find a dearth of detail here in terms of understanding whether the investment is justified. In many cases, you will be better off analyzing the audit records captured for Copilot interactions to figure out what’s really going on.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Keep Group Display Names Short to Avoid Problems

A recent discussion revealed that Graph API requests against the Groups endpoint for groups with display names longer than 120 characters generate an error. As you might know, The Groups Graph API supports group display names up to a maximum of 256 characters, so an error occurring after 120 seems bizarre. Then again, having extraordinarily long group names is also bizarre (Figure 1).

Group display names longer than 30 or so characters make it difficult for clients to list groups or teams, which is why Microsoft’s recommendation for Teams clients says that between 30 and 36 characters is a good limit for a team name. Using very long group names also creates formatting and layout issues when generating output like the Teams and Groups activity report, especially if only one or two groups have very long names.

Replicating the Problem

In any case, here’s an example. After creating a group with a very long name, I populated a variable with the group’s display name (146 characters). I then created a URI to request the Groups endpoint to return any Microsoft 365 group that has the display name. Finally, I executed the Invoke-MgGraphRequest cmdlet to issue the request, which promptly failed with a “400 bad request” error:

$Name = "O365Grp-Team with an extraordinary long name that makes it much more than 120 characters so that we can have some fun with it  with Graph requests"
$Uri = "https://graph.microsoft.com/v1.0/groups?`$filter= displayName eq '${name}'"
$Data = Invoke-MgGraphRequest -Uri $Uri -Method Get

Invoke-MgGraphRequest: GET Invoke-MgGraphRequest: GET https://graph.microsoft.com/v1.0/groups?$filter=%20displayName%20eq%20'O365Grp-Team%20with%20an%20extraordinary%20long%20name%20that%20makes%20it%20much%20more%20than%20120%20characters%20so%20that%20we%20can%20have%20some%20fun%20with%20it%20%20with%20Graph%20requests'
HTTP/1.1 400 Bad Request
Cache-Control: no-cache

The Get-MgGroup cmdlet also fails. This isn’t at all surprising because the Graph SDK cmdlets run the underlying Graph API requests, so if those requests fail, the cmdlets can’t apply magic to make everything work again:

Get-MgGroup -Filter "displayName eq '$Name'"

Get-MgGroup_List: Unsupported or invalid query filter clause specified for property 'displayName' of resource 'Group'.

The same happens if you try to use the Get-MgTeam cmdlet from the Microsoft Graph PowerShell SDK.

Get-MgTeam -Filter "displayName eq '$Name'"

Get-MgTeam_List: Unsupported or invalid query filter clause specified for property 'displayName' of resource 'Group'.
Status: 400 (BadRequest)
ErrorCode: BadRequest
Date: 2023-10-06T04:53:39
 

The Workaround for Group Display Name Errors

But here’s the thing. The Get-MgGroup cmdlet (and the underlying Graph API request) work if you add the ConsistencyLevel header and an output variable to accept the count of returned items. The presence of the header makes the request into an advanced query against Entra ID.

Get-MgGroup -ConsistencyLevel Eventual -Filter "displayName eq '$Name'" -CountVariable X | Format-Table DisplayName
 
DisplayName
-----------
O365Grp-Team with an extraordinary long name that makes it much more than 120 characters so that we can have some fun …

Oddly, the Get-MgTeam cmdlet doesn’t support the ConsistencyLevel header so this workaround isn’t possible using this cmdlet. Given that Teams (the app) finds its teams through Graph requests, this inconsistency is maddening, and it’s probably due to a flaw in the metadata read by the ‘AutoREST’ process Microsoft runs regularly to generate the SDK cmdlets and build new versions of the SDK modules.

None of the Teams clients that I’ve tested have any problem displaying team names longer than 120 characters, so I suspect that the clients do the necessary magic when fetching lists of teams.

Inconsistency in Entra ID Admin Center

The developers of the Entra ID admin center must know about the 120 character limit (and not about the workaround) because they restrict group names (Figure 2).

A StackOverflow thread from 2017 reported that attempts to use the Graph to create new groups with display names longer than 120 characters resulted in errors. However, it’s possible to now use cmdlets like New-MgGroup to create groups with much longer names.

Given that the Groups Graph API allows for 256 characters, it’s yet another oddity that the Entra ID admin center focuses on a lower limit – unless the developers chose to emphasize to administrators that it’s a really bad idea to use overly long group names.

Time to Update SDK Foibles

I shall have to add this issue to my list of Microsoft Graph PowerShell SDK foibles (aka, things developers should know before they try coding PowerShell scripts using the SDK). The fortunate thing is that you’re unlikely to meet this problem in real life. At least, I hope that you are. And if you do, you’ll know what to do now.

Make sure that you’re not surprised about changes that appear inside Microsoft 365 applications or the Graph by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers stay informed.

Old Limit with SignInActivity was 999 – New Limit for Azure AD Accounts is 120

Because it retrieves details of Azure AD accounts, the List Users API is one of the most heavily used of the Microsoft Graph APIs. It also underpins the Get-MgUser cmdlet from the Microsoft Graph PowerShell SDK. Microsoft generates the cmdlet from the API using a process called AutoRest, which means that changes made to the API show up soon afterward in the cmdlet.

I’ve documented some of the issues that developers must deal with when coding with the cmdlets from the Microsoft Graph PowerShell SDK. The cmdlets have been stable recently, which is a relief because tenants are migrating scripts from the Azure AD and MSOL modules. However, last week an issue erupted in a GitHub discussion that caused a lot of disruption.

In a nutshell, if you use List Users to fetch Azure AD accounts and include the SignInActivity property, the API limits the page size for results to 120 items. Calls made without specifying SignInActivity can set the page size to be anything up to 999 items.

An Unannounced Change

To help manage demand on the service, all Graph API requests limit the number of items that they return. To retrieve all matching items for a request, developers must fetch pages of results until nothing remains. When a developer knows that large numbers of items must be fetched, they often increase the page size to reduce the number of requests.

Microsoft didn’t say anything about the new restriction on requests that fetch Azure AD account data with sign-in activity. Developers only discovered the problem when programs and scripts failed. I first learned of the issue when some of the users of the Office 365 for IT Pros GitHub repository reported that a Graph request which included a $top query parameter to increase the page size to 999 items failed. For example:

$uri = "https://graph.microsoft.com/beta/users?`$select=displayName,userPrincipalName,mail,id,CreatedDateTime,signInActivity,UserType&`$top=999"
[array]$Data = Invoke-RestMethod -Method GET -Uri $Uri -ContentType "application/json" -Headers $Headers
Invoke-RestMethod : The remote server returned an error: (400) Bad Request.
At line:1 char:16
+ ... ray]$Data = Invoke-RestMethod -Method GET -Uri $Uri -ContentType "app ...
+                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest)
   [Invoke-RestMethod], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.I

As shown in Figure 2, testing with the Get-MgUser cmdlet revealed some more information in the error (“Cannot query data for more than 120 users at a time”). This was the first time I learned about a query limit:

According to a response reported in the GitHub discussion, Microsoft support reported

“The PG have confirmed that this endpoint will be transitioning from beta to General Availability (GA).

As part of this transition, changes to its behavior has been made, this includes not requesting more than 120 results per call. They recommend requesting less than 120 results per call, which can be done by setting the top parameter to, say 100.”

It’s likely that Microsoft made the change because retrieving sign-in activity data for Azure AD accounts is an expensive operation. Reducing the page size to 120 possibly makes it easier to process a request than if it asked for 999 items.

Beta Version of List Users Moving to Production

When the product group (PG) says that the endpoint is transitioning from beta to GA, it means that instead of needing to use https://graph.microsoft.com/beta/users to access sign-in activity, the data will be available through https://graph.microsoft.com/V1.0/users. If you use the Microsoft Graph PowerShell SDK, you won’t have to run the Select-MgProfile cmdlet to choose the beta endpoint. Moving the beta version of the API to the production endpoint is a good thing because there are many other account properties now only available through the beta endpoint (like license assignments).

If you use the Microsoft Graph PowerShell SDK, the Get-MgUser cmdlet is unaffected by the change if you specify the All parameter. This is because the cmdlet handles pagination internally and fetches all pages automatically without the need to specify a page size. For instance, this works:

$AccountProperties = @( ‘Id’, ‘DisplayName’, ‘SignInActivity’)
[array]$Users = Get-MgUser -All -Property $AccountProperties | Select-Object $AccountProperties

Moving to Production

Although it’s good that Microsoft is (slowly) moving the beta versions of the List Users API towards production, it’s a pity that they introduced a change that broke so many scripts and programs without any warning. At worse, this so exhibits a certain contempt for the developer community. At best, it’s a bad sign when communication with the developer community is not a priority. That’s just sad.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Pay as You Go Model for Microsoft 365 APIs

About fifteen months ago, Microsoft introduced the notion of metered APIs where those who consumed the APIs would pay for the resources they consume. The pay-as-you-go (PAYG) model evolved further in July 2022 when Microsoft started to push ISVs to use the new Teams export API instead of Exchange Web Services (EWS) for their backup products. The Teams export API is a metered API and is likely to the test case to measure customer acceptance of the PAYG model.

So far, I haven’t heard many positive reactions to the development. Some wonder how Microsoft can force ISVs to use an API when they don’t know how high the charges metering will rack up. Others ask how Microsoft can introduce an export API for backup when they don’t have an equivalent import API to allow tenants to restore data to Teams. I don’t understand this either as it seems logical to introduce export and import capabilities at the same time. We live in interesting times!

PAYG with Syntex Backup

To be fair to Microsoft, they plan to go down the same PAYG route with the new backup service they plan to introduce in 2023 as part of the Syntex content management suite. Customers will have to use an Azure subscription to pay for backups of SharePoint Online, OneDrive for Business, and Exchange Online (so far, Microsoft is leaving Teams backup to ISVs).

All of which brings me to the December 2 post from the Microsoft Graph development team where Microsoft attempts to describe what they’re doing with different Microsoft 365 APIs. Like many Microsoft texts, too many words disguise the essential facts of the matter.

Three Microsoft 365 API Tiers

Essentially, Microsoft plans to operate three Microsoft 365 API tiers:

• Standard: The regular Graph-based and other APIs that allow Microsoft 365 tenants to access and work with their data.
• High-capacity: Metered APIs that deal with high-volume operations like the streaming of data out of Microsoft 365 for backups or the import of data into Microsoft 365.
• Advanced: APIs developed by Microsoft to deliver new functionality. Microsoft points to Azure Communications Services as an example. These APIs allow developers to add the kind of communication options that are available in Teams to their applications.

My reading of the situation is that Microsoft won’t charge for standard APIs because this would interfere with customer access to their data. Microsoft says that standard APIs will remain the default endpoint.

However, Microsoft very much wants to charge for high-capacity APIs used by “business-critical applications with high usage patterns.” The logic here is that these APIs strain the resources available within the service. To ensure that Microsoft can meet customer expectations, they need to deploy more resources to meet the demand and someone’s got to pay for those resources. By using a PAYG model, Microsoft will charge for actual usage of resources.

Microsoft also wants customers to pay for advanced APIs. In effect, this is like an add-on license such as Teams Premium. If you want to use the bells and whistles enabled by an advanced API, you must pay for the privilege. It’s a reasonable stance.

Problem Areas for Microsoft 365 APIs

I don’t have a problem with applying a tiered model for APIs, especially if the default tier continues with free access. The first problem here is in communications, where Microsoft has failed to sell their approach to ISVs and tenants. The lack of clarity and obfuscation is staggering for an organization that employs masses of marketing and PR staff.

The second issue is the lack of data about how much PAYG is likely to cost. Few want to write an open-ended check to Microsoft for API usage. Microsoft is developing the model and understands how the APIs work, so it should be able to give indicative pricing for different scenarios. For instance, if I have 100 teams generating 35,000 new channel conversations and 70,000 chats monthly, how much will a backup cost? Or if my tenant generates new and updated documents at the typical rate observed by Microsoft across all tenants of a certain size, how much will a Syntex backup cost?

The last issue is the heavy-handed approach Microsoft has taken with backup ISVs. Being told that you must move from a working, well-sorted, and totally understood API to a new, untested, and metered API is not a recipe for good ISV relationships. Microsoft needs its ISVs to support its API tiered model. It would be so much better if a little less arrogance and a little more humility was obvious in communication. Just because you’re the big dog who owns the API bone doesn’t mean that you need to fight with anyone who wants a lick.

Make sure that you’re not surprised about changes that appear inside Office 365 applications by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers stay informed.

Even in First Release, Graph X-Ray Proves Its Worth

When Microsoft decided to build the administrative tools for Exchange Server 2007 around PowerShell, they realized that it would take time for administrators to become accustomed to PowerShell. Sensibly, Microsoft included a cmdlet logging facility in the Exchange Management Console (EMC) to allow administrators to see the PowerShell code used to execute different actions, such as creating a new mailbox. Cmdlet logging gave administrators prototype code to build scripts around and it’s still the best learning tool I have encountered in a Microsoft server product.

Roll on sixteen years and cmdlet logging doesn’t exist in the modern Exchange Admin Center (EAC). Many, including me, have moaned at Microsoft about this deficiency. One response is that EAC is no longer built on PowerShell, which makes it very difficult to generate and display PowerShell code for administrators to copy and reuse. It’s a great pity.

All of this brings me to a new browser extension called Graph X-Ray. Created by Microsoft employees but not a formal product, Graph X-Ray displays the Graph API commands run to execute actions in consoles like the Azure AD admin center and the Intune admin center. Not every action in these consoles depends on Graph APIs, but enough do in important areas like users, groups, and device management to make this an interesting facility.

Raw Graph or Graph SDK

Anyone developing code for Microsoft 365 can get value from Graph X-ray, whether you’re using compiled languages like C# or JavaScript or writing PowerShell scripts. Using Graph APIs in PowerShell normally means that scripts run faster, especially if the code must process more than a few objects. Scripters have the choice to include “raw API calls” or use cmdlets from the Microsoft Graph PowerShell SDK. The script to create a tenant configuration report is a good example of using raw API calls while the script to generate an Office 365 licensing report uses the SDK cmdlets. In either case, you need to understand how Graph API queries are formed and executed, and that’s where the Graph X-Ray extension proves its worth.

Restoring Deleted Microsoft 365 Groups

Take the example of restoring a deleted Microsoft 365 group. Before you can restore a group, you need to know what groups are in a soft-deleted state. Groups remain in the soft-deleted state for 30 days after deletion to allow administrators to restore groups using options in the Microsoft 365 and Azure AD admin centers. After the 30-day retention period lapses, Azure AD removes the groups permanently and they become irrecoverable.

In a large tenant, many groups might be waiting for permanent deletion, including inactive groups removed by the Microsoft 365 Groups Expiration policy. The Get-UnifiedGroup cmdlet can generate a list of soft-deleted groups using a command like this:

Get-UnifiedGroup -ResultSize Unlimited -IncludeSoftDeletedGroups:$True | ? {$_.WhenSoftDeleted -ne $Null} | Sort WhenSoftDeleted | Format-Table DisplayName, PrimarySmtpAddress, WhenSoftDeleted

The cmdlet works, but it’s slow. To speed things up, I tried using the Get-MgDirectoryDeletedItem SDK cmdlet. The cmdlet works when listing deleted user accounts, but no matter what I did, I couldn’t find a way to make it return a list of deleted groups.

Using the Graph X-Ray

I downloaded the Graph X-Ray extension for the Edge browser add-on (other versions are available for Chrome and a Microsoft Store app). To load the add-on, I opened the Developer Tools option in Edge and selected Graph X-Ray. A new blade opened in the browser to display the Graph API commands used for actions performed in the Azure AD admin center (Figure 1).

It’s important to emphasize that this is very much an MVP release. Things are by no means perfect, but enough value is present to allow Graph X-Ray to be very helpful. For example, the command reported when the Azure AD admin center lists deleted groups is:

Get-MgDirectoryDeletedItem -DirectoryObjectId $directoryObjectId -Property "id,displayName,mailEnabled,securityEnabled,groupTypes,onPremisesSyncEnabled,deletedDateTime,isAssignableToRole" -Sort "displayName%20asc" -Top 20

This is fine, but nowhere does it tell you how to populate the $directoryObjectId variable. On a more positive note, the raw Graph API query showed the structure needed to return deleted groups, and I was able to use that information to submit the query with the Invoke-MgGraphRequest SDK cmdlet, which worked. It’s worth noting that the Invoke-MgGraphRequest cmdlet exists to allow scripts to execute raw Graph API queries when an SDK cmdlet isn’t available (or doesn’t work).

Equipped with new-found knowledge about how to find deleted groups, I coded this script to report the set of soft-deleted groups including when each group is due for permanent deletion.

Connect-MgGraph
Select-MgProfile Beta
$uri = "https://graph.microsoft.com/beta/directory/deleteditems/microsoft.graph.group?`$select=id,displayName,groupTypes,deletedDateTime&`$orderBy=displayName%20asc&`$top=100"
[array]$Groups = (Invoke-MgGraphRequest -Uri $Uri).Value
If (!($Groups)) { write-Host "No deleted groups available for recovery" ; break }
$Report = [System.Collections.Generic.List[Object]]::new() # Create output file for report
$Now = Get-Date
ForEach ($Group in $Groups) {
     $PermanentRemovalDue = Get-Date($Group.deletedDateTime).AddDays(+30)
     $TimeTillRemoval = $PermanentRemovalDue - $Now
     $ReportLine = [PSCustomObject]@{ 
          Group                = $Group.DisplayName
          Id                   = $Group.Id
          Deleted              = $Group.deletedDateTime
          PermanentDeleteOn    = Get-Date($PermanentRemovalDue) -format g
          DaysRemaining        = $TimeTillRemoval.Days        } 
       $Report.Add($ReportLine) 
}
$Report | Sort {$_.PermanentDeleteOn -as [datetime]} | Out-GridView

The add-on includes the facility to download the commands it captures in a script (GraphXRaySession.PS1). There’s likely to be some duplication of commands in the downloaded script, but it’s great to have such an easy method to copy the commands for later use.

More Insight from Graph X-Ray

Moving on to restoring a soft-deleted group, Microsoft’s documentation for the Restore-MgDirectoryObject cmdlet is woefully deficient in terms of useful examples. An attempt to pass the identifier of a deleted group to the cmdlet failed:

Restore-MgDirectoryObject -DirectoryObjectId $GroupId
Restore-MgDirectoryObject : Resource '2eea84f2-eda3-4a72-8054-5b52c063ee3a' does not exist or one of its queried reference-property objects are not present.

Once again, I turned to Graph X-Ray to find out what command powered the restore deleted group option in the Azure AD admin center. The raw API reported by Graph X-Ray is a POST (update) query like this:

POST /directory/deleteditems/2eea84f2-eda3-4a72-8054-5b52c063ee3a/restore

It’s easy to take this command and repurpose it for use with the Invoke-MgGraphRequest cmdlet:

$uri = “https://graph.microsoft.com/beta/directory/deleteditems/8783e3dd-66fc-4841-861d-49976f0617c0/restore”
Invoke-MgGraphRequest -Method Post -Uri $Uri

More Please!

I wish Microsoft would provide similar insight across all the Microsoft 365 admin consoles. Being able to see the Graph API commands used to perform real-life actions is a powerful learning aid. If Microsoft is serious about driving the adoption of the Graph and the Graph SDK, they could do worse than invest in this kind of tooling. I hope that they do.

Keep up to date with developments like the Graph API commands by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers understand the most important changes happening across Office 365.

Item, Meeting, and People Insights Derived from User Signals

In August 2020, I wrote about how to control item insights using the Graph Explorer. Item insights are one of the three insights generated from Graph signals gathered from user activity and displayed in various places within Microsoft 365:

• Item insights: Recommendations about documents available to a user which they might be interested in. This information shows up in the profile card in the “Files” section.
• Meeting insights: When users open a meeting in their calendars, Outlook displays insights relevant to the meeting, such as documents and email covering the content of the meeting.
• People insights: Information about people deemed to be relevant to an individual and who have a “public relationship” with that individual. For example, they have the same manager (as noted in Azure AD) or share a common membership of a public group or distribution list with fewer than 30 members. People insights show up on the profile card in the “works with” section, with the people who appear ranked in order of the public and private communications (direct email or meetings rather than group-based communications) between the listed individuals and the owner of the profile card.

Item and meeting insights are configurable through the Search & Intelligence section of Settings in the Microsoft 365 admin center (Figure 1). Both can be enabled or disabled, and item insights can be disabled for a set of users defined as members of a group.

Recently, Microsoft added the ability to customize people insights, but only by running a Graph query. No doubt the necessary GUI will soon appear in the Microsoft 365 admin center.

The Need for User Privacy

In most cases, insights are useful. Looking at the list presented in the people card (Figure 2), users probably don’t realize how the Graph determines who appears in the list and what a public relationship is versus a private relationship. All they see is a set of individuals who the owner of the profile card “works with.”

However, in places where personal privacy is taken more seriously than in more liberal regimes, exposing the list of people that someone works with could be considered a breach of that person’s privacy. Although the list is based on public relationships that anyone can see by checking the membership of the relevant groups, it’s still too much for some.

Suppressing People Insights

To suppress people insights, a tenant can:

• Disable people insights completely. No one in the tenant sees this data.
• Disable people insights for a group of people. Everyone else sees the people insights.

For instance, if an organization decreed that the employees from a specific country should not see people insights, you could create a group (not a dynamic group) and add the relevant employees as members. The next step is to retrieve the identifier for the group and the tenant identifier, which is easily done by running the Get-AzureADGroup and Get-AzureADTenantDetail cmdlets:

Get-AzureADGroup -SearchString Disabled

ObjectId                             DisplayName           Description
--------                             -----------           -----------
c9758609-d33b-4eea-976b-d8e43a2ad135 DisabledGraphInsights Members of this group do not have graph insights

Get-AzureADTenantDetail

ObjectId                             DisplayName             VerifiedDomain
--------                             -----------             --------------
d662313f-14fc-43a2-9a5a-d2e27f4f3476 Office 365 for IT Pros  Office365itpros.com

Given that the Azure AD module is due for deprecation in March 2024, you should consider using the Microsoft Graph PowerShell SDK cmdlets:

Get-MgGroup -Filter "startsWith(displayname, 'Disabled')"

Id                                   DisplayName           Description                                       GroupTypes
--                                   -----------           -----------                                       ----------
c9758609-d33b-4eea-976b-d8e43a2ad135 DisabledGraphInsights Members of this group do not have graph insights  {Unified}

Get-MgOrganization | Select Id, DisplayName

Id                                   DisplayName
--                                   -----------
d662313f-14fc-43a2-9a7a-d2e27f4f3478 Office 365 for IT Pros

With the two identifiers, we can go to the Graph Explorer and update the Insight settings control for people insights. This is done by running a Graph patch query to:

https://graph.microsoft.com/beta/organization/d662313f-14fc-43a2-9a5a-d2e27f4f3476/ settings/peopleInsights

The query includes a request body containing the identifier of the group holding the set of users to exclude from people insights:

{
  "disabledForGroup": " c9758609-d33b-4eea-976b-d8e43a2ad135"
}

Figure 3 shows the Graph Explorer after running the patch query. The response (200) means that the Graph accepted the command, and the response preview shows the current configuration for people insights in the tenant. Here we can see that people insights are enabled (isEnabledInOrganization is True) and that a group of excluded users exists.

No More People Insights

After updating the organization setting, it can take up to 24 hours before its effect is noticed in profile cards. Figure 4 shows the result. The organization data is present, but the people insights are absent for any user who’s a member of the excluded group.

Controlling people insights is an easy thing to do. If you have some users who object to this information being available to others, it will take less than five minutes to create the excluded group and update the organization settings. Add a few extra minutes to populate the group, and then it’s just a matter of waiting for the update to percolate to applications.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across Office 365. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.

Critical Piece When Connecting to the Microsoft Graph

By now, most people who write PowerShell code to interact with Microsoft 365 workloads understand that sometimes it’s necessary to use Microsoft Graph API queries instead of “pure” PowerShell cmdlets. The Graph queries are usually faster and more reliable when retrieving large quantities of data, such as thousands of Microsoft 365 Groups. Over the last few years, as people have become more familiar with the Microsoft Graph, an increased number of scripts have replaced cmdlets with Graph queries. All these scripts use Entra ID (Azure AD) access tokens, as does any utility which interacts with the Microsoft Graph, like the Graph Explorer (Figure 1).

In the remainder of this article, I explore what an Entra ID access token contains.

The Need for Access Tokens

Graph queries need authentication before they can run and the Graph API uses modern authentication. Entra ID registered applications bridge the gap between PowerShell and the Graph. The apps hold details used during authentication such as the app name, its identifier, the tenant identifier, and some credentials (app secret or certificate. The app also holds permissions granted to access data through Graph APIs and other APIs. When the time comes to authenticate, the service principal belonging to an app uses this information to request an access token from Entra ID. Once Entra ID issues the access token, requests issued to the Invoke-RestMethod or Invoke-WebRequest cmdlets can include the access token to prove that the app has permission to access information.

At first glance, an access token is a confused mass of text. Here’s how PowerShell reports the content of an access token:

eyJ0eXAiOiJKV1QiLCJub25jZSI6IlFQaVN1ck1VX3gtT2YzdzA1YV9XZzZzNFBZRFUwU2NneHlOeDE0eVctRWciLCJhbGciOiJSUzI1NiIsIng1dCI6Ik1yNS1BVWliZkJpaTdOZDFqQmViYXhib1hXMCIsImtpZCI6Ik1yNS1BVWliZkJpaTdOZDFqQmViYXhib1hXMCJ9.eyJhdWQiOiJodHRwczovL2dyYXBoLm1pY3Jvc29mdC5jb20iLCJpc3MiOiJodHRwczovL3N0cy53aW5kb3dzLm5ldC9iNjYyMzEzZi0xNGZjLTQzYTItOWE3YS1kMmUyN2Y0ZjM0NzgvIiwiaWF0IjoxNjQ0ODQ1MDc3LCJuYmYiOjE2NDQ4NDUwNzcsImV4cCI6MTY0NDg0ODk3NywiYWlvIjoiRTJaZ1lEaW1McEgwTSt5QTk5NmczbWZUUXlYN0FBPT0iLCJhcHBfZGlzcGxheW5hbWUiOiJHZXRUZWFtc0xpc3QiLCJhcHBpZCI6IjgyYTIzMzFhLTExYjItNDY3MC1iMDYxLTg3YTg2MDgxMjhhNiIsImFwcGlkYWNyIjoiMSIsImlkcCI6Imh0dHBzOi8vc3RzLndpbmRvd3MubmV0L2I2NjIzMTNmLTE0ZmMtNDNhMi05YTdhLWQyZTI3ZjRmMzQ3OC8iLCJpZHR5cCI6ImFwcCIsIm9pZCI6IjM4NTRiYjA4LTNjMmMtNGI1Ny05NWZjLTI0ZTA3OGQzODY4NSIsInJoIjoiMC5BVndBUHpGaXR2d1Vva09hZXRMaWYwODBlQU1BQUFBQUFBQUF3QUFBQUFBQUFBQmNBQUEuIiwicm9sZXMiOlsiVGVhbVNldHRpbmdzLlJlYWRXcml0ZS5BbGwiLCJUZWFtTWVtYmVyLlJlYWQuQWxsIiwiR3JvdXAuUmVhZC5BbGwiLCJEaXJlY3RvcnkuUmVhZC5BbGwiLCJUZWFtLlJlYWRCYXNpYy5BbGwiLCJUZWFtU2V0dGluZ3MuUmVhZC5BbGwiLCJPcmdhbml6YXRpb24uUmVhZC5BbGwiLCJBdWRpdExvZy5SZWFkLkFsbCJdLCJzdWIiOiIzODU0YmIwOC0zYzJjLTRiNTctOTVmYy0yNGUwNzhkMzg2ODUiLCJ0ZW5hbnRfcmVnaW9uX3Njb3BlIjoiRVUiLCJ0aWQiOiJiNjYyMzEzZi0xNGZjLTQzYTItOWE3YS1kMmUyN2Y0ZjM0NzgiLCJ1dGkiOiI3RVkyWnVXV2JFYVF0T3piVVlwOUFBIiwidmVyIjoiMS4wIiwid2lkcyI6WyIwOTk3YTFkMC0wZDFkLTRhY2ItYjQwOC1kNWNhNzMxMjFlOTAiXSwieG1zX3RjZHQiOjEzMDI1NDMzMTB9.N9yvmkCedti2fzT44VfBkN7GvuCInrIgiMgNxdyZeAyxnbdZjEhxHmNdU6HLLHQ3J-GonpPdt28dKwYxgLcrSibGzSPVHddh6MDPYutSwfIxh2oRanxhgFOWVJADfbFoCxsRFDhKJNT39bsauIUiRNzGzbb6dvWuZQ8LrgWjZzjae2qxVxj9jvYgjXEypeYZgLvPOzJiBCuluAMH3TjPuS-CuglFK_edn4CS-ztCwM0hmDFD5BLNZqng5P2KqGTEgjkMKoyIJ8yTGBJpASfdqqEFqWzQwcQ9ese924qNC3hJR_5TWHp2Fl73bpdhwBHRL5UwGTPi9_ysYdndKhXwgA

Deciphering an Access Token

Access tokens issued by Entra ID comply with the OAuth 2.0 bearer token standard (RFC6750) and are structured as JSON-formatted Web Tokens. We can’t see the JSON content because it is base64Url encoded and signed. However, if you paste the token into a site like https://jwt.ms/, the site will decrypt the list of claims included in the token and we’ll see something like the details shown below for the access token featured above:

{ "typ": "JWT", 
"nonce": "gq3zmJhybfXGDGqt6RO2PX9s0cimmRpSRrTO90sQ4w4", 
"alg": "RS256",
 "x5t": "Mr5-AUibfBii7Nd1jBebaxboXW0", 
"kid": "Mr5-AUibfBii7Nd1jBebaxboXW0" 
}.
{ "aud": "https://graph.microsoft.com", 
"iss": "https://sts.windows.net/a662313f-14fc-43a2-9a7a-d2e27f4f3478/", 
"iat": 1644833772, 
"nbf": 1644833772,
 "exp": 1644837672,
 "aio": "E2ZgYJif1+eocevtzqRIrgDGA2V3AQ==",
 "app_displayname": "ReportDLs", 
"appid": "76c31534-ca1f-4d46-959a-6159fcb2f77a", 
"appidacr": "1",
 "idp": "https://sts.windows.net/a662313f-14fc-43a2-9a7a-d2e27f4f3478/", 
"idtyp": "app",
 "oid": "4449ce36-3d83-46fb-9045-2d1721e8f032",
 "rh": "0.AVwAPzFitvwUokOaetLif080eAMAAAAAAAAAwAAAAAAAAABcAAA.",
 "roles": 
[ "Group.Read.All", "Directory.Read.All", "User.Read.All" ],
 "sub": "4449ce36-3d83-46fb-9045-2d1721e8f032", 
"tenant_region_scope": "EU", 
"tid": "a662313f-14fc-43a2-9a7a-d2e27f4f3478",
 "uti": "BU1RVc7mHkmBq2FMcZdTAA", 
"ver": "1.0", 
"wids": [ "0997a1d0-0d1d-4acb-b408-d5ca73121e90" ],
 "xms_tcdt": 1302543310 
}
.[Signature]

The deciphered token divides into three parts: header, payload, and signature. The aim of a token is not to hide information, so the signature is not protected by encryption. Instead, it’s signed using a private key by the issuer of the token. Details of the algorithm and private key used to sign an access token are in its header. An application can validate the signature of an access token if necessary, but this is not usually done when running a PowerShell script. The payload is the location for the claims made by the token and is the most interesting place to check.

Another way to check what’s in an access token is to use the JWTDetails PowerShell module, which is available in the PowerShell Gallery. To install this (very small) module, run:

Install-Module -Name JWTDetails -RequiredVersion 1.0.0 -Scope AllUsers

Afterward, you can examine a token with the Get-JWTDetails cmdlet. Here’s an example revealing that the access token issued to an app allows it to access Exchange Online using the IMAP4 or POP3 protocols:

Get-JWTDetails -Token $Token

aud             : https://outlook.office.com
iss             : https://sts.windows.net/b662313f-14fc-43a2-9a7a-d2e27f4f3478/
iat             : 1671891468
nbf             : 1671891468
exp             : 1671895368
aio             : E2ZgYDAQS/prW6b0Zsah6KMXtnTEAQA=
app_displayname : POP3 and IMAP4 OAuth 2.0 Authorization
appid           : 6a90af02-6ac1-405a-85e6-fb6ede844d92
appidacr        : 1
idp             : https://sts.windows.net/a662313f-14fc-43a2-9a7a-d2e27f4f3478/
oid             : b7483867-51b6-4fdf-8882-0c43aede8dd5
rh              : 0.AVwAPzFitvwUokOaetLif080eAIAAAAAAPEPzgAAAAAAAABcAAA.
roles           : {POP.AccessAsApp, IMAP.AccessAsApp}
sid             : 1475d8e7-2671-47e9-b538-0ea7b1d43d0c
sub             : b7483867-51b6-4fdf-8882-0c43aede8dd5
tid             : a662313f-14fc-43a2-9a7a-d2e27f4f3478
uti             : COCw22GGpESVXvfdhmEVAQ
ver             : 1.0
wids            : {0997a1d0-0d1d-4acb-b408-d5ca73121e90}
sig             : PdScMpYqwA25qJL1z8q589sz/Ma5CGQ4ea9Bi0lnO2yByrIs530emYPnFPfQNN9EPBIvv4EaAoTLomrw4RMBWYoQSAgkBUXVrYGnC
                  jzAU6a2ZNZgo7+AORHk4iyLO0FpbLEaMJvCvI5vWhP9PHOxnGLcIsCbOmyrCK6lxxIKtBx851EpLrhpyvJ3p05NSw0D/mKzXPRKtc
                  rzQcUwECxOUugbm1zdq8JaE/PmSggBb87VZy7p1S2BXhxQZ5QU17JeIADyhCGm1Ml+avuIHsVS2iat/LPEi/nktbrXMcOzROpUKyZ
                  /7uVhxQ0cscJ6WGxbd+zJm36s25Yp1vMzSHaRxQ==
expiryDateTime  : 24/10/2022 15:22:48
timeToExpiry    : 00:59:34.7611307

Claims and Scopes

The list of claims in the access token includes simple claims and scopes (groups of claims). A claim is an assertion about something related to the token. In this case, the claims tell us details like:

• The tenant (tid).
• The intended consumer of the token (aud): https://graph.microsoft.com.
• The app name (app_displayname).
• The app identifier (appid).
• The security token service (STS) responsible for issuing the token (iss): https://sts.windows.net/a662313f-14fc-43a2-9a7a-d2e27f4f3478/.
• The generation time for the token (iat).
• The time when the token expires (exp). All dates are in Unix epoch time, so 1644837672 means 11:21:12 GMT on February 14, 2022. By default, access tokens issued by Entra ID last one hour, except those used by applications which support continual access evaluation (CAE), where Entra ID issues 28-hour access tokens because it can terminate access at any time and force the user to reauthenticate should a critical user event (like a password change) happen.
• The identifier for the object in the Microsoft identity system used for authentication (oid). In this case, the script uses a registered Entra ID app, so the value is the service principal for the app. You can test this by running the Get-MgServicePrincipal cmdlet from the Microsoft Graph PowerShell SDK:

Get-MgServicePrincipal -Filter "Id eq '4449ce36-3d83-46fb-9045-2d1721e8f032'"

DisplayName Id                                   AppId                                SignInAudience ServicePrincipalTy
                                                                                                     pe
----------- --                                   -----                                -------------- ------------------
ReportDLs   4449ce36-3d83-46fb-9045-2d1721e8f032 77c31534-ca1f-4d46-959a-6159fcb2f77a AzureADMyOrg   Application

Scopes are a logical grouping of claims, and they can serve as a mechanism to limit access to resources. The roles claim contains a scope of Graph API permissions starting with Group.Read.All and ending with User.Read.All. We therefore know that this app has consent from the organization to use the permissions stated in the scope when it executes Graph API queries. The list of permissions is enough to allow the PowerShell script (in this case, one to generate a report of distribution list memberships) to query the Graph for a list of all groups and read the membership of each group.

From bitter experience, I know how easy it is to get Graph permissions wrong. One way to check is sign into the Graph Explorer and run the query (here’s an example) to check what permissions the Explorer uses to execute the query. However, you can also dump the access token to check that the set of permissions in the access token matches what you expect. It’s possible that you might have requested some application permissions for the app and failed to gain administrator consent for the request, meaning that the access token issued to the app by Entra ID won’t include the requested permissions.

Using the Access Token

Once we’re happy that we have a good access token, we can use it with Graph queries. Here’s how to fetch the list of distribution groups in a tenant. The access token is included in the $Headers variable passed to the Invoke-RestMethod cmdlet.

$Headers = @{Authorization = "Bearer $token"}

$Uri = "https://graph.microsoft.com/V1.0/groups?`$filter=Mailenabled eq true and not groupTypes/any(c:c+eq+'Unified')&`$count=true"
[array]$DLs = (Invoke-RestMethod -Uri $Uri -Headers $Headers -Method Get -ContentType "application/json")
$DLs = $DLs.Value

And if everything goes to plan, we should have a set of distribution lists to process. If not, it’s bound to be a problem with your access token, so it’s time to return to square one and restart the acquisition process.

Learn more about how Office 365 really works on an ongoing basis by subscribing to the Office 365 for IT Pros eBook. Our monthly updates keep subscribers informed about what’s important across the Office 365 ecosystem.

Spring Clean Time for Apps Coming Soon

Last year, I wrote about the need to review and clean up Entra ID integrated applications. That article describes how to extract information from Entra ID o a CSV file and use the CSV to create a Microsoft List. To make it easy to access the list, we create a channel tab in Teams. Everything works to identify suspect apps that might need removal. I think that you should perform such a review periodically. It just makes sense.

Another way to monitor potentially suspicious app activity is to review sign in data for service principals. The intention is to identify unrecognized service principals signing into the tenant and figure out what apps are involved. Sign-ins can originate from well-known service principals used by Microsoft apps, third-party apps, or the service principals automatically created by Entra ID when tenants register apps to interact with the Graph (for instance, to authenticate calls made to Graph APIs in PowerShell scripts). Sign-in data for service principals is available through the Entra admin center (Figure 1) and now it’s accessible using the Microsoft Graph List SignIns API.

The reason why this update is important is that access to sign-in data via the Graph makes it possible to download the information for analysis or store it for long-term retention in an external repository. Although you can download sign-in data as a CSV file from the Entra admin center, it’s more flexible to access the information via Graph queries, especially when you want to probe the activity patterns of certain service principals.

Getting Sign-In Data from the Graph

Any application which wants to interact with the Graph requires consent for permissions to access data. In this instance, consent is needed the Directory.Read.All and AuditLog.Read.All application permissions. Delegate permissions can also be used, and in this case the account used must hold an administrative role capable of accessing the Entra ID sign-in logs.

A suitably-permissioned application can issue queries against the SignIns API. To fetch service principal sign-in data, the query executed by the application must use a Lambda qualifier to filter data. Apart from setting a date range to search for sign-in data, the important point is to filter against the signInEventTypes property to select sign-in events for service principals. Here’s an example of a query to fetch sign-in data for between 17:30 and 22:3 on 19 January.

https://graph.microsoft.com/beta/auditLogs/signIns?&$filter=createdDateTime ge 2022-01-19T17:30:00Z and createdDateTime le 2022-01-19T22:30:00Z and signInEventTypes/any(x:x eq 'servicePrincipal')

To test the query (or one which suits your purposes), use the Graph Explorer to see what the query returns.

I wrote a simple PowerShell script (downloadable from GitHub) to fetch service principal sign-in data for the last seven days. A quick summary of the data revealed that many sign-ins came from an app named Office 365 Reports. Curiously, an app used by a PowerShell script that I had posted on GitHub also showed up with 22 sign-ins. The Information Barrier Processor is the app used by Microsoft 365 to check user accounts against information barrier policies to ensure that no one is communicating with anyone when they shouldn’t.

$Report | Group SpName | Sort Count -Descending | Select Name, Count

Name                                         Count
----                                         -----
Office 365 Reports                             369
Graph Microsoft 365 Groups Membership Report    22
Information Barrier Processor                   21
Security and Audit                               5
PS-Graph                                         1

Resolving the large set of sign-ins was easy. The data stored in the list (Figure 2) revealed the service principal to belong to an Office 365 Reporting app originally published by Cogmotive (acquired by Quadrotech and then by Quest Software). I haven’t used the app in years, but the sign-ins kept on coming.

Over time, it’s easy to accumulate crud in the form of service principals installed for one reason or another. Testing an ISV product is a classic example, which is a good reason to always conduct tests in a test tenant instead of the production tenant. Or if you stop using an app, remember to clean up by removing service principals and other app debris that the app vendor might leave behind.

The sign-ins for the app used by the PowerShell script probably exist because I shared a copy of the script with my tenant identifier, the app identifier, and the app secret in place. I quickly replaced the script with a copy containing obfuscated credentials, but failed to change the app secret, meaning that anyone with an original copy could run the code. Now alerted, I removed the app secret. My suspicions were confirmed when a batch of failed sign-ins subsequently occurred for the app. This goes to prove how easy it is to create a potential compromise if you’re not careful.

Removing a Service Principal with PowerShell

You can clean up unwanted service principals with either the Entra admin center or PowerShell. I always have a PowerShell session open, so I chose that route. In this example, we find the object identifier for a service principal using its display name as a filter for the Get-MgServicePrincipal cmdlet. When sure that this is the right service principal to remove, we use the object identifier to remove the service principal with the Remove-MgServicePrincipal cmdlet.

$SP = Get-MgServicePrinicpal -filter "displayname eq 'Office 365 Reports'"

$SP

DisplayName        Id                                   AppId                                SignInAudience     
-----------        --                                   -----                                --------------     
Office 365 Reports 9ac957ae-160b-48d3-9a6f-f4c27acca040 507bc9da-c4e2-40cb-96a7-ac90df92685c AzureADMultipleOrgs 

Remove-MgServicePrincipal -ServicePrincipalId $Sp.id

Adding Context

A list of service principals known to the tenant is a valuable input to a review for unwanted or unnecessary apps holding some form of consent (permissions) to organization data. Adding context to the data by knowing which service principals are actively signing into the tenant makes it easier to prioritize action. The data is there, it’s available, and it’s up to you to decide what to do with it.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Microsoft 365 Notifications User Counts Come from the Graph

Message center notification MC315739 (January 18, roadmap item 83946) brings news of a big change for the information included in notifications. Soon, along with the text describing new features or changes to existing Microsoft 365 features, notifications will include service usage data relevant to the change. Deployment starts for targeted release tenants in mid-January and should be complete worldwide for all tenants by mid-February.

Let’s take the change announced in MC302456 as an example. This notification describes how users can maintain their guest accounts in other tenants from Teams. To help administrators understand how many people will be affected by the change, the service communications API queries the Microsoft Graph reports API to retrieve the monthly active user data for Teams and reports this information in the notification.

Figure 1 shows a mock-up included in MC315739 to illustrate how Microsoft 365 notifications highlight user data. On the left, you see a notification for a change affecting multiple workloads together with the usage data for each workload (Outlook is really Exchange Online, but obviously non-Outlook clients can connect to Exchange Online mailboxes). On the right, you see a notification for Kaizala, which doesn’t store its usage data in the Microsoft Graph, so it’s impossible to display this information.

Editorial comment: The need for Kaizala is possibly now much reduced by the general availability of the Teams Walkie-Talkie feature.

The Problem with Microsoft Graph Usage Data

The Microsoft Graph reports API allows access to usage data about some Microsoft 365 services. Coverage is good for base workloads (SharePoint Online, Exchange Online, Teams, and OneDrive for Business) and not so good elsewhere (Planner, Stream, Forms, Whiteboard, etc.). Nevertheless, the usage data is detailed enough to build a picture of user activity over the last ninety days. If you’d like to know how to use the API with PowerShell, consider running the User Activity Analysis script to see how to make calls against the reports API and the kind of data the API returns. For example, this code creates a query to retrieve Teams activity data for users over the last 30 days. Data returned by the reports API is always a few days behind the actual date.

$TeamsUserReportsURI = "https://graph.microsoft.com/v1.0/reports/getTeamsUserActivityUserDetail(period='D30')"

[array]$TeamsUserData = (Invoke-RestMethod -Uri $TeamsUserReportsURI -Headers $Headers -Method Get -ContentType "application/json") -Replace "...Report Refresh Date", "Report Refresh Date" | ConvertFrom-Csv

The data returned by the API is in an array. Here’s the item in the area for an account:

Report Refresh Date        : 2022-01-16
User Principal Name        : Jane.Smith@office365itpros.org
Last Activity Date         : 2022-01-15
Is Deleted                 : False
Deleted Date               :
Assigned Products          : POWER BI (FREE)+ENTERPRISE MOBILITY + SECURITY E5+BUSINESS APPS (FREE)+MICROSOFT POWER AUTOMATE FREE+MICROSOFT VIVA TOPICS+MICROSOFT DEFENDER FOR CLOUD APPS – APP GOVERNANCE+OFFICE 365 E5 WITHOUT AUDIO CONFERENCING
Team Chat Message Count    : 58
Private Chat Message Count : 14
Call Count                 : 1
Meeting Count              : 5
Has Other Action           : No
Report Period              : 30

The data looks good and is useful. However, some workloads (like Teams) return data for both tenant and guest accounts, so the numbers reported in message center notifications will reflect that data. You might be concerned about how a change will affect guest users, but I hazard a guess that most tenant administrators will focus on the effect on tenant users.

Another issue (acknowledged in MC315739) is the non-specific nature of the report. Usage across all clients and all features is included into one workload figure. For instance, a change affecting Microsoft Lists in SharePoint Online and OneDrive for Business might affect just the five people who create and manage Lists, but the notification will say that the change affects everyone who has used SharePoint Online or OneDrive for Business in the last month. You won’t know either if a change is specific to a client platform, like Android or iOS.

Counting all and sundry who use a workload isn’t such a big problem for new features. It is more important for updated features and becomes even more critical when Microsoft deprecates some functionality. You then want to know precisely who is affected, or at least, how many are affected.

Another aspect of an all-up number is that it doesn’t take account of multi-geo deployments. You’ll know that some people in the organization might need to be informed about a change, but not their location.

Still a Good Change

Even with the caveats listed above, including user data in Microsoft 365 notifications is still a good change. If you see a notification where a low number of users will experience an impact, you can probably spend less time preparing for that change and more on changes affecting large user populations. The availability of data through Graph APIs limit what the developers can do to slice and dice usage data to make it more precise and informative. This will probably happen over time. In the interim, take the user information presented in Microsoft 365 notifications as a starting point to help you understand the likely impact of individual changes on users. Use this data in conjunction with your knowledge of the tenant and how people work within the organization, and the monthly active user data for affected workloads will be helpful. Taken as an exact guide, it won’t be.

I guess I might have to update my script to extract and report information from message center notifications to accommodate this change…

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Use Teams, PowerShell, or the Graph

Vasil Michev, the Technical Editor of the Office 365 for IT Pros eBook, comes up with all sorts of weird and wonderful insights into Microsoft 365. A recent question he discussed on his blog was how to find the creation date for a tenant. It’s a good question because it forces respondents to know where to look for this information and is exactly the kind of poser we like to tease out as we write content for the book.

As Vasil points out, the obvious answer is to fire up the Teams admin center because the tenant creation date appears on a card displayed on its home screen (Figure 1). The Teams admin center is the only Microsoft 365 portal which shows this information. Why the Teams developers thought that it was useful to highlight the tenant creation date is unknown. After all, the date won’t change over time and static information is not usually featured by workload dashboards.

Opening an administrative portal is no challenge. Vasil suggests several alternate methods to retrieve the tenant creation date. It seemed like fun to try some of these methods against my tenant. Here’s what I found.

Using Exchange Online Data

If you’ve used Exchange Online from the start, you can check the creation date of the Exchange organization configuration object, created when an administrator enables Exchange Online for the first time.

(Get-OrganizationConfig).WhenCreated

Monday 27 January 2014 20:28:45

It’s an interesting result. Exchange Online reports its initiation in January 2014 while Teams is quite sure that the tenant existed in April 2011. I’ve used Exchange Online for email ever since I had a tenant, so the disconnect between Exchange Online and the tenant creation date is interesting.

Another way of checking Exchange data is to look at the creation dates for mailboxes. This PowerShell snippet finds all user mailboxes and sorts them by creation date. The first mailbox in the sorted array is the oldest, so we can report its creation date:

[array]$Mbx = Get-ExoMailbox -ResultSize Unlimited -Properties WhenCreated -RecipientTypeDetail UserMailbox | Sort {$_.WhenCreated -as [datetime]} 
Write-Host ("The oldest mailbox found in this tenant is {0} created on {1}" -f $Mbx[0].DisplayName, $Mbx[0].WhenCreated)

The oldest mailbox found in this tenant is Tony Redmond created on 27/01/2014 20:36:38

(Dates shown are in Ireland local format. The equivalent U.S. format date is 01/27/2014).

Grabbing all mailboxes to check their creation date will not be a fast operation. Even using the REST-based Get-ExoMailbox cmdlet from the Exchange Online management module, it will take time to retrieve all the user mailboxes in even a medium size tenant.

As it turns out, the oldest mailbox is my own, created about eight minutes after the initiation of Exchange Online. However, we’re still in 2014 when the tenant proclaims its creation in 2011, so what happened?

A search through old notes revealed that Microsoft upgraded my original Office 365 tenant created in 2011 to an enterprise version in 2014. It seems that during the tenant upgrade, Microsoft recreated the instance of Exchange Online. That explanation seems plausible.

Administrator Accounts

Another method is to examine the creation dates of administrator accounts to find the oldest account. This is usually the administrator account created during tenant setup. In other words, when you create a new tenant, you’re asked to provide the name for an account which becomes the first global administrator. If we look at the administrator accounts in the tenant and find the oldest, it should be close to the tenant creation date shown in the Teams admin center. That is, unless someone deleted the original administrator account.

Azure AD is the directory of record for every Microsoft 365 tenant, so we should check Azure AD for this information. The steps are:

• Find the set of accounts which currently hold the global administrator role. We omit the account returned with the object id 25cbf210-02e5-4a82-9f5c-f41befd2681a as this is a service principal used by Microsoft Rights Management services (you can confirm this by running Get-AzureADServicePrincipal -ObjectId 25cbf210-02e5-4a82-9f5c-f41befd2681a).
• Check each account to find the creation date. This is slightly complicated when using the Azure AD PowerShell module because the creation date is part of the extension properties. We therefore use the Get-AzureADUserExtension cmdlet to extract the date and then store it in the array used to hold details about tenant administrators.
• Sort the accounts by creation date and report the oldest.

Here’s the code I used:

# Find the identifier for the Azure AD Global Administrator role
$TenantAdminRole = Get-AzureADDirectoryRole | Where-Object {$_.DisplayName -eq ‘Global Administrator’} | Select ObjectId
# Get the set of accounts holding the global admin role. We omit the account used by
# the Microsoft Rights Management Service
$TenantAdmins = Get-AzureADDirectoryRoleMember -ObjectId $TenantAdminRole.ObjectId | ? {$_.ObjectId -ne "25cbf210-02e5-4a82-9f5c-f41befd2681a"} | Select-Object ObjectId, UserPrincipalName
# Get the creation date for each of the accounts
$TenantAdmins | ForEach-Object { $_ | Add-Member -MemberType NoteProperty -Name "Creation Date" -Value (Get-AzureADUserExtension -ObjectId $_.ObjectId ).Get_Item("createdDateTime") }
# Find the oldest account
$FirstAdmin = ($TenantAdmins | Sort-Object {$_."Creation Date" -as [datetime]} | Select -First 1)
Write-Host ("First administrative account created on {0}" -f $FirstAdmin."Creation Date")

The older Microsoft Online PowerShell module doesn’t require such a complicated approach to retrieve account creation data. Taking the code shown above and replacing the Get-AzureADUserExtension cmdlet with Get-MsOlUser, we get:

$TenantAdmins | ForEach-Object { $_ | Add-Member -MemberType NoteProperty -Name "Creation Date" -Value ((Get-MsOlUser -ObjectId $_.ObjectId ).WhenCreated) }

Using either cmdlet, the result is:

First administrative account created on 11/04/2011 17:35:11

The Teams admin center also reports April 11, 2011, so using administrator accounts might be a viable way to determine tenant age.

Use the Graph

Microsoft 365 stores information for each tenant in the Microsoft Graph, and it’s the Graph which is the source for the Teams admin center. We can retrieve the same information by running the https://graph.microsoft.com/V1.0/organization Graph query. The createdDateTime property returned in the organization settings is what we need.

Here’s the PowerShell code to run after obtaining the necessary access token for a registered app, which must have consent to use the Organization.Read.All Graph permission. Vasil used the beta endpoint when he showed how to fetch tenant organization settings using the Graph Explorer (which saves the need to write any code), but the V1.0 endpoint works too.

$Uri = "https://graph.microsoft.com/V1.0/organization"
$OrgData = Invoke-RESTMethod -Method GET -Uri $Uri -ContentType "application/json" -Headers $Headers
If ($OrgData) {
  Write-Host ("The {0} tenant was created on {1}" -f $Orgdata.Value.DisplayName, (Get-Date($Orgdata.Value.createdDateTime) -format g)) }

The Redmond & Associates tenant was created on 11/04/2011 18:35

The first administrator account appears to date from 17:35 while the tenant creation time is an hour later. This is easily explained because all dates stored in the Graph are in UTC whereas the dates extracted from Azure AD and reported by PowerShell reflect local time. In April 2011, local time in Ireland was an hour ahead of UTC.

An Old Tenant

After all the checks, it’s clear that I created my tenant in the early evening of April 11, 2011. Given that this was ahead of Microsoft’s formal launch of Office 365 in July 2011, I can claim to use an old tenant, for what that’s worth.

Finding Teams in Groups

The fastest way to fetch the list of Teams in a Microsoft 365 tenant programmatically is to use the Graph API. PowerShell is fast enough in small tenants, but once there’s more than a couple of hundred teams (groups) to process, the Graph is usually a better choice. Unless you’ve got time to wait, of course.

I cover the topic in an article explaining how to fetch a list of Teams using the Groups endpoint. Because the Groups API returns all types of groups, you apply a filter to find the set of Microsoft 365 Groups which are team-enabled. For PowerShell, the commands needed to execute the Graph API call are:

$Uri = “https://graph.microsoft.com/V1.0/groups?`$filter=resourceProvisioningOptions/Any(x:x eq 'Team')”
[array]$Teams = Invoke-WebRequest -Method GET -Uri $Uri -ContentType "application/json" -Headers $Headers | ConvertFrom-Json

The filter used with the query is a Lambda operator. In this case, it requests the Graph to return any Groups it finds where the provisioning option is set to “Team.” Using a filter to find Teams is a well-known technique exploited by developers: we use it in the Graph-based version of the Teams and Groups Activity Report script.

New API to Make it Easier to List Teams

In December, Microsoft published a new List Teams API in the beta version of the Graph. The API access the Teams endpoint `to fetch a list of teams without using a filter and works with both delegated and application permissions. Apps need consent for one of the Team.ReadBasic.All, TeamSettings.Read.All, TeamSettings.ReadWrite.All permissions to use the API. You can apply filters to the List Teams API to find specific teams if you don’t want the full set. The equivalent code to fetch all the teams in the tenant is:

$Uri = "https://graph.microsoft.com/beta/teams"
[array]$Teams = Invoke-WebRequest -Method GET -Uri $Uri -ContentType "application/json" -Headers $Headers | ConvertFrom-Json

Remember that the Graph uses paging to return data, so code needs to be prepared to process multiple pages of data to acquire the full set of teams.

In both cases, the set of Teams returned by the query is in an array called Value. In other words, to see the details of individual teams, you access the array using $Teams.Value. For example, the first team is available using $Teams.Value[0], the second with $Teams.Value[1], and so on. The count of teams returned in the array is available with $Teams.Value.Count. In the case of the Teams List API, it’s also available as $Teams.’@odata.count’.

Beta Queries

The List Teams API currently returns just three properties for a team. Here’s what you get for a team:

$Teams[0]

id                          : 109ae7e9-1f94-48d1-9972-64abab87b89a
createdDateTime             :
displayName                 : French Customers
description                 : Delivering great service to French customers
internalId                  :
classification              :
specialization              :
visibility                  :
webUrl                      :
isArchived                  :
isMembershipLimitedToOwners :
memberSettings              :
guestSettings               :
messagingSettings           :
funSettings                 :
discoverySettings           :

To get the other team properties (for instance, the archive status for a team), you must query the properties of an individual team. This isn’t difficult, and the same downside exists if you use the Groups endpoint to fetch a list of Teams. That endpoint returns many properties for a group, but not the teams properties.

The Graph Explorer is a good way to try out the new API. Make sure that you sign into your tenant and have consent to use at least the Team.ReadBasic.All permission. Then input a query and see what happens. Figure 1 shows the result of running the query: https://graph.microsoft.com/beta/teams?$filter=startswith(displayName, ‘Office 365’) to return the set of teams whose display name starts with Office 365. As noted above, along with the properties of the individual teams, the query also returns the count in @odata.count.

No Need to Update Now

If your programs or scripts often need to retrieve a list of teams for processing, you should keep an eye on the List Teams API to track its development. For now, there’s no need to update existing code. In time, it might be the case that the new API delivers better performance than going through the Groups endpoint or that the query returns all team properties without having to retrieve individual teams. Better performance and functionality are always welcome. Let’s see what happens as the new API makes its way from beta to production.

Learn how to exploit the Office 365 data available to tenant administrators through PowerShell and the Graph with the help of the many examples in the Office 365 for IT Pros eBook. We love figuring out how things work.

Failure Will Break Ability to Send Email via Exchange Online

Message center notification MC297438 arrived in the Microsoft 365 admin center on November 10 to inform me that Microsoft was about to enforce version 1.2 of the Transport Layer Security (TLS) for Direct Routing SIP interfaces. I have no problem with this proposal. It seems perfectly splendid to enforce TLS 1.2 for all manner of communications.

The note then said: “You are receiving this message because our reporting indicates that your organization is still connecting using SMTP Auth client submission via smtp.office365.com with TLS1.0 or TLS1.1 to connect to Exchange Online.”

Deprecating Old TLS

The problem here is that PowerShell uses the system default for TLS unless you specify otherwise. Although Microsoft is excluding SMTP AUTH from the set of connection protocols they will block for basic authentication in all tenants in October 2022, this doesn’t mean that SMTP AUTH is immune from other efforts within Microsoft 365 to remove older, less secure protocols. As Microsoft notes in MC297438, they communicated their intention to remove TLS 1.0 and 1.1 from Microsoft 365 as far back as December 2017, so this development shouldn’t come as a shock to anyone.

I covered this topic in January 2021 and noted that script developers who use the Send-MailMessage cmdlet to send email via Exchange Online should include a line in their scripts to force PowerShell to use TLS 1.2. If you don’t, the deprecation of TLS 1.0 and 1.1 in Exchange Online will prevent scripts being able to send messages.

For the record, the command to force TLS 1.2 connections from PowerShell is:

[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls12

Hopefully, components like multi-function devices which use basic authentication with SMTP AUTH today can use TLS 1.2 connections. If they can’t, those connections will stop working even while basic authentication for SMTP AUTH persists.

Moving to the Graph

Forcing PowerShell to use TLS 1.2 is effective, but it’s a short-term fix. Microsoft will come back to the topic of SMTP AUTH once the dust settles after the removal of basic authentication for the other connection protocols next year. The time will come when Exchange Online ceases support for basic authentication with SMTP AUTH connections.

Microsoft’s preferred method for sending secure email with Exchange Online is to use the Graph APIs. You can do this in two ways by upgrading scripts to replace calls to the Send-MailMessage cmdlet with:

• The SendMail Graph API call
• The Send-MgUserMessage cmdlet from the Microsoft Graph SDK for PowerShell. This approach is possibly easier for people coming from PowerShell and exploring their first use of the Graph APIs.

Graph APIs use modern authentication, so the basic authentication issue doesn’t arise.

It’s time to inventory the scripts in your tenant which send email via Exchange Online to know what needs to be done, make sure that TLS 1.2 is used by all scripts, and consider the best option for future upgrades.

Insight like this doesn’t come with hard work and experience. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

EWS Sunset Will Impact Office 365 Tenants and ISVs

Microsoft’s October 5 announcement that they will decommission 25 Exchange Web Services (EWS) APIs by March 31, 2022, contains some clear signals that EWS is on a short runway to oblivion. Microsoft says that the set of 25 APIs selected to go first (Figure 1) are those least used with Exchange Online.

Microsoft also says that they’ll remove the ability to create new EWS apps starting on September 30, 2022. Both steps are part of the sunset process to ease EWS out of Microsoft 365, with Microsoft noting that “EWS is a legacy API surface that has served us well, but no longer meets the security and manageability needs of modern app development.”

Figuring Out the Impact

Microsoft created EWS as a SOAP-based protocol to allow ISVs access to Exchange Server content. In fact, when it appeared in Exchange 2007, EWS was the first successful API for Exchange that Microsoft delivered since MAPI. Since then, EWS has been used by Microsoft in the Entourage and Outlook for Mac clients and by ISVs for many different purposes, including migration of mailbox and PST data to Exchange Online and as the foundation for backup products.

The introduction of the Microsoft Graph APIs as the preferred access method for data across Microsoft 365 marked the beginning of the end for EWS. The first sign was Microsoft’s July 2018 announcement that EWS would receive no further feature updates. In many cases, a static API is a dead API, especially when its owner’s attention is directed firmly at another API.

It’s hard to deny that Microsoft is wrong to force people towards the Microsoft Graph APIs at this point. The Graph APIs are newer, used across Microsoft 365, support standards like OAuth and OData, and its REST-based model is familiar to many programmers. The wide range of tools like the Graph Explorer and SDKs in different programming languages make it easier for developers. It’s also true that the Graph APIs have more granular security policies. But the biggest thing is that the Graph is where the puck is going.

The second sign was the inclusion of EWS in the set of connectivity protocols which Microsoft plans to disable for basic authentication. The latest turn-off date is October 1, 2022.

All of which brings us to the need for organizations to figure out how EWS is used inside their tenants and draw up plans for its replacement. The focus areas to look for EWS include:

• Custom home-built programs and PowerShell scripts.
• Third-party clients and client add-ons.
• Third-party products.

Every organization is different and EWS has been around for a long time. The two factors make it difficult to give a definitive set of guidelines for what to look. Three obvious areas are migration, connectors, and backup. Any product which streams information into or out of Exchange Online should be questioned to establish what API is in use. Most backup products for Exchange Online use EWS. The protocol was never intended as the foundation for backup, but nothing else is available so ISVs have used what they can.

Searching for a Graph Replacement

As they digest the news that EWS is in terminal decline, ISVs will naturally look for a replacement. Right now, the Microsoft Graph APIs cover Outlook (mail) and other objects stored inside mailboxes like tasks, notes, and contacts. However, there isn’t any obvious Graph API to handle large-scale streaming of mailbox content of the type usually experienced in backup and restore operations.

Microsoft has just released a Graph export API for Teams which is deemed suitable for backup and restore scenarios. However, that API comes with consumption meters and a per-message charging model which creates a whole new economic model for pricing for backup data. The volume of Teams messages is lower than Exchange Online email and the size of Exchange Online mailboxes is larger (even with the new 1.5 TB limit for auto-expanding archives). If Microsoft decides to impose the same consumption pricing model on an export/import API for Exchange Online, it could create many headaches for customers and ISVs alike to deal with the costs associated with scenarios like:

• Moving data from legacy archive solutions to Exchange Online.
• Importing data from third-party systems into Exchange Online (connectors).
• Tenant-to-tenant migrations of Exchange mailboxes and archives.
• Backup processing for Exchange Online mailboxes (user, shared, group, public folder, and other) to external third-party datacenters.

Although all the signs are that EWS will slip away soon, Microsoft hasn’t set a final (public) date for the removal of EWS from Exchange Online. You could put your head into the sand and hope that the protests of customers and ISVs will be sufficient to force Microsoft to keep EWS going for longer. I don’t think that’s a great strategy. It’s better to accept that the days of EWS are numbered (in the low hundreds) and start working on replacement components for your IT infrastructure. You know it makes sense.

Learn about Exchange Online and the rest of the Office 365 ecosystem by subscribing to the Office 365 for IT Pros eBook. Use our experience to understand what’s importance and how changes affect your tenant.

Why Lambda Operators are Sometimes Needed

A reader asked about the meaning of x:x in a Graph API query included in the article about upgrading Office 365 PowerShell scripts to use the Graph. You see this construct (a Lambda operator) in queries like those necessary to find the set of accounts assigned a certain license. For example, to search for accounts assigned Office 365 E3 (its SKU or product identifier is always 6fd2c87f-b296-42f0-b197-1e91e994b900):

https://graph.microsoft.com/beta/users?$filter=assignedLicenses/any(s:s/skuId eq 6fd2c87f-b296-42f0-b197-1e91e994b900)

Find the set of Microsoft 365 Groups in the tenant:

https://graph.microsoft.com/v1.0/groups?$filter=groupTypes/any(a:a eq 'unified')

Find the set of Teams in the tenant:

https://graph.microsoft.com/beta/groups?$filter=resourceProvisioningOptions/Any(x:x eq 'Team')

As you might expect, because the cmdlets in the Microsoft Graph SDK for PowerShell essentially are wrappers around Graph API calls, these cmdlets use the same kind of filters. For example, here’s how to find accounts with the Office 365 licenses using the Get-MgUser cmdlet:

[array]$Users = Get-MgUser -Filter "assignedLicenses/any(x:x/skuId eq 6fd2c87f-b296-42f0-b197-1e91e994b900)" -all

Lambda Operators and Advanced Graph Queries

All these queries use lambda operators to filter objects using values applied to multi-valued properties. For example, the query to find users based on an assigned license depends on the data held in the assignedLicenses property of Azure AD accounts, while discovering the set of Teams in a tenant relies on checking the resourceProvisioningOptions property for Microsoft 365 groups. These properties hold multiple values or multiple sets of values rather than simple strings or numbers. Because this is a query against a multivalue property for an Entra ID directory object, it’s called an advanced query.

Accessing license information is a good example to discuss because Microsoft is deprecating the Azure AD cmdlets for license management at the end of 2022, forcing tenants to upgrade scripts which include these cmdlets to replace them with cmdlets from the Microsoft Graph SDK for PowerShell or Graph API calls. This Practical365.com article explains an example of upgrading a script to use the SDK cmdlets.

If we look at the value of assignedLicenses property for an account, we might see something like this, showing that the account holds three licenses, one of which has a disabled service plan.

disabledPlans                          skuId
-------------                          -----
{33c4f319-9bdd-48d6-9c4d-410b750a4a5a} 6fd2c87f-b296-42f0-b197-1e91e994b900
{}                                     1f2f344a-700d-42c9-9427-5cea1d5d7ba6
{}                                     8c4ce438-32a7-4ac5-91a6-e22ae08d9c8b

It’s obvious that assignedLicenses is a more complex property than a single-value property like an account’s display name, which can be retrieved in several ways. For instance, here’s the query with a filter to find users whose display name starts with Tony.

https://graph.microsoft.com/v1.0/users?$filter=startswith(displayName,'Tony')

As we’re discussing PowerShell here, remember that you must escape the dollar character in filters. Taking the example above, here’s how it is passed in PowerShell:

$Uri = "https://graph.microsoft.com/v1.0/users?`$filter=startswith(displayName,'Tony')"
[array]$Users = Invoke-WebRequest -Method GET -Uri -ContentType "application/json" -Headers $Headers | ConvertFrom-Json

The data returned by the query is in the $Users array and can be processed like other PowerShell objects.

Using Any and All

Getting back to the lambda operators, while OData defines two (any and all), it seems like the all operator, which “applies a Boolean expression to each member of a collection and returns true if the expression is true for all members of the collection (otherwise it returns false)” is not used. At least, Microsoft’s documentation says it “is not supported by any property.”

As we’ve seen from the examples cited above, the any operator is used often. This operator “iteratively applies a Boolean expression to each member of a collection and returns true if the expression is true for any member of the collection, otherwise it returns false.”

If we look at the filter used to find accounts assigned a specific license:

filter=assignedLicenses/any(s:s/skuId eq 6fd2c87f-b296-42f0-b197-1e91e994b900)

My interpretation of the component parts (based on Microsoft documentation) of the filter is:

• assignedLicenses is the parameter, or the property the filter is applied to. The property can contain a collection of values or a collection of entities. In this case, the assignedLicenses property for an account contains a collection of one or more license entities. Each license is composed of the SkuId and any disabled plans unavailable to the license holders.
• s:s “is a range variable that holds the current element of the collection during iteration.” The interesting thing is that you can give any name you like to the range variable. In this case, it could be license:license or even rubbish:debris. It’s just a name for a variable.
• SkuId is the subparam, or value within the property being checked. When there’s only one value in a collection (as when you check for team-enabled groups), you don’t need to specify a subparam. In the case of assignedLicenses, it is needed because we want to match against the SkuId within the collection of entities in the assignedLicenses property.
• 6fd2c87f-b296-42f0-b197-1e91e994b900 is the value to match against items.

I’m Not a Developer

All of this is second nature to professional developers but not so much to tenant administrators who want to develop some PowerShell scripts to automate operations. This then poses the question about how to discover when lambda qualifiers are needed. I don’t have a great answer except to look for examples in:

• Microsoft’s Graph API documentation.
• Code posted online as others describe their experiences working with the Graph APIs.

And when you find something which might seem like it could work, remember that the Graph Explorer is a great way to test queries against live data in your organization. Figure 1 shows the results of a query for license information.

Exploring the Mysteries of the Graph

One complaint often extended about Microsoft’s documentation for the Graph APIs is that it pays little attention to suitable PowerShell examples. The Graph SDK developers say that they understand this situation must change and they plan to improve their documentation for PowerShell over the next year. Although understandable that languages like Java and C# have been priorities up to now, Microsoft can’t expect the PowerShell community to embrace the Graph and learn its mysteries (like lambda qualifiers) without help. Let’s hope that the Graph SDK developers live up to their promise!

Learn how to exploit the Office 365 data available to tenant administrators through the Office 365 for IT Pros eBook. We love figuring out how things work.

Controlling Document Insights

The Microsoft Graph Insights API proves different views of users and documents:

• Trending: Documents a user has access to which are popular with other users.
• Used: Documents a user has accessed recently.
• Shared: Documents shared with a user, including email and attachments in calendar appointments.

Insights are consumed by many apps and Microsoft 365 components such as MyAnalytics, Workplace Analytics, Viva Insights for Teams, and the Office 365 profile card. Figure 1 is a Microsoft graphic to explain the use of the Insights API and its value to “drive productivity and creativity in businesses.”

Delve and Sharing

Delve was the first app to surface insights, but used Office Graph settings to allow users to decide if they wanted to reveal information about their document-centric activities. Some users never want details of their work exposed, even to people who have access to documents, because they either don’t see the need or because they wish to preserve the confidential nature of the information they work with. They can protect content by assigning a sensitivity label with encryption to confidential documents, but this won’t stop document metadata like titles showing up in insights. The feature settings for Delve therefore have a slider to control showing documents in Delve (trending, used, and shared). When the slider is Off, Delve blocks insights based on documents (Figure 2).

Moving to the New Graph Sharing Controls

In April, I wrote about how Microsoft is replacing Office Graph controls over Item Insights with Microsoft Graph controls. The change is now effective in Microsoft 365 tenants and mean that instead of user-driven control over how the Insights API reveals information, a tenant has:

• An organization-wide setting to control insights privacy (isEnabledInOrganization). This is True if controls are active in the organization or False if not.
• An Azure AD group to control the set of accounts who do not want to use insights (disabledForGroup). Individual users can disable insights through the Privacy options of their MyAccount page. However, this does not add their account to the group.

Access to these settings is available through the Search & Intelligence section of the Microsoft 365 admin center (Figure 3).

The question arises how to find the current set of accounts with the option disabled in Delve so that you can add the accounts to the Azure AD group. As it happens, I was asked this question by a Microsoft customer engineer who wanted to help their customer move to the new Microsoft Graph controls.

The first step is to find the set of accounts with Delve insights disabled. This cannot be done with PowerShell only because no cmdlet exists to retrieve the value of the Delve setting. Instead, we can combine PowerShell with a call to the Graph Users API. Here are the steps:

• Get a set of accounts to check. This can be done with Get-ExoMailbox or Get-AzureADUser. We need the object identifier to make the Graph call. Either of these cmdlets wll return the necessary identifiers. I like the mailbox cmdlet because it has better filtering capabilities.
• Loop through the set of accounts and check the value of the Delve setting (contributionToContentDiscoveryDisabled). If True (insights are disabled), report the account.
• Generate whatever output is required (I usually create a CSV file because of its flexibility).

You can download the script I used to report users with Delve insights disabled from GitHub.

Updating the Group

The next step is to review the report and decide which accounts to add to the Azure AD group used to control item insights. To review the data, open the CSV file generated by the script (Figure 4), and remove any accounts which should not be added to the control group.

We can then use the updated CSV file as the input for a script which:

• Retrieves some tenant details.
• Retrieves the current privacy control settings from the Graph.
• Fetches the current membership of the group (we assume here that one is specified).
• Import the set of accounts from the CSV file.
• Loop through the accounts and add them to the Azure AD group if not already a member.

The essential code to fetch the settings from the Graph and update the membership of the control group looks like this:

$InputCSV = "c:\temp\DelveDisabledAccounts.csv"
$TenantDetails = Get-AzureADTenantDetail
$TenantId = $TenantDetails.ObjectId
$TenantName = $TenantDetails.DisplayName
$Uri = "https://graph.microsoft.com/beta/organization/" + $TenantId + "/settings/iteminsights"
$Settings = Invoke-RestMethod -Uri $Uri -Method Get -ContentType "application/JSON" -Headers $Headers -UseBasicParsing

If ($Settings.isEnabledInOrganization -ne $True) {
   Write-Host "Insights control setting not set for" $TenantName ; break }
Else {
   $DisabledGraphInsightsGroup = $Settings.disabledForGroup }

[array]$CurrentMembers = Get-AzureADGroupMember -ObjectId $DisabledGraphInsightsGroup | Select -ExpandProperty ObjectId

Write-Host "Adding users to the Disabled Graph Insights Group"
$Users = Import-CSV $InputCSV
ForEach ($User in $Users) {

   If ($User.ObjectId -notin $CurrentMembers) {
      Write-Host "Adding" $User.Name
      Add-AzureADGroupMember -ObjectId $DisabledGraphInsightsGroup -RefObjectId $User.ObjectId }
}

I haven’t published a script to GitHub for this purpose because the code is straightforward and simple to plug into an existing script (or add to the bottom of the script mentioned above). Happy Insights!

Learn more about how Office 365 really works on an ongoing basis by subscribing to the Office 365 for IT Pros eBook. Our monthly updates keep subscribers informed about what’s important across the Office 365 ecosystem.

Microsoft Graph PowerShell is Quicker

Without a doubt, using Graph API calls is much faster to retrieve Office 365 than PowerShell cmdlets are. It’s more obvious in complex scripts like the Groups and Teams activity script, which is much faster in its Graph variant (5.1) than its counterpart (4.8) due to Graph API calls replacing cmdlets from the Exchange Online and SharePoint Online modules. Speed matters, especially when a tenant supports thousands of groups, and the Graph API version of the script can process large quantities of groups where the pure PowerShell version struggles to cope.

Seeking an Example to Convert

Good as it is to have a speedier script, the complexity of the activity report is possibly not a good test case to illustrate the decision process that you should go through to decide if it’s a good idea to upgrade a script to use the Graph API, and then measure the improvement. Simplicity is better, so let’s explore what needs to be done to upgrade the Microsoft 365 Groups Membership report script. This script uses the following cmdlets:

• Get-AzureADUser: Fetch the list of Azure AD accounts in the tenant. Microsoft announced their intention to retire the Azure AD module in June 2021, so this is a good example of the kind of script update needed to replace Azure AD calls with Graph API calls.
• Get-UnifiedGroup: Fetch the list of Teams in the tenant.
• Get-Recipient: Fetch the list of Groups a user account belongs to.

Two cmdlets are from the Exchange Online Management module, one is from the Azure AD module. A bunch of other processing is done to filter, sort, and process the data fetched by these cmdlets, but essentially the conversation involves replacing these cmdlets with Graph API calls. Sounds easy.

Using the Graph API

Before any script can use the Graph API, it needs to use an Azure AD registered app. The app serves as the holder for permissions to allow the script to access data. Every registered app has a unique identifier. When you create a registered app, you can generate an app secret. When running the app, we need to know the secret to prove we have permission to use the app.

Before making any calls, you need to know your tenant identifier. This is the GUID for a tenant as returned by the Get-AzureADTenantDetail cmdlet. The Connect-MicrosoftTeams cmdlet also returns this information. Bringing everything together, before we can use an app to access Graph APIs, we need to know the app identifier, app secret, and tenant identifier. You’ll often see code like this in scripts:

$AppId = "a09cf913-5ff9-48a2-8015-f28f2854df26"
$AppSecret = "u6X7_i8K-yhh-b4-z5FEmj_wH_M~nIOz4n"
$TenantId = "22e90715-3da6-4a78-9ec6-b3282389492b"

It’s an important part of working with Graph API apps to understand how permissions work and to ensure that apps receive only the permissions necessary to work with the data they process. In the case of our app, we need:

• Group.Read.All: to read information about Microsoft 365 Groups in the tenant.
• GroupMember.Read.All: to read information about the membership of Microsoft 365 Groups.
• User.Read.All: to read information about Azure AD accounts in the tenant.

Like all programming tasks, it soon becomes second nature to assign the correct permissions, sometimes after browsing Microsoft’s documentation to find the correct permission.

An administrator gives consent to allow the app to use its assigned permissions. Hackers can use OAuth consents as a method to gain permissions to access data, so it’s wise to keep an eye on consents given within an organization, with or without Microsoft’s new App governance add-on for MCAS.

Access Token

Equipped with a suitably permissioned app, app secret, and tenant identifier, the app can request an access token by posting a request to the token endpoint. For Graph API calls to Office 365 data, that’s going to be something like:

https://login.microsoftonline.com/tenant-identifier/oauth2/v2.0/token

The bearer token issued in response confirms that the app has the necessary permissions to access data like Users, Groups, and Sites and whether access is read-only or read-write. The token is included in the authorization header of the requests made to the Graph APIs. Access tokens expire after an hour, so long-running programs need to renew their token to continue processing.

All of this sounds complicated, but once you do it for one script, it becomes second nature to acquire an access token and be ready to start using Graph API calls.

Replacing PowerShell Cmdlets

Like anything else, it takes a little while to become used to fetching data using Graph API calls. You must pay attention to the data that’s fetched. First, to limit the demand on resources, the Graph fetches limited data at one time and you must iterate until no more data is available (a process called pagination). Second, in some cases, the property names used by cmdlets vary to what’s used by the Graph API. For example, the Get-UnifiedGroup cmdlet returns the description of a group in the Notes property whereas the Groups API uses description.

The Graph Explorer is an online Microsoft tool to help developers become accustomed to Graph API syntax and data. You should use the Explorer to test calls before including them in a script. Debugging calls using the Graph Explorer saves a lot of time and heartache. Figure 1 shows the Graph Explorer being used to examine the transitive set of groups returned for a user.

The Graph Explorer is a Graph app. Like any other app, it needs permissions to access data. If a call fails, the Explorer tells you which permissions are missing and you can then consent to the assignment.

Fetching Azure AD Accounts

The first cmdlet we need to replace in the script is the one to fetch the list of Azure AD users in the tenant. In PowerShell, this is:

$Users = Get-AzureADUser -All:$true

The Graph Users API fetches the same data. Unlike the Get-AzureADUser cmdlet, a default set of properties is returned and we must be specific if we want other properties. Here’s the call used.

$Uri = https://graph.microsoft.com/v1.0/users?&`$select=displayName,usertype,assignedlicenses,id,mail,userprincipalname
$Users = Get-GraphData -AccessToken $Token -Uri $Uri

Get-GraphData is a wrapper function to take care of pagination and extraction of data returned by Graph API calls in a form like the PowerShell objects returned by cmdlets. You can make your life easier by including a function like this in any script which interacts with Graph API calls. To see an example of the function I use, download the Graph version of the group membership report script from GitHub.

Fetching Teams

The second call fetches the list of team-enabled groups. The script then creates a hash table to store the teams so that it can be used as a lookup to see if a group is team-enabled when reporting its properties. The PowerShell code uses a server-side filter with the Get-UnifiedGroup cmdlet to return the teams.

$Teams = Get-UnifiedGroup -Filter {ResourceProvisioningOptions -eq "Team"} -ResultSize Unlimited | Select ExternalDirectoryObjectId, DisplayName

The Graph Groups API equivalent uses the same kind of filter:

$Uri = "https://graph.microsoft.com/beta/groups?`$filter=resourceProvisioningOptions/Any(x:x eq 'Team')"
$Teams = Get-GraphData -AccessToken $Token -Uri $Uri

Fetching the Groups a User Belongs to

The last call we make is to find the set of groups a user account is a member of. The Get-Recipient cmdlet is very fast at returning the list of groups based on the distinguished name of an account. The user data returned by Get-AzureADUser doesn’t give us the distinguished name (it is an Exchange Online property), so we must run Get-Recipient twice: once to get the distinguished name, and then use the distinguished name to find the groups.

$DN = (Get-Recipient -Identity $User.UserPrincipalName).DistinguishedName
$Groups = (Get-Recipient -ResultSize Unlimited -RecipientTypeDetails GroupMailbox -Filter "Members -eq '$DN'" | Select DisplayName, Notes, ExternalDirectoryObjectId, ManagedBy, PrimarySmtpAddress)

The Graph Users API can resolve a transitive lookup against groups to find membership information for an account. We can therefor use a call like this:

$Uri = "https://graph.microsoft.com/v1.0/users/" + $user.id +"/transitiveMemberOf"
$Groups = Get-GraphData -AccessToken $Token -Uri $Uri

That’s it. All the other command in the script process data fetched from Azure AD or Exchange Online. Apart from the changes detailed above, the same code is used for both the PowerShell and Graph versions of the script.

Microsoft Graph PowerShell Delivers a Result

Your mileage may vary depending on the backend server you connect to, the state of load on the service, and other factors. My tests, which are surely as reliable as an EPA mileage figure, revealed that the PowerShell version processed accounts at a rate of about 0.6/second each. The Graph version reduced the time to about 0.4/second. In other words, a 50% improvement.

Not every script will benefit from such a speed boost and other scripts will need more work to install Graph turbocharging. But the point is that Graph-powered PowerShell is much faster at processing Office 365 data than pure PowerShell is. Keep that fact in mind the next time you consider how to approach building a PowerShell-based solution for Office 365.

Learn more about how Office 365 really works on an ongoing basis by subscribing to the Office 365 for IT Pros eBook. Our monthly updates keep subscribers informed about what’s important across the Office 365 ecosystem.

Applying Governance to Graph-Based Apps

Update (November 3): The App governance add-on is out of preview and generally available.

In mid-July, Microsoft introduced the preview of an app governance add-on for Microsoft Defender for Cloud Apps (MCAS), describing the new capability as a “security and policy management capability designed for OAuth-enabled apps that access Microsoft 365 data through Microsoft Graph APIs.”

There’s no doubt that tenants can accumulate a collection of Graph-enabled apps over time. The apps come from Microsoft, ISVs, and line of business apps. Indeed, any PowerShell script you write to interact with the Graph APIs gains its permissions through consent granted to an app registered with Azure AD. The net result is that you can end up with hundreds of registered apps, all with Graph permissions, some of which you might not know much about. App governance aims to deliver a structured management framework for those apps, leveraging information taken from Azure AD and MCAS.

Microsoft research clearly shows that attackers use illicit consent grants for Graph-based apps to extract and abuse data. Given the likelihood that organizations will have more Graph-based apps to manage over time, it’s important that administrators understand app usage in their tenant. Unfortunately, the need to review and analyze app usage often falls down task lists, which is the hole the app governance add-on attempts to close.

The Need for MCAS

MCAS isn’t included in any Office 365 plan. Office 365 E5 includes Office 365 Cloud App Security (OCAS), a cut-down version of the full-blown MCAS. Both operate on the same basis of data gathered from user and app activity, but MCAS delivers more functionality and covers more apps. For this pleasure, tenants need to pay more to license MCAS, unless they’ve already invested in Microsoft 365 E5, Enterprise Mobility and Security E5, or one of the other licenses which cover MCAS. You need MCAS to use the new add-on, which doesn’t work with OCAS.

Fortunately, Microsoft offers organizations the chance to run a 30-day trial for MCAS by signing up through the Purchase services section of the Microsoft 365 admin center. After starting the MCAS trial and assigning some licenses to accounts with suitable administrator roles, you can go ahead and start a trial of the add-on. Curiously, the add-on allows a 130-day trial, which might be due to its desire to capture and analyze usage data for apps over a reasonable period. Of course, if you sign up for both trials, MCAS expires after 30 days, and you won’t be able to use the add-on afterwards.

Using App Governance

App governance runs within the Microsoft 365 admin center. If your account isn’t licensed or doesn’t hold one of the necessary compliance roles, you’ll be told this unhappy news if you attempt to access the page. Licensed administrators see a preview of the app situation in the tenant (Figure 1).

I had recently been through an audit of apps based on grabbing app data from Azure AD and reviewing the data through Microsoft Lists, so there are fewer apps present in my tenant than existed previously. As I reviewed the data set, I found a couple of additional apps that I could disable or remove. You can disable an app by viewing its properties through App governance; to remove it, you need to use the Azure AD admin center. Disabling is a good first step to removing a potentially problematic app as you can easily enable the app if someone reports that a business case exists for its use.

In my tenant, app governance detected 33 high privileged apps and 13 overprivileged. Microsoft’s definitions for these categories are:

• High privilege: Consent has been granted to the app for Graph permissions that allow the app to access data or change key settings in the organization.
• Overprivileged: Consent has been granted to the app for Graph permissions the app doesn’t use.

To examine details of an app to understand why it falls into a certain category, click Apps, and peruse the list of apps to select and view the app properties (Figure 2).

As you can see, the portal includes several filters to limit the set of displayed apps. The set of available filters misses one to show disabled apps. This means that if you need to find a disabled app, perhaps to reenable it if the app had been disabled in error, you either need to know the name of the disabled app or do a lot of checking to find the right app.

Probing Permissions

Mmany apps fall into the high privilege category because they read user information. For example, the app used by Microsoft Ignite conference attendees to register has permission to see the user’s email and profile. Apps created by ISVs to read and report tenant data need access to the directory, and that is flagged as a high privilege permission because attacker apps also use the permission to find targets. Even Microsoft’s own Information Barriers app is flagged as high privilege because it has the Directory.Rewrite.All and Groups.Rewrite.All permissions. As always, understanding the context of what an app does is necessary to understand why it needs permissions.

App governance allows tenant administrators to automate checks by creating policies to monitor the creation of overprivileged or high privilege apps. This functionality works like the other alert policies available in the Microsoft 365 compliance center with the exception that the input data focuses on apps rather than actions. As you can see in Figure 1, policies quickly flag new apps which violate criteria. But as pointed out above, you then need to check the app to figure out if a problem really exists.

Permission Glitches

The add-on is a preview, so glitches are expected. For example, app governance flagged an app written to support adding organizational contacts to new user mailboxes as overprivileged. When I examined details of the app (Figure 3), the unused permission is Contacts.ReadWrite. This is odd because that’s the exact permission needed to write a new contact record into a mailbox.

Apart from app details and permissions, App Governance promises to show data about usage and users. Taking users first, I thought this information to be quite useless. The information shown on Figure 4 tells me that 237 consented users exist (Azure AD accounts with data the consents granted to the app covers). This figure includes both tenant and guest accounts and results because an administrator granted consent to the app for the entire organization (if consent is given for individual accounts, they are listed here). The five priority users are those marked as priority accounts. None of the priority accounts (including my own) had any trace of data uploaded or downloaded using the app. Given the app is Microsoft’s Graph Explorer, which I use to test Graph API queries almost daily, this was surprising.

After being disappointed with the data available for users, I didn’t hold out much hope for the Usage (beta) tab. And my expectations were met as precisely nothing showed up here. Instead, App governance informed me that no data was present. Oh well, it’s a preview.

DIY App Governance

As mentioned above, it is relatively simple to perform a DIY audit of Microsoft 365 Graph-enabled apps. Home-grown knowledge of apps used in a tenant is an advantage MCAS can’t deliver, but to exploit that knowledge, some work is necessary to acquire, refine, and understand the app inventory – and to keep on checking on a systematic basis.

App governance extends simple auditing by including policy-based management, categorizing apps based on the permissions they hold, and delivering some insights into app usage. Although still just a preview with all that implies, if your organization has MCAS, the add-on is a useful enhancement. If not, although the need to monitor the granting and usage of permissions in Graph-enabled apps is a real need, you might be able to construct your own method to achieve the goal.

Stay updated with developments across the Microsoft 365 ecosystem by subscribing to the Office 365 for IT Pros eBook. We do the research to make sure that our readers understand the technology.

Combining the Power of the Shell and the Graph

Over the last few months, I’ve written many times about using Microsoft Graph API calls in PowerShell scripts to get real work done. Among the many examples are:

• Reporting the activity of Microsoft 365 Groups and Teams.
• Reporting user activity across multiple workloads.
• Investigating Azure AD Access Reviews.
• Decrypting SharePoint Online documents protected with sensitivity labels.
• Calculate counts of distribution list members.
• Sending email with the Graph.
• Creating organization contacts in user mailboxes.
• Working with Planner data.

In addition, Microsoft has stirred the pot by announcing that they won’t support the Azure AD Graph from June 2022. This affects the Azure AD PowerShell module, one of the most heavily used modules for Office 365 tenant management. And we have examples where Microsoft introduces new features, like tenant privacy controls, which can be controlled only through Graph API calls.

As a result of this activity, I’ve received several questions about how to decide when to use Graph API calls in scripts. And as I due to speak about combining PowerShell and the Graph at the (free) TEC 2021 event in September, it seemed like a good idea to formulate some thoughts about how I approach the issue.

Four Steps to Consider

I use a simple four-step process when writing scripts to automate some aspect of Office 365:

Sketch out the solution: Understand what source data is available and how to access it. Define the expected output and the processing needed to achieve the result. Make an initial selection of PowerShell modules and Graph APIs which might be useful, understanding that some data is only accessible to the Graph (and might need a beta API). Do an internet search to see if anyone has already written code to do what you want or something similar. Never reinvent the wheel if someone else has one to use.

Code in PowerShell first: It’s often wise to write the initial code in PowerShell before introducing any Graph APIs. The code you write might work well enough to be the solution you need without doing any further work. This is often the case when a small amount of data is involved, in which case you don’t need the additional overhead necessary to introduce Graph API calls.

Speed Things Up: Usually, the biggest advantage gained through using Graph APIs is speed, especially when fetching large numbers of objects like user accounts or groups. The next step is to find places in your code where large delays occur to run calls like Get-UnifiedGroup and replace those cmdlets with Graph APIs.

Adjust for Production: Every tenant has their own idea of how to run PowerShell scripts in production. After developing a script which can run interactively, you might need to change it to run as a background process and deal with issues like certificate-based authentication (never store passwords in scripts). Because of the need to adjust scripts for production usage, the code I write for books and articles is to illustrate principles rather than being fully worked-out answers.

The most important point in the checklist is the internet search for code. If you don’t find a suitable script to remove the need to create anything new, you’ll probably find the basis or starting point for what you want to do. It astounds me that people post questions in forums when it is perfectly obvious that they haven’t done the basic research to uncover details which can help solve their problem. Unfortunately, too many people expect answers to be handed to them on a plate and aren’t prepared to learn through experimentation and failure. I spend most of my time in the latter state.

Take Things Slowly

The Microsoft Graph isn’t scary. It’s there to be used and like any other tool, it should be used at the right time. PowerShell gets most things done really well when it comes to tenant management. It has its limitations, some of which the Graph can fill in. Starting with simple tasks and moving forward to more complex issues is a great way to learn how to use the Graph with PowerShell. Your task is to provide the brainpower to combine the two to get things done most effectively.

Learn how to exploit the Office 365 data available to tenant administrators through the Office 365 for IT Pros eBook. We love figuring out how things work.

Usual Mix of Missed Dates

As is the norm for Microsoft engineering groups when the end of the company’s fiscal year approaches on June 30, a mad dash happens to complete and ship functionality. This has nothing whatsoever to do with performance reviews and associates bonuses. Instead, it’s all done in the spirit of wanting to make customers happy. Some features don’t make it in the rush to deliver commitments, which is why on July 1 you invariably see Microsoft issue a bunch of updates to previous new feature notifications published in the Microsoft 365 message center. 2021 is no different and the table below lists some of the more interesting delays.

Retirements in July

Two retirements for Office 365 components loom on the horizon for July:

• Skype for Business Online retires on July 31, 2021 (MC266078). If an organization needs to extend support in their tenant, they can ask Microsoft for an extension. No guarantee exists that Microsoft will agree, so it’s best for organizations to move on and transition to Teams.
• Site mailboxes, the unloved and unwanted child of SharePoint and Exchange, get the chop on July 15, 2021 (MC266256). If you haven’t moved data from these mailboxes to somewhere better (with a long-term future), you’ll lose the content.

New Graph Insights Control

In May, Microsoft replaced the privacy settings previously in the Delve app with Graph-based controls. The controls introduced at the time were:

• The isEnabledInOrganization setting to control if a tenant allows Item Insights (for example, information about the documents a user has recently worked on).
• The disabledForGroup setting to specify the GUID of an Azure AD group whose members are excluded from Item Insights. This setting only applies when a tenant allows Insights.

On July 1, Microsoft published MC266073 to remind everyone that the new Item Insights controls are now active. They also introduced a new personal-level setting in the Settings & Privacy section of the MyAccount page (Figure 1) to allow users to exclude themselves from Item Insights. The setting is only available when an organization enables Item Insights, and the user account isn’t a member of the group specified in the disabledForGroup control. Users who don’t want to share information about what they’ve been working on should set the slider to Off.

Setting the privacy control at a personal level does not add the user account to the membership of the Azure AD group specified in disabledForGroup.

Another Year, More Changes

We’re only a few days into Microsoft’s FY22 year. Already it looks as if lots will change. We’ll be busy tracking that change and updating the Office 365 for IT Pros eBook (2022 edition). Make sure you stay abreast of important developments in your Office 365 tenant by subscribing to Office 365 for IT Pros.

Grappling with PowerShell

Sometimes I hate PowerShell. Not the language itself, just my ineptitude, or my inability to remember how to do things, or the speed of some cmdlets which deal with objects like mailboxes and groups. It’s not that the cmdlets are inefficient. They do a lot of work to retrieve information about objects, so they are slow.

This is fine for ad-hoc queries or where you only need to process a couple of hundred mailboxes or groups. The problem is accentuated as numbers grow, and once the need exists to process thousands of objects, some significant time is spent waiting for cmdlets to complete, meaning that scripts can take hours to run.

Microsoft has made significant progress in the Exchange Online PowerShell module to introduce faster cmdlets like Get-ExoMailbox and Get-ExoMailboxStatistics. These REST-based cmdlets are faster and more robust than their remote PowerShell cousins and these improvements are ample justification for the work needed to revisit and upgrade scripts. The module also supports automatic renewal of sessions to Exchange Online and the Security and Compliance endpoints, so it’s all good.

The Sloth of Get-UnifiedGroup

Things aren’t so impressive with Get-UnifiedGroup, which retrieves details about Microsoft 365 Groups. Reflecting the use of Microsoft 365 groups, Get-UnifiedGroup is a complex cmdlet which assembles details from Azure AD, Exchange Online, and SharePoint Online to give a full picture of group settings. Running Get-UnifiedGroup to fetch details of 200 groups is a slow business; running the cmdlet to fetch details of 10,000 groups is a day-long task. The Get-Team cmdlet is no speedster either. In their defense, Microsoft designed these cmdlets for general-purpose interaction with Groups and Teams and not to be the foundation for reporting thousands of objects over a short period.

If you only need a list of Microsoft 365 Groups, it’s also possible to create the list using the Get-Recipient cmdlet.

Get-Recipient -RecipientTypeDetails GroupMailbox -ResultSize Unlimited

Creating a list of groups with Get-Recipient is usually much faster than creating it with Get-UnifiedGroup. However, although you end up with a list of groups, Get-Recipient doesn’t return any group-related properties, so you usually end up running Get-UnifiedGroup to retrieve settings for an individual group before you can process it. Still, that overhead can be spread out over the processing of a script and might only be needed for some but not all groups.

The Graph is Quicker

Which brings me to the Microsoft Graph API for Groups. As I’ve pointed out for some years, using Graph APIs with PowerShell is a nice way to leverage the approachability of PowerShell and the power of the Graph. The script to create a user activity report from Graph data covering Exchange, SharePoint, OneDrive, Teams, and Yammer is a good example of how accessible the Graph is when you get over the initial learning curve.

Three years ago, I wrote about a script to find obsolete Teams and Groups based on the amount of activity observed in a group across Exchange Online, SharePoint Online, and Teams. In turn, that script was based on an earlier script which processed only Office 365 Groups. Since then, I have tweaked the script in response to comments and feedback and everything worked well. Except that is, once the script ran in large environments supporting thousands of groups. The code worked, but it was slow, and prone to time-outs and failures.

Speeding Up the Report

The solution was to dump as many PowerShell cmdlets as possible and replace them with Graph calls. The script (downloadable from GitHub) now uses the Graph to retrieve:

• SharePoint Online site usage data for the last 90 days.
• A list of Microsoft 365 Groups and a list of team-enabled groups.
• The owners of a group.
• The display name of the owners (because the first call returns their Azure AD identifier).
• Some extended properties of a group not fetched when the group list is returned.
• Counts for group members and guests (easier to do since Microsoft added advanced queries for directory objects in October 2020).
• Archived status for teams.

The result is that the script is much faster than before and can deal with thousands of groups in a reasonable period. Fetching the group list still takes time as does fetching all the bits that Get-UnifiedGroup returns automatically. On a good day when the service is lightly loaded, the script takes about six seconds per group. On a bad day, it could be eight seconds. Even so, the report (Figure 1) is generated about three times faster.

Results - Teams and Microsoft 365 Groups Activity Report V5.1
--------------------------------------------------------------   
Number of Microsoft 365 Groups scanned                          : 199    
Potentially obsolete groups (based on document library activity): 121    
Potentially obsolete groups (based on conversation activity)    : 130      
Number of Teams-enabled groups                                  : 72    
Percentage of Teams-enabled groups                              : 36.18%

Total Elapsed time:  1257.03 seconds
Summary report in c:\temp\GroupsActivityReport.html and CSV in c:\temp\GroupsActivityReport.csv

The only remaining use of an “expensive” cmdlet in the script is when Get-ExoMailboxFolderStatistics fetches information about compliance items for Teams stored in Exchange Online mailboxes. The need for this call might disappear soon when Microsoft eventually ships the Teams usage report described in message center notification MC234381 (no sign so far despite a promised delivery of late February). Hopefully, that report will include an update to the Teams usage report API to allow fetching of team activity data like the number of conversations over a period. If this happens, I can eliminate calling Get-ExoMailboxFolderStatistics and gain a further speed boost.

Some Extra Work but Not Onerous

The downsides of using the Graph with PowerShell are that you need to register an app in Azure Active Directory and make sure that the app has the required permissions to access the data. This soon becomes second nature, and once done, being able to process data faster than is possible using the general-purpose Get-UnifiedGroup and Get-Team cmdlets is a big benefit when the time comes to process more than a few groups at one time.

Microsoft Graph PowerShell SDK is the Future

For anyone who’s ever used the Azure AD or Microsoft Online Services (MSOL) PowerShell modules to write PowerShell code to automate some aspect of tenant administration, Microsoft’s June 2 announcement about their future direction for Azure AD PowerShell was big news. In a nutshell, Microsoft is focusing on the Graph APIs for identity management. As a consequence, any software which leverages the Azure AD Graph API, like the Azure AD module, is on the runway to deprecation.

Important Points for the Next Year

Last year, Microsoft announced that they would no longer support or provide security updates for the Azure AD Graph API after 30 June 2022. Now they are being more specific about what this end of support decision means for customers. The following points are important:

• Microsoft’s future investments for identity management are focused on the Microsoft Graph SDK for PowerShell. This is a wrapper around the Graph APIs and is already in use for purposes like setting tenant privacy options for the Insights API.
• The Azure AD and MSOL modules will not be supported for PowerShell 7.
• New identity APIs will be available through the Microsoft Graph PowerShell SDK (Figure 1).
• Microsoft’s investments will center on user, group, and application management, plus role-based access control (RBAC), which is important in terms of making sure that administrators don’t need all-powerful permissions to get work done. Microsoft 365 uses an increasing number of role groups to assign administrative work to different accounts.
• Microsoft also says that they will invest in usability for the Microsoft Graph PowerShell SDK, which is a good thing because the SDK cmdlets aren’t quite as approachable as those in other modules. The documentation is not in good shape either. See my articles covering basic Azure AD user account management and group management for details.

Microsoft says that their goal is that “every Azure AD feature has an API in Microsoft Graph so you can administer Azure AD through the Microsoft Graph API or Microsoft Graph SDK for PowerShell.” They don’t say that every feature will be accessible through the Microsoft Graph PowerShell SDK. In some cases, you’ll need to run pure Graph API calls, but that’s easily done using PowerShell (for an example, see this article on accessing Azure AD access reviews from PowerShell.

Update August 1, 2022: Microsoft has pushed out the previously announced retirement date for the license management cmdlets in the Azure AD and MSOL modules (August 26, 2022) to March 31, 2023. They have delayed the retirement of the Azure AD Graph API until the end of 2022 to give customers extra time to adjust.

It’s Different With the Graph

The net takeaway is that tenants need to review any PowerShell scripts which use the Azure AD or MSOL modules to prepare plans to upgrade scripts to use the Microsoft Graph PowerShell SDK or Graph API calls in the future. Given the number of Office 365 tenants, the pervasive use of PowerShell to automate operations, and the core position of Azure AD in those operations, it’s likely that millions of scripts will need upgrades. I know that I have a bunch of scripts to review and will discuss how the upgrade process proceeds in future articles. Already, I know it won’t be simply a case of replacing all occurrences of Azure AD cmdlets with equivalent Graph SDK calls, like replacing Get-AzureADUser with Get-MgUser. Parameters and output are likely to be different and code will need to be adjusted to cope.

While upgrading scripts is a big job, each script is a one-time activity. Interactive access is another issue. Today, it’s easy to run Connect-AzureAD to connect to Azure AD and then run whatever cmdlets you need to interrogate the directory. The equivalent actions with the SDK are:

First, you connect to the Graph and set the scope (permissions) needed to interact with Azure AD. Unless a suitable access token is available, this starts a device authentication sequence.

Connect-MgGraph -Scopes "User.Read.All","Group.ReadWrite.All"
To sign in, use a web browser to open the page https://microsoft.com/devicelogin and enter the code D7DPGD3WL to authenticate.

Opening a web page and inputting the code causes another dialog to appear to confirm consent for the operation (Figure 2).

After consent is granted, you can then go ahead and issue commands. For example, here’s how to fetch a list of guest accounts:

$Guests = Get-MgUser -Filter "UserType eq 'Guest'"

Like most Graph commands, the amount of data returned is constrained to 100 items, so if you want more, you need to specify the All parameter.

The bottom line is that some more up-front thought is needed (to set permissions) before connecting to the Graph SDK and that the authentication flow is not as seamless as it is when running Connect-AzureAD. No doubt this is an area where Microsoft might look at to remove some rough edges.

Time to Prepare Upgrades

Losing support for the Azure AD and MSOL modules sometime in 2022 is a concern, but we’ve seen other instances when Microsoft has extended support to allow customers extra time to get work done, and anyway, losing support doesn’t mean that code will suddenly stop working. Scripts will continue to run. You just won’t be able to ask Microsoft to fix bugs.

One thing you can guarantee in the cloud is that change happens. This is just another example of how that change occurs.

The Office 365 for IT Pros team will document our learning with upgrading PowerShell scripts from the Azure AD module to use the Microsoft Graph PowerShell SDK in the months ago. It should be fun… Subscribe now to make sure that you stay abreast of developments.

The Option to Anonymize User Data and the Fuss About Microsoft 365 Productivity Score

Last week, I wrote about the criticism leveled at the Microsoft 365 Productivity Score feature. Leaving the hysteria aside, the biggest point missed in the criticism is that the usage data presented in the new feature has existed and been accessible to organizations for a very long time. Aside from the usage reports available in the Microsoft 365 admin center (originally based on the reporting service, which goes back to 2015), Microsoft made the Power BI analytics pack for Office 365 available in 2017 to allow tenants to do more deep-dive analysis of the data.

The standard usage reports and the Power BI app both use Graph data as their consistent source of knowledge. Most ISV reporting applications do the same, with the difference being that ISVs typically extract and process the Graph data before storing it in their own repositories to keep it longer than the 180 days allowed by Microsoft. It was therefore curious that no one has protested the acquisition, storage, and reporting of usage data over the last five years.

Anonymizing User Data in Reports

Moving on, if organizations wish to protect the privacy of usage data, they can anonymize the data by replacing user, group, and site names in reports by selecting an option in the Reports section under Org settings in the Microsoft 365 admin center (Figure 1).

After the option is selected, all views of usage data have group, user, and site names replaced with system generated values (Figure 2).

For SharePoint site usage, the site URL and site owner are obscured (Figure 3).

Anonymize User Data for Any Graph-Based Application

The protection extends to applications which make Graph API calls to fetch usage data. For instance, the extract below shows a user’s Teams usage data for 90 days returned by the Microsoft Teams user activity reports API.

Report Refresh Date        : 2020-12-02
User Principal Name        : FE7CC8C15246EDCCA289C9A4022762F7
Last Activity Date         : 2020-12-02
Is Deleted                 : False
Deleted Date               :
Assigned Products          : POWER BI (FREE)+OFFICE 365 E5 WITHOUT AUDIO
                             CONFERENCING+ENTERPRISE MOBILITY + SECURITY E5+BUSINESS APPS
                             (FREE)+MICROSOFT POWER AUTOMATE FREE
Team Chat Message Count    : 64
Private Chat Message Count : 233
Call Count                 : 14
Meeting Count              : 93
Has Other Action           : No
Report Period              : 90

Choosing the option to anonymize user data like this makes reports less useful because it’s all but impossible to apply context to the data. We can tell that some users are more active than others, but who are the active users and why are they more active than others? Do they use Teams more than email or do they still like email? Is Yammer in use? Are we seeing growth in cloud-based document storage? Do we see more traction in some parts of the company than others.

Anonymizing data also creates some coding challenges. For instance, because user principal names are returned, my User Activity Report script can’t fetch sign in information from Azure AD, which means that another piece of the usage puzzle is missing (Figure 4).

Anonymizing Usage Data Wisely

Obscuring usage data is a good thing to do as a default. It stops people casually browsing information that they might not need or should not see. But if you want accurate data that can be interpreted and used for planning purposes, to improve the effectiveness of your investment in Office 365, or to track down unused licenses that you shouldn’t be paying for, a global administrator can switch the setting back to permit reports to include full information for a limited period. After you’re finished extracting and reporting the data, you can restore anonymity to user data.

Learn much more about reporting Office 365 activity in the Office 365 for IT Pros eBook.

Granting Consent for Data Access by Third-Party and LOB Apps

Described in Office 365 notification MC222892 (September 26), Microsoft has made several important changes to the way that third-party apps are managed in the Teams admin center. The changes are linked to Microsoft 365 roadmap item 67140 and are now available.

The Teams apps section of the admin center supports management of apps and the permission and setup policies used to deploy apps to users. The first change is that the listing of apps includes a permissions column to show when a third-party app needs permission, with the idea being that an admin can take care of consent centrally and so avoid the need for end users to have to seek consent when they want to use an app.

Apps published by Microsoft don’t need to be granted consent. Some third-party apps don’t need consent either because they do not interact with Microsoft 365 data like user accounts or sites. For instance, the Adobe Sign app allows users to sign documents with that service without accessing any Microsoft 365 data.

The Need for Permissions

Third-party apps or LOB apps created by a tenant can access Microsoft 365 data with the Microsoft Graph, but only if they receive permission to access the data. Microsoft Graph divides permissions into sets of actions that an app can perform. When you see View details in the Permissions column, you know that the app needs administrator consent (on behalf of the tenant) to access data via the Graph.

To give consent, select an app and look at the Permissions tab in its details and then Review permissions and consent. You must be able to sign in as a tenant administrator to give consent. Once signed in, you’ll see the permissions requested by the app. Figure 2 shows that the chosen app wants to read user profile information from Azure AD. Be aware that you’re granting consent for org-wide access to the requested information. If you’re happy that the app should have access to this data, click Accept.

When an app has received consent, you’ll see a notice to that effect under Org-wide permissions in the Permissions tab.

Azure AD App Registration

Apps that receive consent are registered with Azure AD. You can find details of all the apps registered in your tenant in the Enterprise applications blade of the Azure AD portal. Figure 3 shows details of an app which received consent through the Teams admin center. You can revoke permissions from an app at any time.

Resource Specific Consent

Office 365 notification MC218561 was announced in July (Microsoft 365 roadmap item 56605) to say that teams owners could give consent to apps to access data in the teams they managed. This feature is known as resource-specific consent (RSC) because the consent is limited to permissions for a specific resource (a group/team). Limiting the scope of the permissions assigned to an app to what it needs to function instead of giving it org-wide access makes a heap of sense.

Now fully deployed across Office 365, RSC is a Teams feature controlling access to team settings, channels, messages, apps, tabs, and membership. It depends on the tenant settings in the Consent and Permissions section of the Enterprise applications blade in the Azure AD portal (Figure 4). See this page for more information.

The ability to give resource-specific consent can be limited to a set of team owners rather than all team owners in the tenant.

Some apps don’t need access to data drawn from across the tenant and only need permissions to interact with specific Teams objects from the set supported by RSC (Figure 5).

You’ll recognize these apps because the RSC permissions they need are listed in the permissions tab of the app details. In Figure 6 we can see that the app needs to read a team’s settings, membership, and messages and create channels.

Add App to a Team

The last feature allows Teams admins to add apps to target teams to avoid the need for team owners to install the apps. This a preview feature that only works for apps designed to be installed within a team (normally accessed via a channel tab). By comparison, Teams app setup policies allow organizations to make apps available to users on a personal basis to use via the app navigation bar.

If you see that an app has “team” included in its capabilities listed under the About tab, you know it supports team scope. Template Chooser, Trello (Figure 7), and Zoho CRM are examples of apps with team scope.

To install an app into a team, select the app in the Manage Apps screen and then choose Add to team. You can then select the team to install the app into (Figure 8).

For more information, see the Teams documentation.

More Information About Apps

Given the growing number of apps in the Teams app store (760 as I write this), it’s obvious that a solid management framework is needed to control third-party apps, especially in how these apps use the Microsoft Graph to access data. The implementation of permission management is solid and is a very useful addition to the Teams admin center.

For more information about app permissions, consent, and RSC, view the Ignite session about Navigating the Microsoft Teams App Lifecycle (app permissions and consent is covered from about 34:20 in the video).

Managing Teams is what Chapter 12 of the Office 365 for IT Pros eBook is all about. You’ll find lots more interesting and useful information in Chapter 12 and all the other chapters of the book.

Gathering Data for Multiple Workloads to Understand User Activity

For the last few months, I have been dabbling with a PowerShell script to extract and report usage data for multiple Office 365 workloads from the Microsoft Graph. The idea is that an Office 365 user activity report generated by fetching activity data from all the workloads reported in the Graph helps administrators to figure out if accounts are in use and if so, what they are used for. If an account isn’t in use, then you might remove it and save some licenses.

One of the joys of PowerShell is how quickly you can put a solution together. The corollary is sometimes that the solution isn’t as efficient as it could be, which often happens when you’re not a professional programmer. When I write a script, the most important thing is often to illustrate a principle and show how something works. When PowerShell scripts are deployed into production, they’re usually upgraded and improved by programmers to meet organizational standards and fit in with other scripts used to manage the infrastructure. For this reason, I don’t bother too much with tweaking for performance.

This script is different. It’s been picked up by several tenants who reported that the script works but it’s slow when asked to process data for thousands of accounts. This deserved some investigation which produced some improvements, such as using PowerShell’s Where method to filter data.

PowerShell Hash Tables

But PowerShell is not a database and storing data about account usage in PowerShell list objects only scales so far. There are many web articles covering PowerShell performance with large amounts of data, many of which point to using hash tables because they are very efficient for finding and retrieving data (see this article about how to use hash tables).

A hash table is a collection of key/value pairs. The keys are unique, and the values are often some information associated with the key. For instance, because Office 365 objects like groups and sites store sensitivity labels as GUIDs, I often create a hash table composed of the GUID (key) and label display name (value) which I can then use to interpret the GUIDs stored in objects. Here’s what the code looks like:

$Labels = Get-Label # Get set of current labels
$HashLabels = @{} # Create hash table
$Labels.ForEach( { # Populate the hash table with the GUID and display name of each label
       $HashLabels.Add([String]$_.ImmutableId, $_.DisplayName) } )

Anytime I need to find the display name of a label, I can do something like this:

$GUID = (Get-UnifiedGroup -Identity “Office 365 for IT Pros”).SensitivityLabel.GUID
Write-Host “Display name of label is” $HashLabels[$GUID]
Display name of label is Limited Access

Apart from their usefulness in situations like described above, hash tables are very fast when you use keyed access. Speed being of the essence when thousands of records are to be processed, I decided to investigate if hash tables could replace the list objects used by the script.

Keys and Values

Finding a key is no problem because the user principal name is unique for each account. Figuring out how to store all the data in the hash table value was another matter. That is, until I noticed that: ”the keys and values in a hash table can have any .NET object type…” In other words, you’re not limited to storing simple values in a hash table.

When the script extracts usage data for a workload (like Teams or Exchange) from the Graph, it processes each record to create a list of accounts and their usage data for that workload. After some experimentation, I was able to populate the hash table by:

• Creating an array of the usage data for the workload for an account.
• Appending the array to the existing usage data extracted from other workloads for the account (as stored in the hash table).
• Writing the updated array back into the hash table.

This might be inelegant, but it works. After all workloads are processed, the result is a hash table keyed on the user principal name with a value composed of an array containing the usage data for all workloads for that user. Access to the data is via the user principal name. For example:

$datatable["Kim.Akers@Office365itpros.com"]

TeamsUPN             : Kim.Akers@office365itpros.com
TeamsLastActive      : 05-Sep-2020
TeamsDaysSinceActive : 5
TeamsReportDate      : 07-Sep-2020
TeamsLicense         : POWER BI (FREE)+ENTERPRISE MOBILITY + SECURITY E5+OFFICE 365 E5 WITHOUT
                       AUDIO CONFERENCING
TeamsChannelChats    : 7
TeamsPrivateChats    : 10
TeamsCalls           : 0
TeamsMeetings        : 5
TeamsRecordType      : Teams

ExoUPN             : Kim.Akers@office365itpros.com
ExoDisplayName     : Kim Akers
ExoLastActive      : 20-Aug-2020
ExoDaysSinceActive : 21
ExoReportDate      : 08-Sep-2020
ExoSendCount       : 8
ExoReadCount       : 19
ExoReceiveCount    : 392
ExoIsDeleted       : False
ExoRecordType      : Exchange Activity

The display is truncated here to show two of the six workload usage data extracted for an account.

Creating the report is then a matter of processing each account to extract the information and format the data. To do string comparisons and other calculations, I found that it was necessary to use the Out-String cmdlet to make the properties taken from the array into trimmed strings. It might be something to do with the way that the hash table values are stitched together from multiple arrays.

Faster Performance

After changing to hash tables, I observed a 70% performance gain in script execution time in my (small) tenant. I expect a much better gain in larger tenants where the advantages of hash table access become more pronounced. This feeling was realized in a test against 20K accounts which proved that the script is now capable of processing at circa 1,000 accounts per minute (Figure 1).

Update September 18: I received a note saying that the script processed 26,808 accounts at the rate of 3184.71 per minute!

The time required to fetch data from the Graph is the same as previous versions as is the time to prepare data for processing. All the improvement is in the report generation, which is where the hash tables excel. The tenant who processed the script against 20,000 accounts used the Office 365 user activity report (example shown in Figure 2) to identify 70 accounts assigned Office 365 E5 licenses that can now be reallocated or released (a potential saving of $29,400 annually).

The Office 365 user activity report script is available from GitHub. If you have a suggestion for improving the performance further, please let comment on GitHub.

OK, we should be writing text for the Office 365 for IT Pros eBook instead of trying to work out how to speed up PowerShell scripts. But you learn a lot about an infrastructure when you program against it, so we’ll keep on scripting…

The GetGraphUserStatisticsReport.PS1 Script

Last June, I wrote about a PowerShell script to interrogate the Microsoft Graph to retrieve usage data from workloads like Exchange Online, SharePoint Online, and Teams. Some of this data is available via PowerShell cmdlets like Get-ExoMailboxStatistics and Get-SPOSite, but using the Graph is usually faster.

Find Inactive Accounts

The idea behind the script is that if you retrieve usage information from a bunch of workloads, you have the basis to figure out what accounts are in active use. Potentially, you might even find some accounts that haven’t been used for a while that you can archive or remove to save some Office 365 licenses. Figure 1 shows the script output as viewed through the Out-GridView cmdlet.

The original script worked, but it had some performance issues once people tried to run it in tenants with thousands of accounts. Long-running PowerShell scripts are not usual, but the issues people ran into pointed to unreasonable delays.

GitHub Contributors

One of the advantages of storing scripts on GitHub is that others can work with the code to detect where problems might lurk. In this case, suspicion fell on how the script processed data in the array storing information fetched from the Graph. It was suggested that better results might be achieved by changing to a PowerShell DataTable object. I did some work to test the assertion but didn’t see a huge gain, probably because I don’t have a big enough tenant to test.

In any case, a simpler fix became obvious. Instead of scanning the big list containing all the records for the records (usually six) for each user, it’s much better to extract the records for the user into an array and use that array. The change is made in version 1.2 of the script, which is now available to download from GitHub. (update: The current version of the script is 2.0).

It would be helpful if those who have admin permissions in tenant with more than a few thousand accounts could test the script (after carefully checking that the script is OK to run in your tenant, creating the necessary registered app in Azure AD, and so on). If you find problems, record them in GitHub and the community will have a chance to work the issues.

V1.3 Update

We’ve just updated the script to V1.3 to incorporate another speed tweak. The Where method is more appropriate and faster when you only want to extract data from a set and don’t need to feed that data through the pipeline. The script extracts records for each user to create an assessment of the user’s activity and using the Where method is faster… A good lesson learned!

Learn more about Office 365 tenant administration by subscribing to the only eBook that’s updated monthly: Office 365 for IT Pros.

Remove the Need for Administrator Access to Create a SharePoint Site Report

Following an article I wrote about using PowerShell to report SharePoint Online site storage usage, a reader asked if it is possible to create such a report without needing to sign in with a SharePoint Online administrator account. They’d like non-admins to be able to report data but don’t want them to run PowerShell scripts or access the admin center.

Global Reader Role and Usage Reports

The Global Reader role allows non-privileged access to reporting data exposed in some interfaces like the Microsoft 365 Admin Center. The reports section of the admin center (Figure 1) includes reports on SharePoint user and site activity.

The Admin Center includes an export option to download data as a CSV file. If you select the site usage report, this seems promising, until you realize that the report includes redirect sites and even the SharePoint Online Tenant Fundamental Site created when the tenant is initialized. Some of the fields output could be better formatted too.

Usage Reports Based on the Graph

The usage reports available in the Microsoft 365 Admin Center use the Graph reports API to generate their data. It’s very easy to write a quick and dirty PowerShell script to call the SharePoint Site Storage Usage API to return the same data as you see in the usage reports. Once the data is downloaded, you can manipulate it to meet your needs.

Granting Permissions for the Graph

To gain access to the Graph, PowerShell needs to use an app to authenticate with the Reports.Read.All and Sites.Read.All permissions. The basic idea is that you register an app for your tenant to use with PowerShell and generate an app secret to use for OAuth authentication. You then assign the necessary Graph permissions to the app. The request to add the permissions to the app must be approved by an administrator before the permissions can be used. You can then use the app identifier and secret to generate an access token for the Graph which contains the permission. Read this post for detailed steps of creating such an app for PowerShell to use.

Update: The current version of the script has been updated to use the Microsoft Graph PowerShell SDK. This removes the requirement to use an app. The information described below shows how to use an app to authenticate and make Graph API requests. The same results are obtainable using the Graph SDK.

Accessing the Graph

After the app is authorized with the necessary permissions, we can use it with PowerShell. This snippet:

• Populates variables with the app identifier, secret, and tenant identifier. These values are unique to an app and to a tenant. You’ll have to change them for your tenant for this code to work.
• Builds a request to get an access token.
• Parses the returned token.

$AppId = "e716b32c-0edb-48be-9385-30a9cfd96155"
$TenantId = "c662313f-14fc-43a2-9a7a-d2e27f4f3478"
$AppSecret = 's_rkvIn1oZ1cNceUBvJ2or1lrrIsb*:='

# Build the request to get the OAuth 2.0 access token
$uri = "https://login.microsoftonline.com/$tenantId/oauth2/v2.0/token"
$body = @{
    client_id     = $AppId
    scope         = "https://graph.microsoft.com/.default"
    client_secret = $AppSecret
    grant_type    = "client_credentials"}

# Request token
$tokenRequest = Invoke-WebRequest -Method Post -Uri $uri -ContentType "application/x-www-form-urlencoded" -Body $body -UseBasicParsing
# Unpack Access Token
$token = ($tokenRequest.Content | ConvertFrom-Json).access_token
$headers = @{Authorization = "Bearer $token"}
$ctype = "application/json"

With a token, we can issue the Graph request to fetch the SharePoint Online storage usage data:

# Get SharePoint files usage data
$SPOFilesReportsURI = "https://graph.microsoft.com/v1.0/reports/getSharePointSiteUsageDetail(period='D7')"
$Sites = (Invoke-RestMethod -Uri $SPOFilesReportsURI -Headers $Headers -Method Get -ContentType "application/json") -Replace "", "" | ConvertFrom-Csv

All that remains to be done is to parse the returned data and generate a report (a CSV file). You can download the script from GitHub. As always, the code is bare-bones and doesn’t include much in terms of error checking.

I also output the report data to Out-GridView (Figure 2) as it’s the easiest way to browse the information.

Advantages and Disadvantages

The big advantage of this approach is that no dependency exists on cmdlets in the PowerShell module for SharePoint Online or an administrator account. All the code is basic PowerShell that can be run by any user.

Because this approach uses data fetched from the Graph, the code is fast too – much faster than the version based on the SharePoint Online cmdlets, and the speed advantage becomes larger as the number of sites grows. This is because the Graph generates the report data and has it ready for fetching while the other approach requires you to generate the data for each site with the Get-SPOSite cmdlet. On the other hand, the Graph data is at least two days old, something that might not be too much of a concern when reviewing storage usage.

The downside is that the Graph usage data includes a limited set of properties. Some useful properties, like site files, active files, and views, aren’t returned by the Get-SPOSite cmdlet, but Get-SPOSite returns information like the site title, group identifier (to get a list of site owners), and sensitivity label among others.

Combine to Get Both

Combining data fetched from the Graph with that fetched by Get-SPOSite is the best of both worlds, even if you’ll need to use a SharePoint administrator account. The question is what data are needed. If you really need the extended information about a site, you’ll have to use the SharePoint Online module. But if all you need is simple storage data, the Graph can provide that information quickly, albeit if it’s slightly out-of-date.

Creating a List of Teams

I’ve written a couple of articles about using Microsoft Graph queries with PowerShell to access data that you can’t normally get to with cmdlets. For instance, this example explains how to report the somewhat bizarre email addresses assigned to Teams channels. When a channel is email-enabled, people can post to the channel by sending email to the assigned address, which is a good way to introduce information into Teams. Apart from being posted as new topics in the channel, messages are captured in the channel’s folder in the SharePoint document library belonging to the team.

Limited Data Returned by Design

But as a comment to the article notes, when you use the Invoke-WebRequest cmdlet to send a Graph command to fetch information about the set of Teams in a tenant, the Graph responds with 100 teams. This is what the Graph intends to do because it doesn’t want the response to be too large. And the response is good enough to prove the principle of working with the Graph through PowerShell. However, once you get to production, you probably need to deal with more than 100 teams and need to be able to fetch all the teams in the tenant and process them.

Processing Four Thousand Teams

A few days ago, Mike Tilson posted a note in the Office 365 Facebook group asking if it is “possible to run a report (hopefully via PowerShell) to report on what Microsoft Teams apps (third party apps like Polly) are installed across your environment?” As it happens, the esteemed technical editor for the Office 365 for IT Pros eBook, Vasil Michev, had written a script to report on apps and tabs. I took his script and made (in my mind) some improvements, and I gave Mike a copy of the script.

Mike’s response was that the script worked great in a small environment but had 2,000+ teams (among 4,000-odd groups) to report on in production. The code needed to be upgraded to process larger numbers.

NextLink is the Key

The solution is a thing called a nextlink, available when Microsoft Graph queries have more data to provide to clients than is returned to the original request (server-side paging). A page is the set of data returned for a Graph call and several pages might need to be retrieved to fetch the full set of objects you want to process The documentation says “When a result set spans multiple pages, Microsoft Graph returns an @odata.nextLink property in the response that contains a URL to the next page of results.” In effect, the nextlink tells the Graph the next set of data to return if an application wishes to request it following its first call. To be sure that you get all data, you need to fetch it page by page until the nextlink is null. This is called pagination.

If you use the Graph Explorer to play with Microsoft Graph queries, you’ll see that a nextlink turns up in calls like “all groups in my organization.” You know it’s a nextlink because it includes the term “skiptoken” as in https://graph.microsoft.com/v1.0/groups?$skiptoken=X%274453707402 (a real nextlink is much longer). Microsoft Graph queries can generate nextlinks after fetching less than 100 items. For instance, the default number of folders retrieved from a mailbox is 10.

Solving the Problem

The uprated code:

• Makes the original call to fetch teams and stores the object identifier and display name for each team in a hashtable. A hashtable is suitable when you only need to store two properties. If you need to store more, create and populate a generic list instead.
• Checks if a nextlink is returned.
• If a nextlink is found, make another call to the Graph to fetch another set (page) of data and store those items in the hashtable. This call uses the nextlink to tell the Graph where to start retrieving data.
• Continue until the nextlink is null, meaning that no more data remains to be fetched.

Here’s the code, based on an original solution created by Mike Tilson.

$Uri = "https://graph.microsoft.com/V1.0/groups?`$filter=resourceProvisioningOptions/Any(x:x eq 'Team')"
[array]$Teams = Invoke-WebRequest -Method GET -Uri $Uri -ContentType "application/json" -Headers $Headers | ConvertFrom-Json
If ($Teams.Value.Count -eq 0) { Write-Host "No Teams found - exiting!"; break }
$Teams.Value.ForEach( {
   $TeamsHash.Add($_.Id, $_.DisplayName) } )
$NextLink = $Teams.'@Odata.NextLink'
While ($NextLink -ne $Null) {
   $Teams = Invoke-WebRequest -Method GET -Uri $NextLink -ContentType $ctype -Headers $headers | ConvertFrom-Json
   $Teams.Value.ForEach( {
      $TeamsHash.Add($_.Id, $_.DisplayName) } )
   $NextLink = $Teams.'@odata.NextLink' }

Get-Team Works Too

Another pragmatic solution to the problem of how to fetch all teams in a tenant is to use the Get-Team cmdlet. Using Get-Team is much slower than the code listed above (minutes instead of seconds), but the cmdlet handles the paging for you.

After fetching the set of teams, we can begin to process each team to discover what apps and tabs are installed in it. The Graph calls in the script to fetch channels, tabs, and apps don’t use paging. A team can have up to 200 channels, so I guess the call to fetch channels might need to change to handle such a well-endowed (and possibly confusing) team.

Still Work to Do

Although we can now fetch all the teams in a tenant, things aren’t perfect yet. I found a problem processing archived teams, where the attempt to retrieve app information fails. I can’t see how to identify an archived team from the information returned, so more research is needed. (Update: check the isArchived property in team settings to find archived teams).

Another issue is that the auth token expires after an hour and stops the script. A refresh token is needed at this point. (Update: the latest version of the Teams and Groups Report script illustrates how to renew an access token).

If you’d like to improve the code and make it even better, you can get the script from GitHub. In the meantime, enjoy using Microsoft Graph queries to process data with PowerShell.

Need help understand how to use PowerShell to manage Office 365 Groups and Teams? The Office 365 for IT Pros eBook contains a ton of examples to help you get going.

By Gustavo Velez (gustavo@gavd.net)

Leveraging the Power of the Graph

The Microsoft Graph API offers a fast and easy way to find and modify Office 365 data. Combining the Graph with Microsoft Flow makes it possible to automate the steps in a task. Some impressive results can be created even by people who don’t have a programming background.

The Microsoft Graph API is a unified programmatic RESTful interface for all the components of Office 365. Via a single REST endpoint. Microsoft Graph API provides access to objects such as users, groups, mail, messages, notes, tasks, calendar, and Office Graph. Data can be retrieved from multiple Microsoft cloud services such as Exchange, OneDrive, SharePoint, OneNote, Planner and Azure Active Directory. One of the Microsoft Graph components is the Security API, an interface and schema used to integrate security solutions from Microsoft and other vendors.

Office 365 Flow

Microsoft Flow is the automation engine in Office 365. A flow is a workflow that combines tasks and data from multiple Office 365 applications and services to get work done. To create a flow, the user specifies what action should take place when a specific event occurs. No programming experience is required.

Once a flow is built, it can be managed on the desktop or through a mobile device. Flow integrates out-of-the-box with many Office 365 services and apps, including SharePoint, Power BI, PowerApps, and Dynamics 365, but can be extended to reach other systems.

Putting Graph and Flow Together

After a tenant is up and running, it’s good to know the current state of the Office 365 services consumed by users. But monitoring services and taking any necessary actions can be a boring and tiresome activity. This kind of repetitive work can be automated using tools as Flow and Graph. And, because the creation of flows can be done by non-programmers, a flow can be quickly built by administrators.

In this article, we will use the Office 365 Secure Score as an example. Secure Score is a self-assessment tool to test the security level of an Office 365 tenant and generates an assessment of how well the tenant is protected against the risk of a security breach or attack. Secure Score is available in the Security & Compliance Center or this link. Click the Improvement actions tab to see recommendations to improve tenant security and increase the tenant’s Secure Score (Figure 1).

Our aim is to get the recommendations using the Microsoft Graph Security API. The idea is to generate a weekly report containing security recommendations and email it to the administrator.

First Step: Register an application in Azure Active Directory

Because Flow communicates with Graph as an “external” component, it needs to be authenticated and authorized to get information from Office 365. This is done by registering the application in the Azure AD and assigning the app the needed permissions. The first step is to open the Azure Active Directory portal and select “App registrations” and then “New registration” (Figure 2). Give the app a name, select the account type and leave the Redirect URL empty.

After the app is registered, note the “Application (client) ID” and “Directory (tenant) ID” values (Figure 3) because these data are used later in the process.

Next, grant the registration permissions to access the Graph, selecting the option “API Permissions” on the left side menu. Click on “Add permissions”, then “Microsoft Graph” and, in the new window, “Application permissions”. Because we will use the “Secure Scores” of the Security API, scroll down in the window, open the “Security” section, and select the option “SecurityEvents.Read.All” (Figure 4). Always the lowest possible access level for any app you register. Save the changes.

Remember to grant the permissions assigned to the application using the button in the “Grant consent” section (Figure 5).

The next step is to create a secret that allows the application to run without any user credentials. Select “Certificates and secrets” on the menu at the left side, and then click the button “New client secret.” A popup window will start asking for how long the secret will be valid: one, two years, or never expires. Select one of the options and generate the secret. Make a note of the secret along with the Client and Tenant IDs (Figure 6).

Creating the Flow

It is now time to create the Flow and get the information required. Open https://flow.microsoft.com and go to “My flows”. Select “+ New” and “Scheduled – from blank.. Assign a name to the Flow and select how often, and when, it will start (Figure 7).

The Flow designer starts. The “Recurrence” step should be already populated using the information in Figure 7. Use the “+ New step” button to create a “Initialize variable” step to contain the first value needed (the Client ID) to use the Azure AD registration, giving it a name, selecting “String” as “Type”, and copying its value into the Flow. Repeat the operation to add two variables for the Tenant ID and the Secret (Figure 8).

Add a new variable to contain the filter for the query to Graph. This is necessary because URLs in Flow don’t allow to use whitespaces, and Graph doesn’t understand the convention to use “_x020_” to encrypt the spaces. The value for the variable will be “?$filter=userImpact eq ‘High'” (Figure 9).

Now add a new step to the Flow, this time of type “HTTP” to send the request to Graph. This activity requires a “Premium” account for Flow (Plan 1 or Plan 2). In the step, click on the “Show advanced options”. In the “Authentication” box select “Active Directory oAuth”. Configure the options using the following values:

null
• Method: GET
• URI: We’ll explain this in a little while.
• Authentication: Active Directory OAuth
• Authority: https://login.microsoft.com
• Tenant: use the variable with the directory id
• Audience: https://graph.microsoft.com
• Client ID: use the viable with the application id
• Credential Type: Secret
• Secret: use the secret variable

For the URI you can use any of the multiple queries possible to approach the Graph Security API. In this example we are using this:

https://graph.microsoft.com/v1.0/security/secureScoreControlProfiles?$filter=userImpact eq ‘High’

Because the query is filtered to get a similar response as the response given by the Office 365 user interface (User Impact High), we are using a combination of the string for the query and the value of the variable for the filter (Figure 10).

Save the Flow and run it because we need the output for the next step. From the output result window of the Flow, copy the “Body” response (Figure 11).

The Graph response is in JSON form, and it contains a plethora of data. We are interested in only the Title (that contains the description of the issue) and Remediation fields. Return to the Flow and open it for editing. Add a new action of the type “Parse JSON” at the end of the flow to extract the needed data. Use the “Body” of the HTTP query output as “Content” and click on the “Use sample payload to generate schema.” Flow opens a pop-up window where you can paste the result of the query run (Figure 11). Flow will generate the schema for the parser and create objects for each item in the response automatically (Figure 12).

The parser presents the data as text without any formatting. To create something that a user can read, add a new step in the Flow of the type “Create HTTP table”. For the data source (“From” field) select the “value” tag of the parser and select “Custom” in “Columns”. Create two columns, one called “Title” that uses the “title” object created for the parser, and other called “Remediation” for the “remediation” object (Figure 13).

Save and run the Flow again. Open the result and in the Table step you will see the data that we want to have, rendered in the correct format (Figure 14).

To send a message to the system administrator with the query results, add finally a “Send an email notification” step to the Flow. Use the “Output” of the table for the Body of the Email (Figure 15).

Save the Flow and run it manually. An Email will be received in a couple of seconds with the recommendations found by Graph (Figure 16).

From now on, the Flow will run automatically each month based on the configured schedule.

Conclusions

The Microsoft Graph offers a unified way to approach and interact with Office 365. At the other side of the equation, Flow allows us to use Graph, not only to get information from Office 365 but also to take decisions and make modifications if necessary. Both systems combined give the administrators of Office 365 a powerful tool to automate monitoring and maintenance activities.

Gustavo writes Chapter 23 of the Office 365 for IT Pros eBook, where he discusses the finer points of Microsoft Flow and PowerApps. If you haven’t tried to create anything with Flow, you might be surprised at what’s possible.

The One True API

As noted in my post about using the Graph Explorer tool, the Microsoft Graph has become the de facto API for most if not all Office 365 workloads. Some legacy APIs persist, like Exchange Web Services (for more information about using EWS, see Glen Scales’ blog), but the wind of Microsoft investment and engineering staff is behind the Graph, so it should be your future focus for programmable access to Office 365 data.

The biggest advantage the Graph brings to the programming community is to deliver a standardized method of dealing with the myriad forms of data found inside Office 365 from email to documents to people to calendars and anything in between. Authentication is based on OAuth, so it follows standards too. Anyone who knows how to use a REST-based API will be comfortable with the Graph.

PowerShell and the Graph

A word about PowerShell is in order here. The advent of the Graph doesn’t mean that PowerShell is going to go away anytime soon. Although many parts of Office 365 are now built using the Graph, PowerShell is still used throughout the suite to get stuff done. More importantly, PowerShell is used to solve administrative problems by a very large community of Office 365 administrators. The basic division between PowerShell and the Graph is that PowerShell never goes near data inside user repositories. You can use PowerShell to fetch details about a user mailbox, but not the actual contents of the mailbox. In the past, that work would be done by EWS; now, it’s a task for the Graph.

So, if you’re looking for something to automate administrative operations for Exchange, Teams, or Azure Active Directory, PowerShell is probably the best choice. On the other hand, if you want to manipulate messages, meetings, or contacts, go for the Graph. In some cases, like Teams or Planner, you might not be able to do something with PowerShell and have to use the Graph, but that’s OK because you can call Graph operations from PowerShell (here’s an example of using PowerShell and the Graph to process Teams data). Choice is good, and it’s good to have a choice of tools (if only we were granted the insight to make the right choice each time).

A New Series

To help programmers understand the Graph, Microsoft is running a “30 days of the Microsoft Graph” series of blog posts. Each post is intended to take 5-15 minutes to read about a topic linked to the Graph. The first few introductory posts are now online and are worth reading if you haven’t heard about the Graph.

The Office 365 for IT Pros eBook isn’t about programming Office 365, so we don’t touch on the Graph (too much). On the other hand, because a lot of what the book covers is how to automate common administrative operations, we include a lot of PowerShell. Over 1,100 examples in fact.