Critical Need to Update Scripts Using PnP PowerShell Before September 9 2024

On August 21, 2024, the Pattern and Practices (PnP) team announced a major change for the PnP PowerShell module. To improve security by encouraging the use apps configured with only the permissions needed to process data within the tenant, the PnP PowerShell module is moving away from the multi-tenant Entra app (the PnP Management Shell, application identifier 31359c7f-bd7e-475c-86db-fdb8c937548e) used up to this point to require tenants to register a unique tenant-specific app for PnP.

Reading between the lines, the fear is that attackers will target the current PnP multi-tenant app and attempt to use it to compromise tenants. The multi-tenant app holds many Graph API permissions (Figure 1) together with a mixture of permissions for Entra ID, SharePoint Online, and the Office 365 service management API. Being able to gain control over such an app would be a rich prize for an attacker.

Swapping out one type of Entra app for another might sound innocuous, but it means that the sign-in command for PnP in every script must be updated. The PnP team will remove the current multi-tenant app on September 9, 2024, so any script that isn’t updated will promptly fail because it cannot authenticate. That’s quite a change.

The Usefulness of PnP PowerShell

I don’t use PnP PowerShell very often because I prefer to use Graph APIs or the Microsoft Graph PowerShell SDK whenever possible. However, sometimes PnP just works better or can perform a task that isn’t possible with the Graph. For instance, creating and populating Microsoft Lists is possible with the Graph, but it’s easier with PnP. SharePoint’s support for Graph APIs is weak and PnP is generally a better option for SharePoint Online automation, such as updating site property bags with custom properties (required to allow adaptive scopes to identify SharePoint Online sites). Finally, I use PnP to create files in SharePoint Online document libraries generated as the output from Azure Automation runbooks.

Creating a PnP Tenant Application

The first thing to do is to download the latest version of the PnP PowerShell module (which only runs on PowerShell 7) from the PowerShell Gallery. The maintainers update the module regularly. I used version 2.9.0 for this article.

The easiest way to create a tenant-specific application for PnP PowerShell is to run the Register-PnPEntraIDApp cmdlet:

Register-PnPEntraIDApp -ApplicationName "PnP PowerShell App" -Tenant office365itpros.onmicrosoft.com -Interactive

Make sure that you sign in with an account that has global administrator access. The cmdlet creates an Entra ID app and populates the app with some default properties, including a default set of Graph API permissions and a self-signed certificate for authentication. It doesn’t matter what name you give the app because authentication will use the unique application identifier (client id) Entra ID creates for the new app. The user who runs the cmdlet must be able to consent for the permissions requested for the app (Figure 2).

The Graph permissions allow read-write access to users, groups, and sites. Other permissions will be necessary to use PnP PowerShell with other workloads, such as Teams. Consent for these permissions is granted in the same way as for any other Entra ID app. Don’t rush to grant consent for other permissions until the need is evident and justified.

Using the Tenant App to Connect to PnP PowerShell

PnP PowerShell supports several ways to authenticate, including in Azure Automation runbooks. Most of the examples found on the internet show how to connect using the multi-tenant application. To make sure that scripts continue to work after September 9, every script that uses PnP PowerShell must be reviewed to ensure that its code works with the tenant-specific application. For instance, a simple interactive connection looks like this:

Connect-PnPOnline -Url https://office365itpros.sharepoint.com -ClientId cb5f363f-fbc0-46cb-bcfd-0933584a8c57 -Interactive

The value passed in the ClientId parameter is the application identifier for the PnP PowerShell application.

Azure Automation requires a little finesse. In many situations, it’s sufficient to use a managed identity. However, if a runbook needs to add content to a SharePoint site, like uploading a document, an account belonging to a site member must be used for authentication. This example uses credentials stored as a resource in the automation account executing the runbook.

$SiteURL = "https://office365itpros.sharepoint.com/sites/Office365Adoption"
# Insert the credential you want to use here... it should be the username and password for a site member
$SiteMemberCredential = Get-AutomationPSCredential -Name "ChannelMemberCredential"
$SiteMemberCredential
# Connect to the SharePoint Online site with PnP
$PnpConnection = Connect-PnPOnline $SiteURL -Credentials $SiteMemberCredential -ReturnConnection -ClientId cb5f363f-fbc0-46cb-bcfd-0933584a8c57

[array]$DocumentLibraries = Get-PnPList -Connection $PnpConnection | Where-Object {$_.BaseType -eq "DocumentLibrary"}
 
# Display the name, Default URL and Number of Items for each library
$DocumentLibraries | Select Title, DefaultViewURL, ItemCount

Ready, Steady, Go…

September 9 is not too far away, so the work to review, update, and test PnP PowerShell scripts needs to start very soon (if not yesterday). Announcing a change like this 19 days before it happens seems odd and isn’t in line with the general practice where Microsoft gives at least a month’s notice for a major change. I imagine that some folks coming back from their vacations have an unpleasant surprise lurking in their inboxes…

Understand Why Unlicensed OneDrive Accounts Exist

At the end of July, I reported Microsoft’s plan to charge for unlicensed OneDrive for Business accounts. The idea is simple. Ninety days after a OneDrive for Business account enters an unlicensed state, SharePoint Online will move the account into Microsoft 365 Archive. The tenant must then decide what to do with the accounts with the options being to manage the accounts or leave accounts to rot in the archive. Unlicensed accounts arise when an account no longer has access to a service plan for OneDrive (see the product names and service plans reference). Usually, an account enters the unlicensed state for OneDrive when an administrator deletes an account or removes a license like Office 365 E3 or E5 from the account.

Managing accounts requires the tenant to link Microsoft 365 Archive to an Azure subscription to pay for ongoing storage and restore operations. Storage costs $0.05 per month per gigabyte while retrieval costs $0.60 per gigabyte. Restored accounts remain accessible for 30 days. During this time, someone has to review the material in the account and move it to another repository, such as a different OneDrive for Business account or a SharePoint Online site. Once the 30-day period lapses, SharePoint Online archives the account again.

The OneDrive Report

So good, so far. Archiving old OneDrive accounts that clutter up storage is a good idea. It stops artificial intelligence tools like Copilot for Microsoft 365 using the content held in the obsolete accounts in its response to users and helps to better manage information belonging to ex-employees.

When Microsoft issued MC836942 on July 26, they said that by August 16, 2024, SharePoint administrators would be able to access a new report detailing unlicensed OneDrive for Business accounts. The OneDrive report should now be available through the Reports section of the SharePoint admin center in all tenants (Figure 1).

Note the warning that if accounts are left in Microsoft 365 Archive for more than 180 days after becoming unlicensed and the tenant does not take out an Azure subscription to pay for the Microsoft 365 Archive storage costs, SharePoint Online can delete the accounts. No documentation is currently available to cover this point, but it seems reasonable that Microsoft should remove old and unwanted OneDrive accounts if the owning tenant is unwilling to pay the storage costs to keep them in the archive.

Four Categories of Unlicensed OneDrive Accounts

Unlicensed OneDrive accounts fall into four categories:

• Retention period: The owning account is unlicensed but SharePoint Online has retained the OneDrive account because the retention period configured in the SharePoint admin center has not expired.
• Retention policy: A Microsoft 365 retention policy or retention labels prevent the deletion of an unlicensed OneDrive account. It is quite common for tenants to apply a blanket retention policy to all SharePoint Online sites and OneDrive accounts to retain information for multiple years. If this happens, the unlicensed OneDrive accounts cannot be removed until the retention period defined by the policy lapses.
• Active user with no license: The account that owns the OneDrive account is still active (is not deleted), but no longer has access to a service plan for OneDrive.
• Duplicate accounts: The account that owns the OneDrive account has several OneDrive accounts. This used to happen more often several years ago when account provisioning was not as good as it is now. I have not seen a duplicate account created in the recent past.

Figure 1 shows that my tenant has 34 unlicensed OneDrive accounts held by a retention policy. This is expected because I use a broad retention policy to govern removal of material from SharePoint Online and OneDrive for Business. Currently, you cannot see details of the accounts within each of the four categories on-screen. Instead, you must download the CSV file containing the details. In their documentation, Microsoft promises that an interactive UI will be available from January 2025, saying that “You can select a username to view the details.” Presumably, this means that the various sections in the on-screen report will expand to show usernames, and you can then expand a username to see its details, such as those available in the CSV file (Figure 2).

Time to Review Unlicensed OneDrive Account Information

Now that information about unlicensed OneDrive accounts is available in the SharePoint admin center, tenant administrators should check the report and review its content to determine if anything unexpected is present. I don’t imagine that anything strange will turn up, but you never know. Following the review, administrators might decide to adjust retention periods and policies to allow the removal of OneDrive accounts belonging to deleted Entra ID accounts or prepare for long-term storage in Microsoft 365 Archive.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across the Microsoft 365 ecosystem. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.

Controlling Stream Video Versions Designed to Consume Less Disk Storage

Microsoft 365 message center notification MC797116 (30 May 2024, Microsoft 365 Roadmap item 395380) addresses the question of storage consumption in SharePoint Online and OneDrive for Business for videos managed by Stream. The issue is simple. Any time a video owner updates the non-video content, Stream creates a brand-new version of the video that consumes the same amount of storage as the original.

Many reasons exist to change something for a video, like editing the metadata (title, description (Figure 1), or chapters), editing the transcript to correct flaws in the automatic text generated by the transcription bot, adding callouts through the interactivity feature, and so on.

Many Stream Video Versions

Behind the scenes, SharePoint Online or OneDrive for Business treat changes to non-video content in the same way as they handle changes made to Word documents or Excel spreadsheets and create new versions. The impact on storage is obvious if you look at the version history for a video. Figure 2 shows the version history for a 402 MB video that consumes 5,226 MB for the 13 versions stored by OneDrive.

Usually, this method of storing versions doesn’t affect OneDrive for Business accounts. Given that most videos are likely Teams meeting records, few videos are updated, and the version count remains small. In addition. The large storage quotas assigned to OneDrive for Business accounts accommodate a few extra versions without a problem.

The issue is more obvious in SharePoint Online where the tenant-wide storage quota comes under pressure from user demand for document storage, retention processing, and versioning. Buying additional SharePoint Online storage is expensive, and few tenants want to go down that route.

Microsoft announced intelligent versioning for SharePoint Online in July 2023, but according to Microsoft 365 roadmap item 145802, the rollout won’t happen until August 2024. Good things take time to get right.

The Change in the Creation of Stream Video Versions

The change Microsoft is introducing to Stream starting mid-July 2024 with the intention to complete worldwide deployment by late August 2024 is to stop generating new versions of videos for changes that do not affect video content. This is a reasonable approach, and it will prevent the kind of video version sprawl seen in the past (as obvious in Figure 2).

The downside is that metadata changes made to Stream videos are irrecoverable. If you restore a version of a video, you get the metadata available at that time. Any subsequent changes made to video metadata are ignored.

These actions no longer create a new version:

• Editing the title or description from within the Stream browser client.
• Adding or editing chapters, transcripts, captions, or interactivity (callouts or forms).
• Toggling media settings (show/hide about video, chapters, interactivity, comments, analytics, etc.).
• Adding audio tracks.

Any change that affects the video content, like trimming some seconds from the start or end of a video, will force Stream to generate a new version of the video. Once the change reaches your tenant, it goes into effect and cannot be reverted to the previous behavior. The change has no effect on existing videos and will not remove any versions that are already being stored. Microsoft says that if you want to remove extraneous versions, you’ll need to wait for SharePoint Intelligent versioning to appear in your tenant and use that to clean up unwanted video versions stored in SharePoint sites.

Storage is Not a Pressing Problem for OneDrive

At this point, I am unsure if the same approach can be taken to clean up video versions in OneDrive for Business accounts. However, given that storage is much less of an issue in OneDrive than it is in SharePoint Online, and that Teams meeting recordings age out over time, this is probably not a big problem. If you’re worried about OneDrive, run the OneDrive for Business account storage and quota report and see if any account needs attention. I bet hardly any will.

Make sure that you’re not surprised about changes that appear inside Microsoft 365 applications by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers stay informed.

Microsoft Will Remove-SPOExternalUser Between July 29 and August 9

Message center notification MC806103 (27 June 2024) reports the deprecation of the Remove-SPOExternalUser cmdlet from the SharePoint Online management PowerShell module. Microsoft suggests that administrators replace the cmdlet with the Remove-AzureADUser cmdlet, which is a perfectly reasonable strategy if only the cmdlet isn’t part of the retired and soon-to-be deprecated AzureAD module.

Between July 29. 2024 and August 9, 2024, Microsoft will disable the Remove-SPOExternalUser cmdlet. When the block arrives in a tenant, attempts to run the cmdlet will be greeted with:

“To streamline scope and permissions for external users, enhance access management, and strengthen our security posture, this cmdlet has been deprecated. Alternatively, please use the Remove-AzureADUser cmdlet in Microsoft Entra ID for user management.”

Microsoft 365 is so Large that No One Understands Everything

MC806103 is a classic example of Microsoft being such a large organization that no one knows what’s happening across the board, or even what’s happening within Microsoft 365. In this case, the SharePoint Online people want to deprecate the Remove-SPOExternalUser cmdlet. That’s a good idea because the cmdlet has low usage (I don’t think I have ever used it) and doesn’t really make sense inside the Microsoft 365 ecosystem where external access for applications like SharePoint Online is now governed using guest accounts. It makes perfect sense to remove overlapping or conflicting features and replace them with what you’d consider a component that’s closer to the core.

Entra ID is the directory of record for Microsoft 365. Individual workloads like SharePoint Online have their own directory, but everything flows back to Entra ID. Replacing the SharePoint Online cmdlet with an Entra ID cmdlet is the right thing to do. The problem is that the program manager in charge of making the transition obviously doesn’t know that the Entra ID team has been trying to deprecate the AzureAD and AzureADPreview modules since 2020. For the last few years, Microsoft has conducted an ongoing campaign to move tenants off these modules to use the Microsoft Graph PowerShell SDK.

What makes this laughable is that Microsoft launched the Entra PowerShell module in preview on June 27 in the hope that a dedicated Entra module (built on top of the Microsoft Graph PowerShell SDK) would help the remaining customers who have scripts that use the AzureAD and AzureADPreview modules to move to a modern platform. Obviously, whoever wrote MC806103 had no idea that this development was in train.

The Right Way to Replace Remove-ExternalSPOUser

The Get-SPOExternalUser cmdlet reports the external users registered for a SharePoint Online tenant. The last time I discussed its use, I observed that the Get-SPOExternalUser cmdlet is an odd cmdlet in some ways, but it does generate a list of external users from the SharePoint directory.

An external user record looks like:

RunspaceId    : 9630573b-c675-4697-a029-72d535e48613
Email         : charu.someone@microsoft.com
DisplayName   : Charu Someone
UniqueId      : 100320009C9C6789
AcceptedAs    : charsomeone@microsoft.com
WhenCreated   : 20/02/2020 19:45:02
InvitedBy     :
LoginName     :
IsCrossTenant : False

Remove-SPOExternalUser works like this:

Remove-SPOExternalUser -UniqueIDs ($User).UniqueId -Confirm:$false
Successfully removed the following external users
100320009C9C6789

The cmdlet removes the external user entry from SharePoint Online. It also removes the matching guest account, if one exists, from Entra ID. In my tenant there are quite a few lingering external accounts that don’t have matching Entra ID guest accounts. These accounts are just another form of digital debris that needs to be cleaned up.

The right way to remove an external account is to use the Remove-MgUser cmdlet from the Microsoft Graph PowerShell SDK:

$User = Get-MgUser -filter "mail eq 'andrew@proton.me"
Remove-MgUser -UserId $User.Id

Or, if you decide to use the preview Entra module:

$User = Get-EntraUser -SearchString 'AdamP@contoso.com'
Remove-EntraUser -ObjectId $User.Id

Either cmdlet has a much longer future ahead of it than the Remove-AzureADUser cmdlet has. In both cases, SharePoint Online synchronizes with Entra ID and removes the matching external user record.

It’s Just Hard to Keep Up

I don’t blame the individual program manager responsible for MC806103. It’s hard to keep up with everything that goes on within Microsoft 365 and all too easy to assume that a solution that works (for now) is the right long-term recommendation. Perhaps Microsoft needs a clearing house to cross-check dependencies outside the control of an individual development group before they publish information to customers?

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across the Microsoft 365 ecosystem. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.

Support for Sharing Links Expiration Added for Company-wide and Specific People Links

The change announced in message center notification MC799277 (6 June 2024) to make expiration dates available for all types of sharing links should now have reached targeted release tenants. General availability will follow soon afterwards.

Until now, SharePoint Online and OneDrive for Business have supported expiration dates for anyone sharing links. Microsoft was correct to start with these links because they are transferable. In other words, anyone in possession of the link can access the file or folder pointed to by the link.

Although anyone links are revocable and therefore can be annulled if a link becomes too widely available, making them the first sharing link to support expiration was a good thing. Even with expiration dates, many organizations prohibit anyone links because they consider these links to be too dangerous. Users forget to set expiration dates, the links circulate in email and can easily escape outside the organization, and so on.

Company-wide and Specific People Sharing Links

Company-wide (aka people in your organization) and specific people links deliver tighter control over sharing because SharePoint Online validates the account that attempts to redeem and use a link to make sure that they meet the sharing criteria. For example, if you’re not signed into a tenant account, SharePoint Online won’t allow you to use a company-wide link.

Specific people links are usable with people inside and outside an organization. External people must have a guest account in the tenant to authenticate, either an account created to access other resources like Teams (or most recently, Microsoft Loop), or an account created during the process of gaining access to the shared content. During this process, depending on the conditional access policies active in the tenant, an external person might be asked to configure multi-factor authentication to protect their account.

All of this sounds good, and it means that specific people links are usually a safe way to share externally, especially if coupled with a sensitivity label with encryption to stop any inadvertent leakage of confidential information.

Sharing Links Expiration for All

Using a sharing link that SharePoint Online imposes controls over who can use the link can sometimes do with a little extra help and that’s where the expiration controls come in. You can now set a date (Figure 1) for company-wide and specific people sharing links to expire.

When a link expires, it can no longer be used to access the shared content. The owner of the content must then reshare the content if they wish.

It seems like Microsoft has some loose ends to clean up before you could consider this feature to be complete. For instance, although SharePoint Online shows the expiration date after copying a link (Figure 2), the Manage access dialog for an item doesn’t display expiration dates. This might be due to an incomplete software deployment and the missing bits for an updated Manage access dialog might be still on the way.

More importantly, the SharePoint Online admin center has a setting for Anyone links to set a maximum expiration length in days (Figure 3). However, similar controls aren’t yet available for company-wide and specific people links. That seems like an oversight.

Being picky, I could also point out that setting an expiration period for a sharing link does not affect the SharingSet audit record generated when SharePoint Online or OneDrive for Business configure a sharing link. This is a pity. Microsoft needs to improve the information captured in audit records for sharing events to make them more administrator friendly. For instance, a value like “EventData    : <PermissionsGranted>Contribute</PermissionsGranted><MembersCanShareApplied>False</MembersCanShareApplied>” is meaningful to a computer but less so to a human. If you’re interested in learning how to interpret audit records for sharing events, try this script from GitHub.

Sharing Links Expiration for All Link Types is a Good Change

Even though I think Microsoft has some things to work on to complete the feature, I like that SharePoint Online supports expiration dates for all types of sharing links. It’s a good change and one that should be popular with users, even if administrators can’t find out the kind of usage the feature gets because of the lack of detail in audit records. According to MC799277, Microsoft is due to refresh the documentation, but that hasn’t happened yet. More bits lost in transit!

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Why Some People Can’t Use Shared Files with Copilot for Microsoft 365

After reading the article about the new sensitivity label advanced setting to block access for Microsoft content services to confidential Office documents, a reader asked why some users can use some documents shared using company-wide links with Copilot for Microsoft 365 while others cannot. The situation seemed a little strange because it happened for documents shared with everyone in the organization. The problem couldn’t be due to a sensitivity label because the capability only just rolled out and is limited to the Office applications.

The answer is in Microsoft’s documentation for secure file sharing, which says: “Creating a People in your organization link will not make the associated file or folder appear in search results, be accessible via Copilot, or grant access to everyone within the organization. Simply creating this link does not provide organizational-wide access to the content. For individuals to access the file or folder, they must possess the link and it needs to be activated through redemption.”

In other words, sharing a file with everyone in your organization is only the first step in the process of making information available to Copilot for Microsoft 365. A company sharing link that arrives in your inbox or is shared through a Teams chat is dormant until you redeem it by using the link. At that time, SharePoint Online checks that your account belongs to the organization to conform your access to the file. If confirmed, the file joins the set of “shared with you” information, which makes it available to Copilot for Microsoft 365.

Testing Company-wide Sharing Links with Copilot

A simple test proves the point. Create a file that contains some information that’s unlikely to exist elsewhere within the company. In my case, I created a Word document about a fictional digital SLR camera called the Bunsen BX7. Now share the file with a company-wide link (Figure 1).

After signing into another account, open Copilot for Microsoft 365 chat and attempt to find some information about the topic in the file. Copilot should return nothing because a Bing search of the internet and a Microsoft search of company resources available to the account turn up no mention of the topic. But if you now go and use the link to open the file, Copilot can find the information and use it in its responses.

Figure 2 shows a Copilot for Microsoft 365 chat session. The first prompt about the Bunsen BX7 turns up nothing and Copilot responds with some generic text about digital cameras. The second prompt is after redemption of the company-wide sharing link. Copilot is able to find the document and use the information in its response. You can see that the shared document is listed as a source for the response.

The Desirability of Company-wide Links

The mystery of why some people can use shared documents with Copilot for Microsoft 365 is solved, but thoughts now turn to whether organizations should restrict the use of company-wide links for sensitive documents. The value of these links is that they allow anyone in the organization to access content. The downside is that it’s too easy to create and use company-wide links, which then creates the temptation for people to use these links to share confidential files wider than the organization wants the information to be known.

To guide users away from company-wide links to create sharing links for specific people instead, you can modify the SharePoint tenant configuration to make direct links the default option. Even better you can update individual site settings to disable company-wide links (anyone links are also disabled). For example, the first command sets direct links as the tenant default; the second disables company-wide links for a specific site.

Set-SPOTenant -DefaultSharingLinkType Direct

$Site = "https://office365itpros.sharepoint.com/sites/BlogsAndProjects"
Set-SPOSite -Identity $Site -DisableCompanyWideSharingLinks Disabled

If your organization uses sensitivity labels, you could also consider applying a label that restricts access to a small group of users. That way, even if someone sends a document outside the organization as an email attachment, external recipients won’t be able to open it.

The Challenge of Managing Information in an AI World

The advent of AI assistants creates new information governance challenges for Microsoft 365 tenants. Slowly but surely mechanisms are being developed to help organizations cope and manage the potential for information leakage and misuse. Some Microsoft solutions are no more than sticking plasters to allow customers to progress their Copilot deployments, but overall, the situation seems to be improving. Let’s hope that the trend continues and the current AI hype lives up to its promise.

An Overlooked Feature to Block Downloads for Channel Meeting Recordings

Given the size of Microsoft 365 and the fact that you might not use some app features very often, a good likelihood exists that you’ve never heard of some features. Or you might have forgotten that a feature exists. I experienced this feeling during the review of the Managing Videos chapter for the Office 365 for IT Pros eBook (2025 edition) when Vasil Michev, our technical editor, pointed out that the chapter didn’t mention the ChannelRecordingDownload setting for the Teams meeting policy.

This oversight on our part is inexcusable. The setting has existed for quite a while. It’s mentioned in message center notification MC222640, updated August 2, 2021, and our sole excuse is that this update occurred with a bunch of other changes intended to facilitate the migration of Stream classic to Stream on SharePoint. We’ll address the deficit here.

Updating Meeting Policies to Block Download Access for Channel Meeting Recordings

By default, the value of the ChannelRecordingDownload setting in Teams meeting policies is “Allow.” This means that the SharePoint app (app@sharepoint) saves recordings of channel meetings in the Recordings folder for the channel. For instance, recordings for meetings hosted in the General channel end up in General\Recordings while recordings for meeting in a shared channel end up in the Recordings folder. Private channels don’t have a mailbox and don’t support channel meetings. All channel members have access to the MP4 files created for recordings, including the ability to download the files.

Updating the ChannelRecordingDownload setting to block access to channel meeting recordings is not supported by the Teams admin center Instead, use the Set-CsTeamsMeetingPolicy cmdlet from the Teams PowerShell module to update Teams meeting policies. For example:

Set-CsTeamsMeetingPolicy -Identity "Allow Meeting Recording" –ChannelRecordingDownload Block

You might also need to run the Grant-CsTeamsMeetingPolicy cmdlet to assign the updated policy to the set of accounts that you want to block from downloading.

Grant-CsTeamsMeetingPolicy -PolicyName "Allow Meeting Recording" -Identity Ken.Bowers@office365itpros.com

Where Channel Meeting Recordings Go

After updating the meeting policy, SharePoint applies different rules to the storage of and access to channel meeting recordings. Remember that a channel meeting doesn’t really have an owner or organizer. The person who creates a channel meeting acts as the organizer in that they manage invitations and other meeting settings. However, they do not own the recording in the same way as they would for a personal meeting.

When saving channel meeting recording in SharePoint Online, the SharePoint app checks the meeting policy assigned to the organizer and finds that the policy blocks downloads for channel meeting recordings. Instead of saving the MP4 file in the Recordings folder, it saves the file in the Recordings\ViewOnly folder and alters permissions on the file so that only channel owners can download the recording. Everyone else, including the meeting organizer (unless they are also a channel owner) are limited to online view-only access (Figure 1).

The Microsoft documentation offers sparse information on this point and a quick internet search didn’t turn up much else. I guess everyone else missed this change.

The Can View Only Permission

In other related news, an associated change described in message center notification MC699712 (19 December 2023) might also have escaped your attention. This is the update for the share link settings for Stream videos stored in OneDrive for Business and SharePoint Online.

The change introduced the “Can view, but not download” option to the permission drop down (Figure 2) to make it easier for people to share videos that they didn’t want downloaded. It was possible to block downloads for video files beforehand, but it required additional steps to block downloads.

channel member creates a sharing link for a video file in the \ViewOnly folder, they’re restricted to sending a sharing request to the channel owners to request access. This occurs even when sharing with other channel members, even though those channel members already have access.

The Joy of Finding an Overlooked Setting

I’m not particularly worried about the downloading of my videos that other users have access to through Stream. However, I can understand that this might be a concern for others, and it’s good that Microsoft 365 includes controls to limit access in a reasonably painless manner, even if I have totally overlooked the settings for years.

Learn about using Stream on SharePoint and the rest of Office 365 by subscribing to the Office 365 for IT Pros eBook. Use our experience to understand what’s important and how best to protect your tenant.

Support for Office 365 Connectors Ceasing for Microsoft 365 Groups and SharePoint Online

Message center notification MC798683 (4 June 2024) announces the retirement of Microsoft 365 Groups connectors, a form of what are called Office 365 connectors. The retirement process commences on August 5, 2024, and finishes on September 5, 2024. After that time, connectors will no longer be supported within Outlook (Win32), OWA, and the new Outlook for Windows (aka Monarch).

Connectors take notifications from online data sources and post messages into a target destination. In this case, the target is the Inbox in the mailbox of the Microsoft 365 group configured with the connector. These connectors are used with Outlook groups rather than Teams. You can’t configure a connector for the other folders in a group mailbox, and you can’t configure a connector for any other type of mailbox.

Messages delivered through an Office 365 connector are limited to 28 KB and aren’t intended to be complete articles. Instead, they let users know that something has happened, give them a short snippet about the event, and provide a link to follow for more complete information. Using a connector to post messages from an RSS feed is one of the most common uses, but third-party companies like Asana and Trello have created connectors to bring snippets about information from their services to Outlook and other Microsoft 365 targets.

Microsoft recommends that organizations replace group connectors with the Power Automate app, which has its own set of connectors for different data sources, including the ability to create a cloud flow to post messages to the group mailbox. Some of the Power Automate Connectors (like Salesforce and Jira) require a Power Automate premium license.

Connectors and SharePoint Online

A further blow for Office 365 Connectors comes in message center notification MC793656 (16 May 2024), which announces the retirement of connectors from SharePoint Online webparts. Microsoft says that this is due to “limited usage.” Based on anecdotal evidence and personal experience, I can’t recall ever seeing an Office 365 connector configured with a SharePoint Online webpart.

In any case, from June 15, 2024, site owners are unable to add connectors to SharePoint Online. On August 1, 2024, they’ll be unable to update or manage existing connectors and the connectors will stop receiving inbound notifications.

Teams, Office 365 Connectors, and Workflows

Teams still supports Office 365 connectors, which are configured on a per-channel basis because the target for new notifications are channel conversations. Each notification creates a new conversation.

MC798683 points out that Teams channels also support workflows created using the workflows app (“powered by” Power Automate), and workflows recently turned up in the […] menu for Teams chats (MC683929, last updated 24 May 2024).

I shall have to pay more attention to workflows in the future. I know that the basic stuff works very well (like bringing an RSS feed into a channel). I’m more interested in finding out how to replace the incoming webhook connector, which is used in many ways to bring information from applications into Teams.

So far, my experiments with the Post to a channel when a webhook request is received workflow have not been successful. This seems to work in the same way (publish a URL to post messages to) and it’s easy to find the URL, but more difficult to get the workflow to run. I eventually managed and published my experience about posting an adaptive card to Teams.

Moving to a Single Answer for No-Code Automation

All of this seems to be part of a cunning plan to turn Microsoft 365 users into citizen developers by popularizing the use of Power Automate and the Microsoft Power Platform (Figure 1) for no-code automation wherever possible. According to Microsoft (January 2024), Power Automate has 33 million monthly active users in 350,000 organizations. My assumption is that PowerShell and the Graph are the answer for code-based automation.

It’s hard to argue against rationalization and it does make sense to settle on a single no-code automation platform for Microsoft 365, something that wasn’t viable when Office 365 Connectors appeared around 2016. As always, don’t be surprised when change happens inside Microsoft 365. Just be prepared to cope with the change.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across the Microsoft 365 ecosystem. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.

SharePoint Online Storage, OneDrive for Business, and SharePoint Embedded

Given the vast numbers of files created in SharePoint Online daily (Jeff Teper cited 2.3 billion in December 2023), it must be the case that the storage quotas assigned to tenants are being consumed at an alarming rate. However, I suspect that a large proportion of the files end up in OneDrive for Business and don’t impact storage so much.

These thoughts came to mind when I perused the OneDrive files report for my account to discover just how many applications now store their data in OneDrive for Business. Microsoft has truly made OneDrive for Business the personal storage system for Microsoft 365 holding anything from Office documents to Teams meeting recordings and transcripts to Whiteboards.

But coming back to storage, I often hear confusion in how Microsoft charges for SharePoint storage. Let’s review the current situation.

Three Major Storage Partitions

SharePoint Online covers three major storage partitions:

• SharePoint Online sites.
• SharePoint Embedded applications, like Loop and Designer.
• OneDrive for Business accounts.

The SharePoint Online storage quota assigned to a tenant (1 TB plus 10 GB per licensed user) covers only the first category. The storage consumed by SharePoint sites is well understood because it’s highlighted in the SharePoint admin center and is easy to report with PowerShell. A Graph usage API is also available for SharePoint Online, but currently suffers from a longstanding data issue that prevents site URLs from being shown.

Understanding the storage consumption of SharePoint Embedded applications is less clear. These applications use file storage containers (like document libraries). First-party applications like Loop charge their storage against the tenant storage quota for SharePoint Online. If the applications support SharePoint Online PowerShell or another API to report storage, it’s possible to generate a report about the storage consumed by an app.

Third-party apps built on top of SharePoint Embedded are billed separately through an Azure subscription using a pay-as-you-go metered model. Charges are accrued for API calls and the storage consumed.

OneDrive for Business Storage

The OneDrive service description says that “the default storage space for each user’s OneDrive is 1 TB. Depending on your plan and the number of licensed users, you can increase this storage up to 5 TB.” The default storage assigned to OneDrive for Business accounts is defined through the Settings section in the SharePoint Online admin center (Figure 1).

In a Microsoft 365 enterprise tenant, the storage for OneDrive can be increased to more than 5 TB. The documentation states: “Before requesting an increase you need at least five licenses that include OneDrive Plan 2, you must assign at least one license to a user, and a single user must have already filled 90% of their 5 TB storage.”

The problem here is that Microsoft stopped offering OneDrive Plan 2 in August 2023, apparently to stop offering the “unlimited storage capacity” that was once available for licenses like Office 365 E3 and E5. No official notice was given, and the plan slipped away. Office 365 and Microsoft 365 licenses no longer include a OneDrive service plan.

In any case, if you want to keep an eye on OneDrive storage consumption, it’s easy to generate a report with PowerShell.

Microsoft 365 Archive

Microsoft 365 Archive is a solution that moves SharePoint Online sites from “hot” storage (immediate access) to “cold” storage. The idea is that organizations can keep data online in a form that’s available for eDiscovery but not for user access. Archiving sites also helps to remove information from consumption by AI solutions like Copilot for Microsoft 365 to stop the results generated by AI being affected by old and possibly obsolete information.

Organizations pay for the storage consumed by archived sites through an Azure subscription. The cost per GB is much less than having to pay for regular SharePoint storage and Microsoft doesn’t charge for archive storage if the tenant has regular storage available. If the tenant runs out of regular storage, Microsoft 365 archive switches to its pay-as-you-go model.

Retention Storage

Microsoft 365 Retention Policies and Retention Labels can dictate how long content stored in SharePoint Online (including OneDrive for Business and SharePoint Embedded) is kept before it can be deleted. If files coming within the scope of retention are deleted by users, SharePoint Online keeps them in the site’s preservation hold library. Depending on the settings of retention policies and labels, it’s possible that preservation hold libraries can consume a large amount of storage (Figure 2).

Retained content can be easy to overlook. Microsoft plans to introduce intelligent versioning (originally planned for November 2023), which should help.

Summarizing SharePoint Online Storage

In summary, traditional SharePoint site storage is only one of the ways that tenant storage quota can be consumed. OneDrive for Business stores more data than ever before, but Microsoft has renounced unlimited storage. New applications and retention can consume storage unexpectedly, and Microsoft 365 Archive can help by moving data to cheaper cold storage. What could be easier to understand?

Support the work of the Office 365 for IT Pros team by subscribing to the Office 365 for IT Pros eBook. Your support pays for the time we need to track, analyze, and document the changing world of Microsoft 365 and Office 365.

Folder Deletion with Items in Place Makes it Easier to Clean Out Old Material

In the past, SharePoint Online used to block deletion of files with retention labels. In late 2021, Microsoft decided to make the deletion behavior consistent across SharePoint Online and OneDrive for Business by allowing deletions to occur. Files with retention labels went into the site recycle bin and progressed into the preservation hold library until their retention period expired. After that point, a timer job finds and removes the expired files.

Another welcome change to SharePoint deletion behavior is now rolling out (MC791878, 11 May 2024, Microsoft 365 roadmap item 394689) and should be fully deployed worldwide around this time. The change allows users to delete folders in document libraries that aren’t empty in sites covered by a Purview retention policy.

Removing Old Material with Folder Deletion

This doesn’t sound important, but being able to delete folders without having first to open the folder and remove all the files stored there is the way things should have worked all along. A case can be argued that allowing people to delete folders without checking what’s stored in the folder could lead to inadvertent removal of information.

However, the case is undermined by the fact that the deleted folder (and its items) goes into the recycle bin from where it can be recovered. Even if the deleted folder passes through the normal SharePoint Online recycle bin cycle, administrators can still rescue the files from the site preservation hold library. When a deleted file is restored from the recycle bin, SharePoint Online recreates the folder in the original location if necessary.

The only problem I met testing deletions is when attempting to delete an empty folder and a non-empty folder together. For some bizarre reason, SharePoint Online used the old behavior and refused to remove the non-empty folder (Figure 1). SharePoint Online was quite happy to remove the same folder if processed individually.

Very importantly, after deleting a non-empty folder, OneDrive for Business will not attempt to synchronize the deleted folder back from its offline copy.

Checking Retention Status for SharePoint Sites

If a tenant uses multiple retention policies, it can be challenging to determine which policy governs an individual site or mailbox. To help, the Data lifecycle management section of the Purview compliance portal includes a policy lookup option. At first glance, the list of retention policies shown in Figure 2 seems overwhelming, but several different types of policy are present, including some to publish retention labels to the site and auto-label policies that use trainable classifiers to label files with certain characteristics.

Because multiple policies can have a specific site within their scope, it’s important to note the purpose of each policy in the description.

Why is it Important to have easier Folder Deletion?

Some people never delete any material from SharePoint Online. At least, they don’t until they’re forced to because the tenant storage quota is nearly exceeded, and they want to avoid purchasing some expensive additional storage. The problem here is that deleting non-empty folders in sites governed by retention policies won’t help with a storage quota issue because files retained in the preservation hold library count against the quota. In some cases, the preservation hold library can occupy 40% or more of the storage used by a site.

It’s wise to keep an eye on the storage consumed by sites and then investigate the sites where storage consumption seems excessive. I use a Graph-based PowerShell script to generate a report of individual files in a document library to help understand where storage is eaten up. Obviously, after identifying unwanted files and folders, being able to remove those files more easily is a good thing.

Another reason why it’s good to clean up document libraries is that it stops Artificial Intelligence tools like Copilot for Microsoft 365 using old, obsolete, and potentially inaccurate information. Removing digital debris is something I think we’re all going to become more serious about as the AI era unfolds.

Keep up to date with developments like the AI era for Microsoft 365 by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers understand the most important changes happening across Office 365.

SharePoint Online is a Huge Success But Dark Clouds Lurk Ahead

March 27 marked the 23^rd anniversary of SharePoint Portal Server 2001, the forerunner of what we have today in SharePoint Server and SharePoint Online. The date in SharePoint history was marked by several tweets, including one from the urbane Mark Kashman, the well-known SharePoint marketeer. The tweet included an updated timeline for SharePoint (Figure 1), refreshed from an original version issued to celebrate the product’s 20^th anniversary.

I debate the accuracy of some of the dates listed in the SharePoint history. For instance, Delve and the original Office 365 Video solution are listed for 1 January 2024. My recollection is that these solutions were revealed at the first Ignite conference in May 2015 as part of the “next generation knowledge” portals promised by Microsoft at the time. As we know, marketing promises don’t always transfer into actual technology at the predicted date. Delve and Office 365 Video arrived, but the next generation knowledge portals never did. There’s also no mention of Office 365 Groups (now Microsoft 365 Groups), something that has had a huge impact on SharePoint Online.

Personal SharePoint History

Although I am probably better associated with Exchange, I have a long history with SharePoint going back to Portal Server 2001, which I had deployed at Compaq soon after its release in a nascent attempt to persuade technologists to share their knowledge with their peers. I even helped Microsoft Latin America launch SharePoint Portal Server 2001 at an event in Cancun.

SharePoint Portal Server 2001 worked well at a certain level and I took it forward into HP after the HP-Compaq merger in 2002 where it displaced a large UNIX cluster that HP Services used for document management.

As SharePoint Server developed I became exasperated at the development group’s attempts to build what seemed to be everything into a single server instead of focusing on document management. I thought that SharePoint Server 2007 was a mess and expressed that view quite strongly, something that didn’t make me many friends in Microsoft. The 2010 and 2013 release weren’t much better. The zenith of incompatability within the Office server lineup was reached when Microsoft tried to make Exchange and SharePoint work together in the ill-fated site mailbox project. Only 53 operations had to be carried out with absolute precision to make the two servers co-operate.

The Cloud Made the Difference

SharePoint achieved its full potential in the cloud. Administrators were freed from the task of looking after server farms and could concentrate on leveraging the product’s strengths in document management.

The introduction of Teams in 2017 helped enormously by providing a more user-friendly face for document storage. The growth in Teams usage to 320 million monthly active users propelled SharePoint Online usage into the stratosphere to a point where petabytes of data are added monthly.

The introduction of SharePoint Embedded as a platform for developers to build on is an interesting evolution to encourage even further usage. The Loop app is a good example of an app that uses SharePoint Embedded for storage with a UI that has no connection to what people might think of as traditional SharePoint.

Dark Clouds on the Horizon

Everything seems to be on the up in the SharePoint world, but I see some clouds on the horizon. The fact that Microsoft has been forced to introduce Restricted SharePoint Search to allow customers to progress Copilot for Microsoft 365 projects is an admission of failure in information governance.

Restricting users to searching 100 curated sites might seem like a good answer, but it admits that the tens of thousands of sites created by Teams are an unmanageable tangle. Inside those sites obsolete, misleading, and erroneous information might lurk in documents ready to corrupt the results generated by Copilot. It’s perhaps the greatest challenge faced by those considering Copilot deployments.

Digital debris is a big black cloud over SharePoint. Copilot is an accelerant that highlights the issue, but Microsoft 365 customers without Copilot should also focus on gaining control over the information held in SharePoint. This a wake-up call for tenants to ask questions about how they control the creation of sites (with or without Teams), how documents are stored and managed, how they use retention policies to remove old information, and so on. The issue won’t go away. It grows worse every day as users add petabytes of documents to SharePoint Online and OneDrive for Business.

The Microsoft 365 conference takes place in a month’s time. I’m sure that the SharePoint community will applaud the achievements and popularity of the platform. I hope that they take some time to address the information governance issue and that the current threat to continued success in SharePoint history abates.

Keep up to date with developments in SharePoint Online by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers understand the most important changes happening across Office 365.

No Evidence that Microsoft Cares as Pnp.PowerShell Fills the Gap

I last wrote about the state of SharePoint Online PowerShelll in 2020. At the time, I focused on Microsoft’s PowerShell module (Microsoft.Online.SharePoint.PowerShell), which is downloadable from the PowerShell Gallery. Based on the gallery statistics, the module is popular as each version attracts hundreds of thousands of downloads. Microsoft also updates the module monthly. On the surface, everything seems wonderful, and the module is in rude health.

If only this was true, but it’s not. It’s true that Microsoft updates the module to add tenant settings to control new features as they appear (like request files), but there doesn’t seem to be a coordinated plan about how Microsoft plans to support management of SharePoint Online through PowerShell.

Lack of Progress with Graph API

In 2022, Microsoft released the initial (beta) version of a Graph API to access and update SharePoint tenant settings. Apart from supporting the SharePoint settings API through the production (V1.0) endpoint, Microsoft doesn’t seem to have made much progress with the API since 2020. At least, the same set of tenant settings are visible two years on.

On the upside, SharePoint Online tenant settings are accessible using the Microsoft Graph PowerShell SDK. For instance, the Get-MgAdminSharepointSetting cmdlet reports the supported settings:

Connect-MgGraph -NoWelcome -Scopes SharePointTenantSettings.Read.All

Get-MgAdminSharepointSetting | Format-List

AllowedDomainGuidsForSyncApp                    : {}
AvailableManagedPathsForSiteCreation            : {/sites/, /teams/}
DeletedUserPersonalSiteRetentionPeriodInDays    : 60
ExcludedFileExtensionsForSyncApp                : {*.rar, *.zip}
Id                                              :
IdleSessionSignOut                              : Microsoft.Graph.PowerShell.Models.MicrosoftGraphIdleSessionSignOut
ImageTaggingOption                              : enhanced
IsCommentingOnSitePagesEnabled                  : True
IsFileActivityNotificationEnabled               : True
IsLegacyAuthProtocolsEnabled                    : True
IsLoopEnabled                                   : True
IsMacSyncAppEnabled                             : True
IsRequireAcceptingUserToMatchInvitedUserEnabled : True
IsResharingByExternalUsersEnabled               : False
IsSharePointMobileNotificationEnabled           : True
IsSharePointNewsfeedEnabled                     : False
IsSiteCreationEnabled                           : True
IsSiteCreationUiEnabled                         : True
IsSitePagesCreationEnabled                      : True
IsSitesStorageLimitAutomatic                    : True
IsSyncButtonHiddenOnPersonalSite                : False
IsUnmanagedSyncAppForTenantRestricted           : False
PersonalSiteDefaultStorageLimitInMb             : 5242880
SharingAllowedDomainList                        : {Microsoft.com…}
SharingBlockedDomainList                        : {Gmail.com}
SharingCapability                               : externalUserAndGuestSharing
SharingDomainRestrictionMode                    : none
SiteCreationDefaultManagedPath                  : /sites/
SiteCreationDefaultStorageLimitInMb             : 26214400
TenantDefaultTimezone                           : (UTC) Dublin, Edinburgh, Lisbon, London
AdditionalProperties                            : {[@odata.context, https://graph.microsoft.com/v1.0/$metadata#admin/sharepoint/settings/$entity]}

And the Update-MgAdminSharepointSetting cmdlet updates a setting:

$Body = @{}
$Body.Add("IsResharingByExternalUsersEnabled",$true)
Update-MgAdminSharepointSetting -BodyParameter $Body

SharePoint Online PowerShell is Windows PowerShell

Getting back to the PowerShell module, Microsoft has not updated it to support PowerShell 7. This might not be a problem if you always use Windows, but it does limit platform coverage. Attempting to load and use the module with PowerShell 7 usually fails, especially when multifactor authentication is involved.

The Community Approach to SharePoint Online PowerShell

This brings me to the Pnp.PowerShell module, also available from the PowerShell gallery. Based on the download numbers, Pnp.PowerShell seems to be four to five times more popular than the official Microsoft SharePoint Online module. This state is probably due to:

• Development driven by a committed set of community advocates.
• Wider coverage of SharePoint commands. The module spans over 650 cmdlets while the Microsoft.Online.SharePoint.PowerShell module has 250. Part of the reason for the dramatic difference in cmdlet count is that Pnp.PowerShell dips into other Microsoft 365 workloads associated with SharePoint Online like Teams, Planner, Flow, and Entra ID. Another is that Pnp.PowerShell includes cmdlets to create objects like files in SharePoint Online document libraries (here’s an example) that aren’t within the scope of the administrator-centric SharePoint module
• Frequent updates to introduce new features and support for changes within SharePoint Online.
• Solid documentation.

Because Pnp.PowerShell is a community effort rather than something produced by Microsoft, some organizations are reluctant to use it. They fear that support for bug fixes will be limited or that some catastrophic bug will creep in due to a lack of testing. My experience is that the community developers are very responsive and do better testing than many Microsoft development groups (an example being the recent bugs afflicting the Microsoft Graph PowerShell SDK). There’s no reason to avoid using Pnp.PowerShell, subject to the normal requirements to test new versions and ensure that every cmdlet does what you expect.

Moving Forward with SharePoint Online PowerShell

Pnp.PowerShell wins the contest for popularity and coverage when it comes to PowerShell access to SharePoint Online. The official module appears stuck in time, and I know of no advocate within Microsoft who wants to bring it forward. The Graph tenant settings API started but hasn’t done much since 2022. Perhaps Microsoft should simply take Pnp.PowerShell over? Or maybe not, because then we might have three modules in a static state instead of two.

Stay updated with developments across the Microsoft 365 ecosystem by subscribing to the Office 365 for IT Pros eBook. We do the research to make sure that our readers understand the technology.

Restricted SharePoint Search Confines Copilot for Microsoft 365 Access to Curated Sites and User Content

The lights are obviously burning late in SharePoint engineering as Microsoft figures out new methods to help customers avoid inadvertent disclosure of confidential information through text generated by Copilot for Microsoft 365. Last month, we discussed how to exclude SharePoint Online sites and document libraries from search results to block Copilot access. Now, Microsoft has created Restricted SharePoint Search to deliver a more elegant (but possibly flawed) solution to allow organizations to control the sites accessible to Copilot.

Tenants with Copilot for Microsoft 365 licenses can enable Restricted SharePoint Search “from April onward.” By default, Restricted SharePoint Search is disabled and administrators will have to run some PowerShell commands to enable it for a tenant.

What is Restricted SharePoint Search?

Essentially, Restricted SharePoint Search disables enterprise-wide search. Instead, tenants can select up to 100 SharePoint Online sites that users can access (subject to the access defined for the sites). In this context, “users” includes Copilot for Microsoft 365 because when Copilot is active, it always operates as the signed-in user.

The restriction placed on enterprise search is pretty severe. Given the profusion of Microsoft 365 Groups and Teams, each of which has their own SharePoint Online site (and possibly several sub-sites for shared and private channels), limiting search to a curated list of 100 sites excludes most sites in anything but small tenants. Even my own small tenant supports over 400 sites.

Easing the Restrictions to Accommodate User Data

Microsoft will no doubt argue that the impact of limiting search to 100 sites is ameliorated by allowing users to search content from:

• Frequently visited SharePoint Online sites. However, Microsoft has not defined how many sites are in this category and how users can add sites to the list. For instance, does marking a site as a favorite put it on the list?
• Files in OneDrive for Business accounts that they have access to. This includes the user’s own OneDrive account, so they have full access to all their personal documents and other information stored in OneDrive.
• Files that are directly shared with users.
• Files that users create, edit, or view. In other words, if you touch a file stored in SharePoint Online, Copilot for Microsoft 365 can find and use that content.

I don’t know how Microsoft determined that 100 was a good number for the curated sites list. Determining what sites go onto the list and what sites are excluded will be an interesting exercise for many organizations. It seems like the intention is for tenants to include important corporate sites that everyone needs access to, like those holding HR information or details of released products and public documentation while relying on the frequently visited sites lists to deliver user-specific search results. It would be interesting to know how Microsoft uses Restricted SharePoint Search and if so, how they selected the 100 sites.

Restricted SharePoint Search does not affect how Microsoft Purview solutions like eDiscovery work. One way of thinking about the restriction is that it’s a form of trimming similar to the security trimming that Search already does to make sure that users only ever see sites and files in search results that they are entitled to access. This trimming further limits results to the 100 curated files plus the user’s OneDrive for Business account and files shared with them.

The Impact of Restricted SharePoint Search

It could be that this scheme will work well, but as Microsoft points out, Restricted SharePoint Search “limits the content Copilot can search and reference when responding” and “may impact its ability to provide accurate and comprehensive responses to prompts.”

Microsoft says that the new solution will help customers review and audit site permissions while continuing to deploy Copilot for Microsoft 365. A cynic might say that Restricted SharePoint Search is a cobbled together patch rushed out to assuage the concerns of customers who have heard about potential data disclosure problems and slowed the planning process for Copilot. It’s absolutely the right thing for Microsoft to address those concerns, but Restricted SharePoint Search seems like a sticking plaster that’s been applied until Microsoft can come up with a more flexible long-term solution. I guess we’ll know more when the software reaches customers in April and can assess just how well the 100 site limit works.

Longstanding Service Issue Retrieving SharePoint Usage Data

The Microsoft 365 ecosystem is so large that it’s hard to keep track of everything that changes that show up in different workloads. We’ve always known about the difficulties of tracking new features, deprecations, and other issues, but sometimes it takes a user to report something to focus on a specific problem.

An example is when a reader noted that the Graph-based script to report the storage quota used by SharePoint sites no longer included site URLs in the output (Figure 1). The original script (from 2020) used a registered Entra ID app to authenticate and use the Graph getSharePointSiteUsageDetail API to fetch site detail data.

Problems in the Graph APIs Accessing SharePoint Usage Data

When I investigated the problem, I decided to update the script code to use the Microsoft Graph PowerShell SDK instead. The update did nothing to retrieve the missing data. This isn’t surprising because the problem lies in the Graph API rather than the way the API is called.

The Microsoft 365 admin center uses the same Graph API for its SharePoint site usage report and the same problem of no site URL data is seen there (Figure 2).

Even worse, the SharePoint site activity report in the Microsoft 365 admin center displays no data (Figure 3).

This problem is because the getSharePointActivityUserDetail API returns no data whatsoever. Here’s an example of using the API in PowerShell in an attempt to retrieve SharePoint Online user activity for the last 180 days. The retrieved data should end up in the SPOUserDetail.CSV file.

$Uri = "https://graph.microsoft.com/v1.0/reports/getSharePointActivityUserDetail(period='D180')"
Invoke-MgGraphRequest -Uri $Uri -Method GET -OutputFilePath SPOUserDatail.CSV

However, the output file is perfectly empty apart from the column headers (Figure 4).

The same approach works perfectly with other usage data. For instance, this query works nicely to fetch Exchange Online usage data:

$Uri = "https://graph.microsoft.com/v1.0/reports/getEmailActivityUserDetail(period='D180')"
Invoke-MgGraphRequest -Uri $Uri -Method GET -OutputFilePath $EmailUsage.CSV

A Known Service Issue with SharePoint Usage Data

It’s not surprising that an API should have a problem. The APIs haven’t changed recently, so the root cause is more likely due to a change in the SharePoint Online back end. This feeling is reinforced by service health report SP676147 filed on 21 September 2023 (last updated 9 February 2024) that blithely says that “SharePoint and OneDrive URLs may not be displayed in some usage reports.”

Microsoft goes on to note that:

“We’re continuing our work through the validation of multiple potential mitigation strategies to display the URLs of the affected site usage reports. Due to the complexity of the scenarios involved we anticipate this may take additional time.”

The next update for the service health announcement is due on 1 March 2024. What I’m struggling with is that the usage reports included site URLs without any difficulty for years. Why it should suddenly become an issue is inexplicable. And taking over six months to find a solution is even more so.

Microsoft suggests that developers use the Graph Sites API to retrieve the site URL. For example:

$Uri = ("https://graph.microsoft.com/v1.0/sites/{0}" -f $Site.'Site Id')
$SiteData = Invoke-MgGraphRequest -Uri $Uri -Method GET

This works, but only when using an application permission. Using delegated permissions restricts access to sites that the signed-in user is a member of.

SharePoint PowerShell Still Works

Fortunately, it’s possible to get the site storage quota information using the SharePoint Online management PowerShell module. The Graph APIs read from a usage data warehouse that’s populated using background processes. The data is always at least two days old, but it’s much faster to access than using PowerShell to check the storage for each site. But needs must, and at least the old method still works.

I admit forgetting about the service health announcement, perhaps because it’s been ongoing for so long. I’m genuinely surprised that Microsoft is still working on something that seems so innocuous. And I’m even more surprised that customers aren’t making more of a fuss because the URL is the fundamental way to identify a SharePoint site.

Learn how to exploit the data available to Microsoft 365 tenant administrators through the Office 365 for IT Pros eBook. We love figuring out how things work.

Handling SharePoint Online Document Mismatch Notifications

I first wrote about SharePoint Online document mismatch notifications soon after their introduction in 2020. SharePoint sends these messages when users upload (or create) a document in a site that has a higher-priority sensitivity label than the label assigned to the site. The message (Figure 1) is intended to alert the author that they might have made a mistake when choosing a label or perhaps have stored sensitive information in a site intended for lower-level content.

Although SharePoint Online creates audit events when document mismatches occur, the audit events don’t contain details of the author who provoked the mismatch. To find that information, you need to correlate the notification with other audit records. This method works and it’s a good way to keep an eye on trends.

Suppressing Document Mismatch Notifications with a Mail Flow Rule

So far, so good. However, some organizations dislike document mismatch notifications because they cause users to call the help desk to ask what’s happening and what they should do next. The easy answer is to say that this is a simple matter of lack of user training. Before an organization implements sensitivity labels, they should educate users about the meaning and usage of the labels published to accounts. Just because a user doesn’t understand the difference between “General access” and ”Confidential” is not a reason to suppress informational emails. Another point I’ve heard made is that users tend to ignore system-generated messages, especially if they don’t understand why they received the email.

Another way of thinking about the situation is that if mismatches occur, fast intervention should occur to coach users about how to label information correctly. But to be able to intervene, the help desk (or another body) must know about the mismatch. It’s possible to scan for audit events daily (or more frequently) to find the data. Another solution is to use a mail flow rule to redirect mismatch notifications to an email address where those who help users to understand how to handle sensitive information can learn about issues as they happen.

The mail flow rule is straightforward (Figure 2). The rule looks for messages originating from a specific sender, which is the email address used for the document mismatch notifications (see discussion below). Only messages that match the subject of the notifications are diverted to one or more selected recipients. The targets can be individual recipients or groups. They do not have to be known Exchange Online mail-enabled objects.

Once the rule is enabled, the Exchange Online transport service intercepts document mismatch notifications and redirects the emails to the nominated recipient.

Domain-Specific Address for Service Emails

In both the figures, you’ll notice that a custom email address is used for the sender of the document mismatch notifications. This is a feature known as bring your own domain address for service messages. In other words, you can replace the default addresses used by Microsoft (like noreply@sharepoint.com) with an address that makes sense to those who receive notifications and other service messages. The address must be functional because users might respond to service messages requesting more information. I use a shared mailbox for this purpose.

According to message center notification MC705761 (10 January 2024, Microsoft 365 roadmap item 375694), Teams will soon give tenant administrators the option to send information messages from a selected domain. The update is coming to standard release tenants in mid-February 2024.

No Right Answer

There’s no right or wrong answer for how to deal with document mismatch notifications. It all depends on the individual circumstances within a tenant. Some users are very conscious about information protection and understand the most appropriate label to apply in any condition. Others are unsure. Coaching helps, but sometimes a little extra help doesn’t go amiss.

Learn about using SharePoint Online and the rest of Office 365 by subscribing to the Office 365 for IT Pros eBook. Use our experience to understand what’s important and how best to protect your tenant.

Track Technology Coverage with Custom Document Properties in Word Documents

Since mid-2018, I have published nearly 1,500 articles on this blog and perhaps another 500 articles on other sites, notably Practical365.com, which is where I publish long-form articles (more than 1,200 words).

I’m often asked how I track what I write to make sure that I cover the full breadth of Microsoft 365 instead of remaining focused on just one or two applications. Given my history of writing about Exchange Server, it would be easy to stay engrossed in Exchange Online. Personal bias and interest results in some applications receiving more attention than others from any author, but I do try to cover a reasonable mix of topics drawn from across Microsoft 365. Apart from anything else, writing about a new topic is a great way to become acquainted with that technology.

Using SharePoint Metadata

Keeping track of articles isn’t difficult. The source text for articles are in Word files stored in a document library in a SharePoint Online site. The only change to “normal” document processing is some additional columns to capture metadata that helps track the areas of technology covered by an article and the publication where the article appears.

The custom columns are defined in library settings (Figure 1). The Publication column stores the name of the web site hosting the article and the Published date column captures when the article is online. The Technology column stores the major technical focus (like SharePoint Online), while the Technology sub-area property captures an additional level of detail, if needed.

Newly updated columns are available immediately to the SharePoint web app, and that’s where most updating occurs using the details panel (Figure 2). Usually I wait until the text is finalized and published before completing the details to make sure that the data is accurate.

You can see that I use retention labels and sensitivity labels in the document library. The document library has a default sensitivity label and a default retention label to make sure that everything is protected.

Updating SharePoint Metadata in Word

It’s also possible to update SharePoint metadata in the Word (Win32) app (not the web version). After adding or modifying a column, the Word application must download the new metadata from the document library. This process can take a little time and it’s often easiest to force a download by exiting the document and reopening it.

Access to document properties is through the Quick Parts option of the Insert menu. Click Quick Parts and choose Document Property to see the list of available properties, including the custom columns added to the library (Figure 3). The majority of the properties shown are defined in Word and have no connection to SharePoint Online.

While possible, I don’t suggest that you update custom document columns using this method unless you want the text stored in the column to be part of the document content. The problem is that Word treats the property like any other quick part (such as a field used for figure or table numbering) and includes it in its text. Figure 4 shows updating a choice property in Word.

Because it’s a quick part, Word supports updating of the property at any time. Following an update, SharePoint learns about the new value and updates its metadata.

Generating Output from the Custom Metadata

The advantage of adding custom columns to a SharePoint Online document library and populating the columns with data is to create the basis for reporting. The easiest way is to use the Export to Excel feature to create a worksheet or CSV file. In both cases, the custom columns are in the exported data (Figure 5).

The file can be analyzed or imported into Power BI for visualization, or even imported into PowerShell for analysis and reporting there. I do that, but I suspect that most will be happier to remain in Excel.

Support the work of the Office 365 for IT Pros team by subscribing to the Office 365 for IT Pros eBook. Your support pays for the time we need to track, analyze, and document the changing world of Microsoft 365 and Office 365.

Sensitive by Default Blocks External Access Pending DLP Scanning

The ability to mark documents as “sensitive by default” achieved general availability in July 2020. Despite covering the topic in the DLP chapter of the Office 365 for IT Pros eBook, I never paid the feature much attention because sensitivity labels are often a better way to protect confidential material.

Not every tenant deploys sensitivity labels. Sometimes this is because they want to avoid the complications that can come when dealing with encrypted information, such as how to move encrypted documents between tenants during a migration. And sometimes it’s because the work to prepare to deploy and manage sensitivity labels is incomplete and pending completion. These are the kind of circumstances when the sensitive by default control is useful for tenants that have the necessary Office 365 E3 (or above) licenses to use Data Loss Prevention (DLP) policies with SharePoint Online.

The idea is simple. SharePoint Online uses background processes to implement the instructions in DLP to detect sensitive information in documents and take whatever action the policy settings dictate, such as to block sharing. Because DLP processing does not happen immediately for new files uploaded to to SharePoint Online and OneDrive for Business libraries, a short period exists when it’s possible for users to share sensitive data outside the tenant and inadvertently leak data. The sensitive by default control stops this happening by forcing SharePoint Online to consider all files as sensitive until DLP processes their content.

In effect, this means that SharePoint Online blocks external access to documents until DLP scans the contents. If external users, including guest members of a team, attempt to access a document before DLP scans its content, they see a page to tell them that scanning is in progress (Figure 1). After a few minutes, the scan should complete and access is possible.

Implementing Sensitive by Default

To implement the Sensitive by default control, you:

• Implement at least one DLP policy to scan the SharePoint Online sites that store information intended for external access.
• Run the Set-SPOTenant cmdlet in the SharePoint Online PowerShell module to block access to new files. It can take up to 15 minutes before the change is effective. The block applies to all sites in the tenant and you can’t exclude sites from its effect.

Here’s the command to implement the sensitive by default control:

Set-SPOTenant –MarkNewFilesSensitiveByDefault BlockExternalSharing

With the block in place, users can still share documents externally (if not blocked by the tenant’s sharing settings). However, external people with a sharing link cannot access the content until the document is scanned by a DLP policy.

To revert the block, run Set-SPOTenant to allow sharing without waiting for DLP processing:

Set-SPOTenant -MarkNewFilesSensitiveByDefault AllowExternalSharing

DLP Processing for Sensitive by Default

Any DLP policy that has a “contents contains” condition to process information in SharePoint Online sites can perform the check and release the block. Normally, DLP scanning either passes the document for external access (because DLP doesn’t detect a policy violation) or blocks it (because DLP detects some content that violates the policy if shared externally).

The Microsoft documentation for the feature discusses creating a form of “catch-all” DLP policy to cover all SharePoint Online sites and OneDrive for Business accounts in a tenant. The policy contains a rule to check new content for some arbitrary value. As shown in Figure 2, I use a check for the blood test sensitive information type.

It doesn’t matter that DLP is unlikely to detect this data in my tenant. Apart from that, the DLP policy doesn’t perform any action or notify anyone if it matches content. The sole purpose of the policy is to make sure that DLP processes every file uploaded to SharePoint Online and OneDrive for Business. Other DLP policies handle any problems lurking in documents.

Sensitive by Default and Sensitivity Labels

Applying the sensitive by default control is an effective way to stop external sharing from SharePoint Online and OneDrive for Business. However, it’s a broad-brush policy that covers the entire tenant. Using sensitivity labels to restrict access to documents containing important information might be a better approach, especially when auto-label policies are used to find and apply labels to documents at rest. The two approaches are not mutually exclusive and it’s a good idea to use sensitivity labels to control access to an organization’s most confidential information, including documents shared with external guests in Teams.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Preserve Copies of Cloudy Attachments for eDiscovery

A “cloudy attachment” is the term used when people send a link to a file stored in SharePoint Online or OneDrive for Business instead of attaching individual copies of the file with messages generated by Outlook, Teams, or Viva Engage. The idea is that recipients can work with the original content stored in SharePoint or OneDrive rather than making changes to their personal copies and then attempting to reconcile updates. Cloudy attachments can also result in the output generated through interactions with Microsoft 365 Copilot.

Exchange Online users have had the ability to send cloudy attachments for many years. However, it takes time for people to change the habits of a working lifetime, and it’s only relatively recently that I see more cloudy attachments in use, including when sharing documents across tenants. The passing of time and better internet access allows us to catch up with Microsoft’s vision. Attachments sent through Teams has always been based on sharing, so the percentage of cloudy attachments found in Teams is much higher than email.

The eDiscovery Problem Caused by Cloudy Attachments

Efficient as cloudy attachments are, they pose a problem for eDiscovery searches. The results for Purview eDiscovery standard cases and content searches include links for cloudy attachment sent in messages. However, they do not include the actual content of the linked file. eDiscovery investigators can follow the link to find the content of the attached file, but this can be an onerous task when search results include many messages with attachment links.

Microsoft’s Solution: Auto-label Copies of Cloudy Attachments

Microsoft solution depends on two factors. First, a method is needed to mark cloudy attachments so that they can be easily found by searches. Second, the marked files must be integrated into search results. This is what Microsoft does through a combination of auto-label retention policies and Purview eDiscovery (premium). You’ll need Office 365 E5 licenses to take advantage of their solution.

The first step is to deploy an auto-labeling policy using the content condition “apply labels to cloud attachments and links shared in Exchange, Teams, Viva Engage and Copilot” (Figure 1). The auto-label policy won’t be effective immediately because it must be deployed to all the sites covered with the policy. This can take up to a week, depending on the number of sites to cover.

The auto-label policy forces SharePoint Online to capture copies of files shared through messaging. When active, the a background job monitors for cloudy attachments that come within the scope of the policy (sent from the SharePoint Online sites and OneDrive for Business accounts specified in the policy). When the policy detects an in-policy cloudy attachment, it creates a copy of the file and stores the file in the SharedVersions folder of the Preservation Hold library for the host site (Figure 2).

Because auto-labeling happens using a background timer job, it can take up to an hour before the copy of a cloudy attachment is captured and labeled.

If someone modifies a file after sharing, a new version is captured in the Preservation Hold library. This step ensures that it’s possible for eDiscovery to find an attachment in the exact state at the time it was shared. Given that document content often changes as people work on it, knowing what a recipient sees in an attachment is a critical part of the eDiscovery process.

Copies of cloudy attachments labelled by auto-label policies remain in the Preservation Hold library until their retention period lapses. At that time, the normal method of processing the retention action occurs. Depending on how many cloudy attachments an organization generates, the preservation of captured copies might have a significant impact on the consumption of SharePoint Online storage.

To ensure that the current version of the original shared file is preserved, any files moved or deleted in the locations within the scope of the auto-label policy are automatically copied to the Preservation Hold library. These are temporary copies kept for one day to allow auto-label processing to happen and then removed. This form of temporary retention is unique to files within the scope of auto-labeling policies for cloudy attachments and is a simple safeguard to preserve all the copies of these files that might be needed for eDiscovery.

Unlike other auto-labeling policies which process data at rest to apply retention labels to content that already exists, auto-labeling of cloudy attachments is not retrospective. The only attachments that are captured and retained are those sent once the policy is in force.

Retention Labels Stamped on Captured Copies of Cloudy Attachments

The auto-label policy stamps the captured copies of cloudy attachments with the retention label defined in the policy. Because they have no access to the Preservation Hold library, users who send the messages with the cloudy attachments are unaware that the captured copies have retention labels.

To avoid problems with attachments that are shared multiple times, Microsoft recommends that the retention label chosen for the auto-label policy starts its retention period from the time when the policy applies the label to the copy of the shared attachment. The retention label applied by the policy does not have to be published to users or locations. In fact, it’s probably a good idea to create a retention label specially created for use with cloudy attachment auto-labeling policies.

Discovering Information About Captured Cloudy Attachments

As you can see, the captured copies have obfuscated file names. To discover more about a file, use the Version history option. As you can see in Figure 3, the name of the document and its original location are clearly visible, as is the date when the policy captured the copy.

More information about a captured attachment is available by checking its compliance details. In Figure 4 we can see details of the retention label assigned to the file. If the auto-label policy works, the label should be the one defined in the policy.

The existence of captured attachments means that it now becomes possible to retrieve copies of the attachments during eDiscovery operations. A preview feature in the workflow for Purview eDiscovery premium cases leverage this capability to collect copies cloudy attachments in the state they were shared. Investigators can review content either at the time when a file was shared or its current state.

Good for eDiscovery People

Obviously this is a feature that is of interest to those working with eDiscovery cases, specifically with access to Purview eDiscovery (premium). For all that, it’s an interesting example of how a change made in applications (cloudy attachments) creates issues down the line for other technology. More information about the retention of cloud attachments and how auto-label policies work is available in Microsoft documentation.

Support the work of the Office 365 for IT Pros team by subscribing to the Office 365 for IT Pros eBook. Your support pays for the time we need to track, analyze, and document the changing world of Microsoft 365 and Office 365.

SharePoint Embedded Latest Idea to Exploit SharePoint Storage

Microsoft executives love to cite statistics to prove the power of technology during conference keynotes. Thus we heard from SharePoint overload Jeff Teper that Microsoft 365 users currently generate new 2.3 billion documents for SharePoint Online daily. Teper delivered the statistic with great gusto during his keynote at the European SharePoint Conference (ESPC 23) in Amsterdam last week, and it, along with the announcement about the public preview for SharePoint Embedded, got me thinking about the role of SharePoint Online within the Microsoft 365 ecosystem.

Over two billion documents created daily seems like a lot of content. Given that Office 365 has over 400 million paid seats and (possibly more importantly) Teams has more than 320 million monthly active users, it works out at about six new pieces of content per user daily. Microsoft didn’t say if the figure was for business days, so we’ll assume it is a daily average. Given that people work every day, that’s a reasonable assumption.

Six new pieces of content is a lot for some people and not for others. It’s not just Office documents because SharePoint Online has become the storage engine for many Microsoft 365 applications. Of course, classic SharePoint and OneDrive for Business usage drives a lot f consumption, but then there’s Teams meeting recordings, whiteboards, Stream videos, files migrated from on-premises and other platforms, Microsoft Lists, Loop workspaces, and all the other information introduced into SharePoint through apps and connectors.

As always, Microsoft provides very little context for a headline figure used in conference sessions, so we assume that very item created in SharePoint Online storage counts towards the total. The actual number of Office documents created in SharePoint Online by humans daily is probably much smaller than 2.3 billion, but there’s no doubt that more content is created than ever before.

Apps and Technology Improve the SharePoint Experience

I believe that Teams exerts huge influence over the demand for SharePoint Online storage. Soon after its introduction, I said that Teams delivers a human interface for SharePoint (and OneDrive for Business). Sure, people use the SharePoint browser client to work with documents, but it’s much easier to use Teams and store files in SharePoint sites without realizing that’s what’s happening.

The OneDrive sync client is an important factor here too. Microsoft’s first sync client was awful, but the current generation works very well and makes it easy to keep local copies of files synchronized with the server. Features like auto save build on OneDrive synchronization and make sure that work done in Office documents is now hardly ever lost, even after catastrophic PC failures.

SharePoint Embedded, Apps, and Costs

To return to the original announcement, SharePoint Embedded is a renaming of Syntex Repository Services, revealed earlier this year as the storage location for Loop workspaces. SharePoint Embedded is also used by Microsoft Designer (Figure 1). The big news is that Microsoft is making SharePoint Embedded available to customers and ISVs to store their application data.

Holding data in SharePoint Embedded is an attractive notion because it allows applications to take advantage of Microsoft security and compliance features and the availability delivered by the Microsoft Cloud datacenter network. The downside is ceding some level of control over applications to Microsoft and an uncertain cost model.

Computing services must be paid for and SharePoint Embedded uses the pay-as-you-go model for storage and the Graph API transactions executed by applications to interact with SharePoint. Cost and licensing details are available online but the raw detail must be converted into actual costs based on an application profile before developers understand how much they need to charge to turn a profit. Experience will help people estimate costs more accurately over time. For now, Microsoft outlines the costs listed in Table 1. These are for the preview of SharePoint Embedded and may change when the service is generally available.

I can find no detailed information about what class A and class B transactions mean, unless these refer to the same definitions as used for the Teams Graph APIs where class A means transactions that perform a security or compliance function and class B do not.

Interpreting the potential costs for an application based on SharePoint Embedded will require some testing and analysis of the information reported for the Azure subscription used to pay.

More Business Opportunities

The large and increasing size of the Microsoft 365 installed base creates significant business opportunities for Microsoft to upsell services to its customers. SharePoint Premium (introduced at Ignite 2023 and tagged as the “next evolution for Syntex“) includes advanced content management and some new services, like SharePoint Signature (apply eSignatures to documents). Microsoft 365 Backup and Microsoft 365 Archive are other monetization opportunities, both of which use the pay-as-you-go model.

SharePoint Embedded is another way for Microsoft to create value from its Microsoft 365 infrastructure. It seems like a good idea. Time will tell if it is.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Understand the Impact Loop Workspaces Have on SharePoint Online Quota

Message center notification MC678308 (updated 2 November 2023) explains that the storage consumed by Loop workspaces (created with the Loop app rather than Loop components in Teams and Outlook) will count against tenant storage quotas. During the preview, Microsoft allowed people to use the Loop app without a license and create as many workspaces as they liked. The only limitation was on the size of an individual workspace, which was capped at 5 GB. Workspace data is held in a special form of SharePoint storage called Syntex repository services and Microsoft didn’t limit the storage occupied by the workspaces.

All good things come to an end. As the Loop app approaches the end of its public preview stage and moves toward general availability (I expect an announcement at the Ignite conference), Microsoft has revealed its hand with respect to licensing and storage. Only people with certain Microsoft 365 product licenses will be able to create new workspaces

Loop Counts Against Storage Now

According to MC678308, Microsoft will start counting Loop workspaces against tenant storage quotas between late October and late November 2023. When the change goes into effect for a tenant, the maximum size of a workspace increases from 5 GB to 1 TB.

The exact impact on a tenant is hard to know unless you use the Get-SPOContainer cmdlet in the SharePoint Online management module to fetch details of each tenant. For example, this command fetches details of existing workspaces:

[array]$LoopWorkspaces = Get-SPOContainer -OwningApplicationID a187e399-0c36-4b98-8f04-1edc167a0996
If (!($LoopWorkspaces)) {
    Write-Host "Can't get Loop workspaces - exiting"; break
}

The details reported by Get-SPOContainer miss some important information. For instance, while the creation date for a workspace is available, the last updated date is not, nor is detail about the person who last updated the workspace. Understanding the date when a workspace was last changed is critical to knowing if a workspace is in active use.

Reporting Loop Workspaces

This code generates a report with details of the storage used by each workspace and whether the workspace owners have one of the four licenses required to create new Loop workspaces:

$Report = [System.Collections.Generic.List[Object]]::new()
$TotalBytes = 0; $LicenseOK = 0; $i = 0
ForEach ($LoopSpace in $LoopWorkspaces) {
    $i++
    Write-Output ("Analyzing workspace {0} {1}/{2}" -f $LoopSpace.ContainerId, $i, $LoopWorkspaces.count)
    # Get detail of the workspace
    $LoopSpaceDetails =  Get-SPOContainer -OwningApplicationID a187e399-0c36-4b98-8f04-1edc167a0996 -Identity $LoopSpace.ContainerId
    # Get detail about the owner
    [array]$Owners = $LoopSpaceDetails.Owners
    ForEach ($Owner in $Owners) {
        $LicenseFound = $Null; $LoopLicenseStatus = "Unlicensed";  $LicenseName = $Null
        # Find if the Loop service plan is successfully provisioned for the account
        [array]$UserLicenseData = Get-MgUserLicenseDetail -UserId $Owner
        $LoopLicense = $UserLicenseData | Select-Object -ExpandProperty ServicePlans | `
             Where-Object {$_.ServicePlanId -eq $LoopServicePlan} | Select-Object -ExpandProperty ProvisioningStatus
        If ($LoopLicense -eq 'Success') {
            $LicenseOK++
            $LoopLicenseStatus = "OK"
        }
        # Find what SKU the Loop service plan belongs to
        $User = Get-MgUser -UserId $Owner -Property Id, displayName, department, UserPrincipalName
        [array]$SKUs = $UserLicenseData.SkuId
        ForEach ($Sku in $Skus) {
            $LicenseFound = $LoopValidLicenses[$Sku]
            If ($LicenseFound) {
                $LicenseName = $LicenseFound
            }
        }
    }
    [array]$Members = $Null
    [array]$Managers = $LoopSpaceDetails.Managers
    ForEach ($Manager in $Managers) {
        $Member = Get-MgUser -UserId $Manager
        $Members += $Member.DisplayName
    }

    $StorageUsed = "{0:N2}" -f ($LoopSpaceDetails.StorageUsedInBytes/1MB)
    $TotalBytes = $TotalBytes + $LoopSpaceDetails.StorageUsedInBytes

    $ReportLine = [PSCustomObject]@{
        ContainerId    = $LoopSpace.ContainerId
        App            = $LoopSpaceDetails.OwningApplicationName
        Name           = $LoopSpace.ContainerName
        Description    = $LoopSpace.Description
        Owner          = $User.DisplayName
        UPN            = $User.UserPrincipalName
        License        = $LoopLicenseStatus
        Product        = $LicenseName
        Members        = ($Members -Join ", ")
        Created        = $LoopSpaceDetails.CreatedOn
        SiteURL        = $LoopSpaceDetails.ContainerSiteUrl
        "Storage (MB)" = $StorageUsed
    }
    $Report.Add($ReportLine)
}

Figure 1 shows an extract of the information captured by the script. You can see that the James Ryan account is deemed to be unlicensed. This is because the account doesn’t hold a product licenses containing the Microsoft Loop service plan. Also note that new users all receive the Ideas workspace to help get them started with the Loop app. The workspace isn’t large (0.11 MB), but it’s a bit cheeky for Microsoft to charge for it.

Checking individual workspace containers is not a fast operation. The script can be sped up by removing the Get-MgUser commands used to fetch details about the licenses possessed by workspace owners.

You can download the complete script from GitHub. Remember that the intention of the script is to illustrate a principal rather than being a complete solution. Feel free to make whatever changes you deem to meet the circumstances of your tenant.

Update: The original script was limited to reporting the first 200 workspaces in a tenant. An updated script handles pagination to find and report all workspaces.

No Immediate Impact

It’s unlikely that Loop workspaces will have much of an impact on SharePoint Online tenant storage quotas in the immediate future. Documents will continue to be the major consumer of quota, even when tenants have the Microsoft 365 licenses necessary for users to create new Loop workspaces. Even so, it’s a good idea to keep an eye on how Loop is being used and how much space its files occupy.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Loop Components in Channel Conversations

Message center notification MC681251 (13 October 2023, Microsoft 365 roadmap item 123491) announces support for using Loop components in channel conversations in both the classic and new Teams client. Targeted release is due in November 2023 with standard release tenants getting the new capability toward the end of the year.

About two years ago, Teams chat became the first application to support Loop components, and the components are currently supported in other clients like OWA and Outlook desktop in addition to the full Loop app. We’ve also seen developments such as Microsoft enabling support for the Loop task list component in Planner and the code block component and recently had the surprise that Microsoft will only include access to the Loop app in certain Microsoft 365 product SKUs. You can say that there’s been a lot of activity since November 2021.

Pop-up Announcement for Loop in Channels

Recently, I’ve complained about the number of “helpful” (aka irritating) pop-ups that Teams insists in showing to users. The arrival of Loop components in channels was dutifully announced in a new pop-up (Figure 1).

In this instance, the pop-up was helpful because informed me how to invoke a Loop component in a channel message by using the Actions and apps button to find the Collaborate with Loop option (Figure 2). The same set of Loop components available in Teams chat can be used in channel messages.

If an organization has disabled Loop components for Teams, the option to use them in channel conversations won’t appear.

Including Loop Components in Channel Messages

When composing a channel message, you can incorporate Loop components with text and other elements. Figure 3 shows a message being composed with a Loop table component, some explanatory text, and the compulsory emoji. The image used to highlight the announcement post was created using Microsoft Designer (see this article).

When a user posts a message including a Loop component, Teams stores the Loop component in the channel folder of the SharePoint site belonging to the team. All team members have access to the Loop component. The other message elements are in the Teams message store.

Working with a Loop component in a channel conversation is just like working with these objects in other applications. Anyone with access to the component can change its content and the other users with access see the changes in almost real-time (Figure 5).

Anyone participating in the channel conversation can copy the Loop component into another app and reuse the information there. For instance, they could copy the component into an Outlook email sent to someone for their comment. That recipient can update the component without being a member of the team. This is a really nice way to bring additional wisdom into a discussion.

To allow access non-team members who receive a component through another app, I suspect that Teams generates a sharing link. The link uses the default type of sharing link for the organization, which in my tenant is “only people in your organization” and the default permission, which is edit. I base this on Microsoft’s documentation for known issues with Loop, one of which is what happens when the default sharing link is for specific people only. I could be wrong, but that’s what it seems is happening.

Support for Loop components is available in all channel types (regular, private, and shared). However, guest members of the team cannot access Loop components stored in private and regular channels and external members can’t use them in shared channels.

Mobile Support

The Teams iOS and iPadOS clients support view-only access to Loop components in channel messages. The same support isn’t currently available for the Android client. No mobile client can create a Loop component in a channel message.

Loop for Whiteboard

According to message center notification MC681250 (13 October), Microsoft is also bringing Loop components (task lists, tables, voting tables, progress trackers, and checklists) to Whiteboard for the web, the Whiteboard app in Teams (apart from Teams Room devices), and Whiteboard for Windows. Deployment should be complete by the middle of November 2023.

No Access for External Collaborators

The continuing lack of support for external access to Loop components is a major frustration with the technology. Support for sharing with external people is obvious available because it works for guest access to Office documents stored in the SharePoint Online sites used by Teams. Those documents support co-authoring where the applications synchronize content frequently to update co-authors.

The current restriction might be due to the more frequent synchronization model used by Loop. Extending to people outside the tenant might create synchronization difficulties and detract from the Loop experience. Another obvious issue is that guest accounts don’t have to belong to Microsoft 365 tenants, so the necessary infrastructure might not exist for some guests to interact with Loop. For instance, what does a guest account with a Gmail account do?

Collaborating through Loop components in Teams channels is a powerful tool for working with people within the same organization. Regretfully, its usefulness for some tenants is limited by a lack of support for external team members. I hope that Microsoft lifts that restriction soon, perhaps initially for guests from other Microsoft 365 tenants. That would certainly make Loop components in channel conversions much more interesting to me.

Learn about using Loop and the rest of Microsoft 365 by subscribing to the Office 365 for IT Pros eBook. Use our experience to understand what’s important and how best to protect your tenant.

Easier to Create SharePoint Lists with PnP.PowerShell

Updated 14 August 2024

Last week, I wrote about how to use cmdlets from the PnP.PowerShell module to create and populate a list in a SharePoint Online site using data generated by the Teams Directory script. As benefits a module deeply rooted in SharePoint history, the cmdlets worked well and the script wasn’t too difficult to write.

The Microsoft Graph is supposed to be “the gateway to data and intelligence in Microsoft 365. It provides a unified programmability model that you can use to access the tremendous amount of data in Microsoft 365…” I don’t have much argument with this assertion because in most cases, it’s true. That is, until you come to SharePoint Online where the coverage of SharePoint objects and data is not as good as for other workloads.

This article describes some of the challenges involved in writing a script based on Microsoft Graph PowerShell SDK cmdlets and Graph API requests to create SharePoint lists similar to what I did using PnP.PowerShell. Let’s see how I got on.

How to Create SharePoint List Script in a Nutshell

The script is simple in concept. The data comes from a CSV file generated by the Teams Directory script. The script creates a list in a SharePoint Online site and populates the list with items imported from the CSV file. The plan was to use cmdlets from the Microsoft Graph PowerShell SDK (V2.8) because there appears to be cmdlets available for everything the script needs to do.

Connecting to the Graph and the Target Site

The first steps connect to the Graph with the relevant permissions and retrieve details of the site holding the list. The script then checks if the list already exists and if found, removes the list. Rebuilding a list from scratch is easier than attempting to synchronize changes.

Connect-MgGraph -Scopes Sites.ReadWrite.All, Sites.Manage.All -NoWelcome
$ListName = "Teams Directory - Graph"
# Get target site 
Write-Host "Fetching details of the target site and list..."
$Site =  Get-MgSite -Search 'Office 365 for IT Pros Communications'
# Get List
$List = Get-MgSiteList -SiteId $Site.Id -Filter "displayName eq 'Teams Directory - Graph'"
If ($List) {
    # Delete the list
    Write-Host ("Removing previous version of list {0}" -f $List.DisplayName)
    Remove-MgSiteList -SiteId $Site.Id -ListId $List.Id
}

Removing a list like this won’t work if a retention label applies to the list.

Create SharePoint List with New-MgSiteList

The next step creates the list. The Graph SDK includes the New-MgSiteList cmdlet, but no matter what I did with the cmdlet, it refused to co-operate. Even the example from the Microsoft documentation failed with the following error:

New-MgSiteList_Create: Unable to determine type of provided column definition
 
Status: 400 (BadRequest)
ErrorCode: invalidRequest
Date: 2023-10-20T16:44:06

As described in this SDK bug report, the problem is that the columns shown in the example define the data type for each column but not what’s acceptable in the column (see this page for more detail about the supported types for list columns). For instance, the text data type can be plain text or rich text or both. If you don’t want to be this specific when creating a list (because you want to customize the list through the GUI afterwards), you can run the Invoke-MgGraphRequest cmdlet to create the list as shown below:

Write-Host "Defining the new list"
$Uri = ("https://graph.microsoft.com/v1.0/sites/{0}/Lists" -f $Site.Id)
$ListDetails = '{
    "displayName": "Teams Directory - Graph",
    "description": "Discover teams to join in Office 365 for IT Pros",
    "columns": [
      {
        "name": "Deeplink",
        "description": "Link to access the team",
        "text": { }
      },{
        "name": "Description",
        "description": "Purpose of the team",
        "text": { }
      },
      {
        "name": "Owner",
        "description": "Team owner",
        "text": { }
      },      
      {
        "name": "OwnerSMTP",
        "description": "Primary SMTP address for owner",
        "text": { }
      },
      {
        "name": "Members",
        "description": "Number of tenant menbers",
        "number": { }
      },
      {
        "name": "ExternalGuests",
        "description": "Number of external guest menbers",
        "number": { }
      },
      {
        "name": "Access",
        "description": "Public or Private access",
        "text": { }
      },
    ],
  }'
Invoke-MgGraphRequest -Uri $Uri -Method POST -Body $ListDetails | Out-Null

The Graph request creates a blank list. The new list includes the specified columns and a single column called Title inherited from the template. If you want to use a column called Title, you can leave it as is. If not, you can rename the column, which is what the script does to make the Title column to be TeamName. The internal name of the column remains Title, which is important to remember when updating records.

$List = Get-MgSiteList -SiteId $Site.Id -Filter "displayName eq 'Teams Directory - Graph'"
$ColumnId = (Get-MgSiteListColumn -SiteId  $Site.Id -ListId $List.Id | `
    Where-Object {$_.Name -eq 'Title'}).Id
Update-MgSiteListColumn -ColumnDefinitionId $ColumnId -SiteId $Site.Id -ListId $List.Id `
  -Description 'Name of the team' -DisplayName 'Team Name' -Name 'TeamName' | Out-Null

Adding Records to the List

After preparing the list, the script populates it with data imported from the Teams Directory. I ran into issues with the New-MgSiteListItem cmdlet. This could be a documentation issue, but some internet forums (like this example) indicate that this cmdlet has not had a happy history. I ended up creating each item as a custom object, wrapping the item data inside another custom object, converting it to JSON, and using the JSON content as a payload to post to the items endpoint:

$Uri = ("https://graph.microsoft.com/v1.0/sites/{0}/lists/{1}/items" -f $Site.Id, $List.Id)
ForEach ($Team in $TeamsData) {
  Write-Host ("Adding directory record for team {0} {1}/{2}" -f $Team.Team, $i, $TeamsData.Count)
  $i++
  $FieldsDataObject  = [PSCustomObject] @{
        Title          = $Team.Team
        Deeplink       = $Team.Deeplink
        Description    = $Team.Description
        Owner          = $Team.Owner
        OwnerSMTP      = $Team.OwnerSMTP
        Members        = $Team.Members
        ExternalGuests = $Team.ExternalGuests
        Access         = $Team.Access
  }
  $NewItem = [PSCustomObject] @{
        fields         = $FieldsDataObject
  } 
  $NewItem = $NewItem | ConvertTo-Json
  $Status = Invoke-MgGraphRequest -Method POST -Uri $Uri -Body $NewItem
  If ($Status.Id) {
     Write-Host ("Record added to list with id {0}" -f $Status.Id)
  }
}   

This approach works, but I could never write to a hyperlink field (something that the Add-PnPListItem cmdlet can do). Apparently, the Graph doesn’t currently support list hyperlink fields, so I ended up writing the deeplink to a team to a text field. The result is the list shown in Figure 1 where users see deeplinks that are not clickable. Users can copy the link to a browser tab and navigate to Teams that way, but that’s not very user-friendly. For small lists, you can create a hyperlink field in the list and copy deeplinks to that field. Users can then click on the link in the hyperlink field. Such a solution is unacceptable at any scale.

You can download the full script from GitHub.

Choose PnP.PowerShell to Create SharePoint Lists

What I learned from the exercise is that the PnP.PowerShell module is a more robust and reliable tool to use when working with SharePoint Online lists. PnP has its own quirks, but it works. I spent far too long chasing Graph SDK cmdlets that didn’t work as documented or couldn’t do what I wanted, so I recommend that you use PnP until Microsoft sorts out the SDK cmdlets and documentation.

In closing, I asked Bing Chat Enterprise to write a script to create and populate a list in a SharePoint site Online based on the Microsoft Graph PowerShell SDK. The results were impressive (Figure 2).

After this experience, I might use Bing Chat Enterprise more often in the future to sketch out the basics of scripts. In this case, Bing Chat Enterprise was helpful. In others, it’s been awful. But that’s the nature of generative AI in respect of its ability to regurgitate errors couched in what seems to be impressive terms.

Keep up to date with developments like how to create SharePoint lists with the Microsoft Graph PowerShell SDK by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers understand the most important changes happening across Office 365.

Create SharePoint List from Data Extracted from Teams

The article discussing a PowerShell script to generate a Teams directory explains how to create output files in different formats that can be used to make the directory available to users. For instance, you could post a HTML format version of the directory in a SharePoint Online site. Discussion about the post generated some nice ideas, amongst which was the suggestion to output the directory as a SharePoint list (aka Microsoft Lists).

I haven’t done much to manage SharePoint lists with PowerShell, so this seemed like a nice opportunity to explore the idea and increase my knowledge.

Choosing the Right Module to Create SharePoint List

The first order of business is to choose a PowerShell module for the task. I started off with the Microsoft Graph PowerShell SDK, which includes cmdlets like New-MgSiteList and Get-MgSiteList. Unhappily, I ran into several problems with SDK cmdlets (V2.8) that I’ve reported to Microsoft. The documentation and examples for these SDK site cmdlets are not as good as other areas covered by the SDK, so the problems could be due to misunderstanding on my part.

This brought me to the Pnp.PowerShell module (aka “Microsoft 365 Patterns and Practices PowerShell Cmdlets”). PnP is a community effort to create resources that help people to build app on the Microsoft 365 platform. The big advantage of PnP is that its cmdlets can interact with SharePoint Online content like list items where the Microsoft SharePoint management module is limited to tenant and site settings.

Basic Steps in the Script to Add Teams Directory Records and Create SharePoint List

The basic steps in the script are:

• Connect to the site that stores the list. I created a communications site for this purpose.
• Look for the list and if found, remove it because it’s easier to create and populate a new list instead of attempting to synchronize changes since the last update for the team directory.
• Create the list and the columns used to store team directory information. Many templates are available for Lists. I used the Links template and removed one of the two default columns.
• Populate the list with new items. To do this, the script reads the information in from the CSV file created by the original script and writes them as new list items.

PnP.PowerShell Cmdlets Used to Create SharePoint List

Translating the above into PnP PowerShell, the script uses the following cmdlets:

• Connect-PnpOnline to connect to the target site. PnP supports different forms of authentication. For the purpose of this demonstration, the script prompts for credentials of a site administrator and uses those to connect.
• Get-PnPList to check if the target list already exists and Remove-PnPList to remove the list if found.
• New-PnPList to create the target list.
• Add-PnPField to define the set of fields used to store directory information.
• Remove-PnPField to remove the standard Notes field inherited from the Links template. Here’s how the script creates the list and the fields used to store Teams directory information:

New-PnpList -Title $ListName -Template Links -EnableVersioning -Connection $Connection | Out-Null
# Add fields
Add-PnpField -List $ListName -DisplayName 'Team Name' -Internalname TeamName -Type Text -AddToDefaultView | Out-Null
Add-PnpField -List $ListName -DisplayName 'Description' -Internalname Description -Type Text -AddToDefaultView | Out-Null
Add-PnpField -List $ListName -DisplayName 'Owner' -Internalname Owner -Type Text -AddToDefaultView | Out-Null
Add-PnpField -List $ListName -DisplayName 'Owner SMTP Address' -Internalname OwnerSMTP -Type Text -AddToDefaultView | Out-Null
Add-PnpField -List $ListName -DisplayName 'Member count' -Internalname MemberCount -Type Number -AddToDefaultView | Out-Null
Add-PnpField -List $ListName -DisplayName 'External count' -Internalname ExternalCount -Type Number -AddToDefaultView | Out-Null
Add-PnpField -List $ListName -DisplayName 'Access' -Internalname AccessMode -Type Text -AddToDefaultView | Out-Null
# Remove the Notes field inherited from the Links template
Remove-PnPField -List $ListName -Identity Notes -Force

• Add-PnPListItem to populate the list with items imported from the CSV file. Here’s how the script populates the list:

[array]$TeamsData = Import-CSV -Path $CSVFile
[int]$i = 0
ForEach ($Team in $TeamsData) {
    $i++
    Write-Host ("Adding record for team {0} {1}/{2}" -f $Team.Team, $i, $TeamsData.count)
    Add-PnPListItem -List $ListName -Values @{
        "URL" = $($Team.Deeplink);
        "TeamName" = $($Team.Team);
        "Description" = $($Team.Description);
        "Owner" = $($Team.Owner);
        "OwnerSMTP" = $($Team.OwnerSMTP);
        "MemberCount" = $($Team.Members);
        "ExternalCount" = $($Team.ExternalGuests);
        "AccessMode" = $($Team.Access);
    } | Out-Null
}

The original version of the Teams Directory script generates a directory record for each team including a clickable deeplink to allow users to open Teams in the selected team. They can then join the team (public teams) or request the team owner to join (private teams). The deeplink generated by the script is formatted to make it clickable when exported to a HTML report. I updated the script to include a simple deeplink because SharePoint list entries don’t need the formatting.

Figure 1 shows the Teams directory records in a SharePoint Online list. I’m sure that the visual appearance of the list could be improved by tweaking the columns, but what’s here is sufficient to demonstrate the principles behind creating and populating a list.

You can download a copy of the full script from GitHub.

Lots to Explore in Lists

The SharePoint community understands and takes full advantage of lists (here’s an example). Others in the Microsoft 365 world might not. Perhaps this example of extracting information from one area of Microsoft to create a SharePoint list and populate the list with Teams directory information might get your creative juices flowing.

Learn how to exploit the data available to Microsoft 365 tenant administrators through the Office 365 for IT Pros eBook. We love figuring out how things work.

Improving the Speed of reporting Teams SharePoint URLs by Replacing the Get-UnifiedGroup Cmdlet

Last week, following a response to a reader question, I updated an article describing how to create a report of Teams and the URLs for the SharePoint Online sites used to store shared files. The only real improvement I made to the script was to use the Get-ExoRecipient cmdlet to resolve the members of the ManagedBy property to output display names instead of mailbox names. This change is necessary since Exchange Online moved to using the External Directory Object ID (EDOID) as the mailbox name to ensure uniqueness. Not everyone can recognize a mailbox GUID and know what mailbox it refers to.

The script uses the Get-UnifiedGroup cmdlet to find team-enabled groups. After reviewing the code, I wondered if it was possible to speed up processing by replacing the Exchange Online cmdlets with Microsoft Graph PowerShell SDK cmdlets or API requests. It’s always been true that the Get-UnifiedGroup cmdlet is relatively slow. This situation is explainable because the cmdlet fetches a lot of data about a Microsoft 365 group from multiple workloads. Microsoft has improved the performance of Get-UnifiedGroup over the years, but it’s still not the most rapid cmdlet you’ll ever use.

Converting to Graph SDK Cmdlets

Converting the script to use Microsoft Graph PowerShell SDK cmdlets isn’t very difficult. Here’s the code:

# Check that we are connected to Exchange Online
$ModulesLoaded = Get-Module | Select-Object -ExpandProperty Name
If (!($ModulesLoaded -match "ExchangeOnlineManagement")) {Write-Host "Please connect to the Exchange Online Management module and then restart the script"; break}
 
Connect-MgGraph -NoWelcome -Scopes Group.Read.All, Sites.Read.All
Write-Host "Finding Teams..."
[array]$Teams = Get-MgGroup -Filter "resourceProvisioningOptions/any(x:x eq 'Team')" -All
     
If (!($Teams)) {
   Write-Host "Can't find any Teams for some reason..."
} Else {
  Write-Host ("Processing {0} Teams..." -f $Teams.count)
  $TeamsList = [System.Collections.Generic.List[Object]]::new()    
  ForEach ($Team in $Teams) { 
   $SPOSiteURL = (Get-UnifiedGroup -Identity $Team.Id).SharePointSiteURL  [array]$Channels = Get-MgTeamChannel -TeamId $Team.Id
   [array]$Owners = (Get-MgGroupOwner -GroupId $Team.Id).AdditionalProperties.displayName
   $DisplayNames = $Owners -join ", "
   $TeamLine = [PSCustomObject][Ordered]@{
      Team      = $Team.DisplayName
      SPOSite   = $SPOSiteURL
      Owners    = $DisplayNames  }
   $TeamsList.Add($TeamLine)
  }
  $TeamsList | Out-GridView
  $TeamsList | Export-CSV -NoTypeInformation c:\temp\TeamsSPOList.CSV
}

Figure 1 shows the result.

You’ll notice that I still use the Get-UnifiedGroup cmdlet to fetch the Teams SharePoint URL. It’s possible to retrieve this information using the Graph with code like:

   $Uri = ("https://graph.microsoft.com/v1.0/groups/{0}/drive/root/webUrl" -f $Team.Id)
   $SPOData = Invoke-MgGraphRequest -Uri $Uri -Method Get
   [string]$SPODocLib = $SPOData.Value
   $SPOSiteUrl = $SPODocLib.SubString(0, $SPODocLib.LastIndexOf("/"))

Or:

   $Uri = ("https://graph.microsoft.com/v1.0/groups/{0}/sites/root" -f $Team.Id)
   $SPOData = Invoke-MgGraphRequest -URI $Uri -Method Get
   $SPOSiteUrl = $SPOData.WebURL

The Problem with Permissions when Fetching Teams SharePoint URLs

In both cases, the code works. However, the code fails for some teams due to the restriction placed on interactive use of the Graph SDK. When you connect an interactive session to the Graph, you’re restricted to using delegate permissions. The only data that the Graph SDK cmdlets can access is whatever the signed-in user can access. This is very different to the permissions model used by modules like the Exchange Online management module, which allow access to data based on RBAC controls, meaning that a tenant administrator can access everything.

The restriction disappears when running the SDK cmdlets using a registered app or an Azure Automation runbook. Now the cmdlets can use application permissions, so they can access any data permitted by the Graph permissions assigned to the service principal of the app.

Using either version of the code shown above works perfectly and returns the SharePoint site URL, but only for sites accessible to the signed-in user. Attempts to access any other site returns a 403 forbidden error.

I even tried using the Teams Graph cmdlets:

[array]$Channels = Get-MgTeamChannel -TeamId $Team.Id
$Files = (Get-MgTeamChannelFileFolder -TeamId $Team.Id -ChannelId $Channels[0].Id).WebURL
$SPOSiteUrl =  $Files.SubString(0,$Files.IndexOf("sites/")) + "sites/" + $Team.MailNickName

Again, this approach works for teams that the signed-in user is a member of, but not for other teams.

Going Back to Pure Exchange Cmdlets to Report Teams SharePoint URLs

The problem with permissions meant that I had to use a hybrid of Graph SDK cmdlets to get everything except the SharePoint site URL. And while this approach works, it’s slower than the original implementation using only Exchange Online cmdlets. In several runs against 88 teams the hybrid version took an average of 42 seconds to finish. The Exchange version required an average of 31 seconds.

The learning here is that Graph SDK cmdlets aren’t always the best choice for speed, no matter what you read on the internet. It’s always worth testing to find which approach is the most functional and fastest. Sometimes both boxes are ticked, and that’s a result.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

No Good Reason why SharePoint Limits Administrator Access to Document Libraries

A reader asked if a programmatic method exists to set the default sensitivity label for a SharePoint Online document library. The simple answer is “yes,” because the only way initially available to set a default sensitivity label when the feature was in preview was to use the SharePoint REST API. Microsoft subsequently updated the SharePoint browser GUI to allow site owners to set a default sensitivity label for a document library.

Using the REST API still works, but my reader wanted something like a nice simple PowerShell cmdlet. Something like this would be nice:

Set-SpoSite -Identity $SiteURL -DocumentLibrary "Documents" -DefaultSensitivityLabel c29e68f9-bc4f-413b-a741-6db8e38ad1c6

The command would be nicer if you could pass the name of a sensitivity label, but the display names for sensitivity labels can be translated into multiple languages, which might cause some issues in multilingual tenants.

In any case, the Set-SPOSite cmdlet doesn’t support the functionality today and I haven’t heard of any plans to change in this area.

Reasonable to Allow Administrator Access to Some SharePoint Online User Data

I think it’s perfectly reasonable for SharePoint Online administrators to be able to update the default sensitivity labels for document libraries, especially because assigning a default sensitivity label incurs the requirement for Syntex-SharePoint advanced management licenses. An unwitting site owner could decide to assign a default sensitivity label to a document library (Figure 1) without realizing that the organization is now on the hook for some licenses, and that’s not a good thing. SharePoint administrators should be able to review, assign, and remove default sensitivity labels.

But this stance goes against the general approach Microsoft takes to SharePoint Online administration which holds that administrators can operate at the site level but cannot interact with objects within the site. Apparently, a site can have up to 255 document libraries, all of which are invisible to SharePoint administrators unless they’re a member of the site.

I understand the perspective that drives the approach. Administrators shouldn’t have access to user data. However, while Exchange Online administrators can see the folders inside user and shared mailboxes and Teams administrators can remove user data such as chat threads. It’s also possible for administrators to analyze and report the tasks in Planner plans. And sometimes even SharePoint Online administrators can take action with user data, like removing the sensitivity label for protected files using the Unlock-SPOSensitivityLabelEncryptedFile cmdlet. Inconsistency is rife across the Microsoft 365 workloads.

Greater Flexibility Required

I’m not advocating for SharePoint Online administrators to be able to open and examine documents and other files held in document libraries. The ability to report the contents of document libraries is already possible, albeit with some effort. What I would like to see is greater access to document library settings through PowerShell or a Graph API (which means that PowerShell support becomes available through the Microsoft Graph PowerShell SDK). For instance, why shouldn’t an administrator be able to do this to create a simple listing of all files found in the document libraries for a site:

$DocumentLibraries = Get-SpoSite -Identity $SiteUrl -DocumentLibraries
ForEach ($DL in $DocumentLibraries) {
   $Documents = Get-SPODocumentLibrary -Identity $DL 
   ForEach ($Doc in $Documents) {
    Write-Host (“Document found {0} in folder {1}” -f $Doc.Title, $Doc.Folder)
  }
}

SharePoint Online is not the center of its own universe as is the case with on-premises SharePoint Server. SharePoint Online is a highly capable document management service that’s consumed by other Microsoft 365 workloads. As such, its administrative capabilities should be on a par with other workloads, and that means greater flexibility and access to the settings for document libraries. Being able to report, configure, and remove the default sensitivity label for a document library is just the start.

Learn how to exploit the data available to Microsoft 365 tenant administrators through the Office 365 for IT Pros eBook. We love figuring out how things work.

Really Just Sending SharePoint News in an Email

Whover wrote MC671563 (29 Aug 2023) titled “SharePoint News in Outlook” needs some help composing headlines. Microsoft 365 roadmap item 124803 has nothing whatsoever to do with Outlook. Reading the headline, I anticipated something like a new OWA control (available also in the Monarch client, and for Outlook desktop via OPX) that allowed users to browse news items posted to their favorite SharePoint Online sites.

Instead, it’s simply a way to send news items from SharePoint Online via email to allow recipients to read the news using whatever email client they like. Although sending news via email is functional, it’s a bit of a damp squib when you consider that people have exchanged news via email since the dawn of messaging. Something more adventurous would have been nice.

Rollout to targeted release tenants has already happened. Standard release tenants will start to see the new feature in mid-September with full deployment due by late September 2023.

New Emailable News Templates

Essentially what’s happened is that SharePoint Online has six new templates to compose news items that are both posted to their host SharePoint site and emailed (Figure 1).

The templates intended for both posting and email support a limited set of web parts. With that exception, creating a new item is as before (Figure 2).

SharePoint News in Outlook Messages

After the content is ready, click Post and send. SharePoint posts the item to the site and displays a screen to allow the user to add the email addresses to receive the post (Figure 3).

The message that arrives in a user inbox gives the recipient the option to read the information in their favorite email client or in SharePoint (Figure 4). The link to SharePoint Online only works if the recipient can access the host site.

The mechanism used by SharePoint Online is rather like the Teams Share to Outlook feature and shouldn’t cause anyone to kill too many brain cells to master the feature. Some points worth noting are:

• To make sure that the information stays within the tenant, SharePoint Online doesn’t allow external addresses to receive the post. All addresses added to the message must belong to the tenant. The set of valid addresses includes user accounts, Microsoft 365 groups, and distribution lists.
• The feature connects to the mailbox of the author of the news item and creates and sends the message from there (you can do the same thing using Graph APIs or the Graph SDK). A copy of the outbound message is in the Sent Items folder. Using this mechanism ensures that the message travels through the Exchange Online transport pipeline. Exchange Online can then apply any transport rules or DLP policies that match the message. The full path of the message is available through message trace, including any transport events that happen such as the application of transport rules.

One exception exists to the rule that limits transmission to internal recipients. If you operate in a Microsoft 365 multi-tenant organization (MTO), user accounts from other tenants in the MTO synchronize to your tenant as member accounts. SharePoint Online allows news items to be emailed to MTO synchronized accounts from other tenants. It might be that the SharePoint developers decided to support MTO accounts because they are deemed trustworthy because they come from a tenant that has a cross-tenant synchronization arrangement with your tenant. Or they simply didn’t realize that MTO accounts exist. I fear that the latter is the true reason.

Analytics for SharePoint News in Outlook

Page analytics are available for each news item. Microsoft says that the analytics reflect total page reads sourced from SharePoint Online and Outlook (email). News sent by email can be reported in terms of page views but SharePoint can’t capture how long people spend reading news items received by email.

A Feature Seeking a Problem

As I played with sending SharePoint news items via email, the question crossed my mind about what demand exists for such functionality. It’s easy to copy and paste interesting news snippets into regular email if you want to. No analytics are available, but again you wonder if this is important. Perhaps organizations exist that place great importance on SharePoint news items and insist on the ability to email the latest information. If so, I haven’t met them.

Microsoft’s blog on the topic isn’t particularly illuminating until you read the comments from real people who know more about SharePoint news than I do. Those comments are worth reviewing before you decide to dedicate any effort to deploying this feature.

Learn about using SharePoint Online and the rest of Office 365 by subscribing to the Office 365 for IT Pros eBook. Use our experience to understand what’s important and how best to protect your tenant.

Stop Non-Group Members Accessing Restricted Sites

For years, I have been under the impression that SharePoint Online imposed access control for sites connected to Microsoft 365 through group membership. But then MC671823 came along on 29 August 2023 on the topic of Restricted Access Control for SharePoint and OneDrive Sites. According to Microsoft 365 roadmap item 163991, the deployment date is December 2023, but MC671823 says that the feature is now rolling out through both the SharePoint Online admin center and PowerShell. Confusion reigns.

Microsoft says that the feature allows administrators to “restrict site access to specified users using Microsoft 365 group or AAD security groups. Users not added in the specified group(s) will not be able to access the site even if they were previously granted site access.” The idea is “to reduce the risk of oversharing or permission sprawl within their organizations.” Both are good aspects to restrict, especially for organizations considering the implementation of Microsoft 365 Copilot, where the danger exists that Copilot might use sensitive information in its responses.

I think the basic thinking behind restricted sites is that it’s possible for site administrators to add extra users to sites that might expose confidential information to those people. By restricting access to known groups, you remove the risk. Of course, there’s nothing to stop a administrator for a site connected to a Microsoft 365 group adding someone to the group membership (by definition, they’re a group owner), but then everyone else in the group can see that a new member is present and could ask some awkward questions.

Configuring Restricted Sites

The first thing to do is to configure the tenant to support restricted sites. Download the latest version of the Microsoft.Online.SharePoint.PowerShell module from the PowerShell gallery (or use our script to update your Office 365 modules). I used version 16.0.24009.12000. Connect to the module and run:

Set-SPOTenant -EnableRestrictedAccessControl $True

If you don’t take this step, you won’t be able to configure restricted access for individual sites using either the SharePoint Online admin center or PowerShell. Like any setting applied to a SharePoint Online tenant, wait for a couple of hours to allow the change to take effect. You can then update individual sites. For instance, to update a site with PowerShell, run the Set-SPOSite cmdlet:

Set-SPOSite -Identity "https://office365itpros.sharepoint.com/sites/ProjectHiddenSecret" -RestrictedAccessControl $True

Restricted access control has been enabled on the site https://office365itpros.sharepoint.com/sites/ProjectHiddenSecret. The site access is restricted to members of the group b248090e-2bca-4d14-8aa6-3969a157a2a6.

Get-SPOSite -Identity "https://office365itpros.sharepoint.com/sites/ProjectHiddenSecret"  | Format-List Restrictedaccess*

RestrictedAccessControl       : True
RestrictedAccessControlGroups : {b248090e-2bca-4d14-8aa6-3969a157a2a6}

The GUID (b248090e-2bca-4d14-8aa6-3969a157a2a6) returned by the Get-SPOSite cmdlet is the group identifier pointing to the group SharePoint Online uses to control restricted access. Because this is a site connected to a Microsoft 365 group, the GUID resolves to that group:

Get-MgGroup -GroupId b248090e-2bca-4d14-8aa6-3969a157a2a6

DisplayName           Id                                   MailNickname        Description                      GroupTy
                                                                                                                pes
-----------           --                                   ------------        -----------                      -------
Project Hidden Secret b248090e-2bca-4d14-8aa6-3969a157a2a6 ProjectHiddenSecret A project full of hidden secrets {Uni...

Alternatively, edit the site settings to enable restricted site access (Figure 1).

Restricted Sites Not Connected to Microsoft 365 Groups

An extra step is needed to configure sites that aren’t connected to Microsoft 365 groups. In this case, you must specify the identifier for one or more (up to 10) comma-separated Entra ID security groups to use for access control. Dynamic security groups are supported. Here’s an example:

Set-SPOSite -Identity "https://office365itpros.sharepoint.com/sites/TestSite001" -RestrictedAccessControl $True -RestrictedAccessControlGroups d347eec5-62f1-4436-af41-e53fa18090be

Restricted access control has been enabled on the site https://office365itpros.sharepoint.com/sites/TestSite001. The site access is restricted to members of the group d347eec5-62f1-4436-af41-e53fa18090be.

Finding Restricted Sites

To find sites with restricted access, you must scan each site and then resolve the group identifiers using code like this:

[array]$Sites = Get-SPOSite -Limit All
ForEach ($Site in $Sites) {
    $SiteDetails = Get-SPOSite -Identity $Site.Url
    If ($SiteDetails.RestrictedAccessControl -eq $True) {
       [array]$Groups = $SiteDetails.RestrictedAccessControlGroups
       ForEach ($G in $Groups) {
         $Group = Get-MgGroup -GroupId $G
         Write-Host ("Site {0} owned by group {1}" -f $SiteDetails.Title, $Group.displayname) 
       }
    }
}

Licensing Restricted Sites

Making a SharePoint Online site subject to restricted access requires the Syntex-SharePoint advanced management license. At least, that’s what we learn from Microsoft’s documentation, which says that “some features” require the license without offering any further guidance. My assumption is that any user accessing a restricted site needs the license.

Another Layer of Protection

Restricted sites add another security layer to protect confidential information stored in SharePoint Online. If you pay for the advanced management license, you can also assign the block download policy to sites to stop site members downloading files from the site. Add sensitivity labels to block access unless people have the right to open files, and you’ve got a nice set of protections to prevent unauthorized access to information.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

FileRecycled Audit Event Replaces FileDeleted

In December 2021, I wrote about using events captured in the unified audit log to analyze file deletion activity in SharePoint Online and OneDrive for Business. Recently, some readers complained that the script (available from GitHub) wasn’t finding events for file deletions. A major advantage of PowerShell is that you see all the code and can modify the code to meet your needs. This also means that you can debug the code. My usual response to people who report problems with scripts is to prompt them to do some basic debugging by running the code line-by-line until the problem becomes apparent. Apart from learning how the script works, debugging is a great way to improve PowerShell skills.

In any case, a quick check revealed the problem. Microsoft changed the name of the operation captured in file deletion audit events from FileDeleted to FileRecycled. The change seems to have come into force in March 2023. At least, that’s the date of the first FileRecycled audit event generated by SharePoint Online I can find in my tenant. Microsoft didn’t say anything about the change. It just happened without warning.

File Deletion or Recycling

A case can be argued that FileRecycled is a more accurate description of the action than FileDeleted is (see this documentation update request from August 2022). SharePoint Online doesn’t actually delete an item until it goes through the two-stage recycle bin and exceeds the 93-day retention period that items remain in the recycle bin. The initial action is to move an item from a document library to the site recycle bin, hence the justification to use the FileRecycled name in audit records.

I wouldn’t have a problem if Microsoft told people about the change. Not everyone scans the documentation to detect name changes for audit log activities. Unless you checked the data returned by the Search-UnifiedAuditLog cmdlet or noticed the details for file deletions (or rather “recycled file”) operations returned by the audit log search in the Purview compliance portal (Figure 1), the change would probably have escaped undetected.

Microsoft also changed the UI of the audit search solution so that if you select “Deleted file” from the list of activities to search for, you’ll find events logged when SharePoint Online removes files from the recycle bin.

The Impact of Unannounced Changes

The problem here is that when Microsoft makes unannounced changes to audit data, it potentially affects scripts written by organizations to move data from the audit log to an external repository like Splunk. Among the reasons why organizations populate external repositories with audit data are:

• Long-term retention of audit data. Until recently, Microsoft only kept audit data for 90 days. On July 19, 2023, Microsoft announced a doubling of the audit data retention period to 180 days for Audit standard (Office 365 E3) customers. Audit premium customers have a 365-day audit data retention period with an optional add-on license available to increase the period to 10 years.
• Better search and investigation facilities. Although organizations have built tools to interrogate the unified audit log, the fact remains that the contents of audit log entries often need processing to extract useful information (like this example of extracting information about changes made to Entra ID account properties).

Obviously, if a new name is introduced for a common auditable activity like file deletion, it’s likely that processes to export audit data will ignore these events. I haven’t found any other activity renames but suspect that some might be lurking in the audit log.

Updates without Warning Reduce Confidence

The bottom line is that reliable audit data is an important part of a compliance ecosystem. If audit data is missing or becomes difficult to interrogate, those who work with audit data lose a little faith because it isn’t as comprehensive and accurate as they expect. And that’s a great pity.

Making Sharing of Files and Folders Easier

Apart from Microsoft 365 roadmap item 124933, I can’t find a formal announcement about the Simplified Sharing Experience, but I have been aware that Microsoft recently updated the share dialog used by Microsoft 365 apps to make it easier and more straightforward to use. According to a LinkedIn post, (Figure 1) Microsoft ran an A/B experiment to test the new dialog. I guess I was one of the testers! In any case, the new sharing dialog is now available in all Microsoft 365 tenants. Users of OneDrive consumer will see the upgraded dialog in the second half of 2023.

The Role of the Share Dialog

The share dialog is what people see when they share a document or folder with others inside or outside their organization. According to Microsoft, the dialog is used over 800 million times monthly across 52 different Microsoft 365 experiences (desktop, browser, and mobile). In other words, Microsoft 365 apps offer users the opportunity to share in 52 different places across the suite. The most common of the experiences are likely in SharePoint Online, OneDrive for Windows, and Teams.

Microsoft says that they focused on creating a dialog that makes it simpler for users to perform core sharing tasks. When someone invokes the new screen (Figure 2) to share a file or folder, they see a simpler layout pre-populated with the default sharing link as specified by the tenant or site policy (in this case, the sharing link allows access to people within the organization). The name of the sensitivity label assigned to the document is also shown to provide a visual indicator about its relative confidentiality.

To complete the link, add the people to notify and enter a note to tell them what to do, and click Send to have the message sent by email or Copy link to copy the sharing link to the clipboard.

If you need to change the type of sharing link, select the cogwheel to expose the link settings (Figure 3). Again, everything is very straightforward and simple. If you choose a link that allows external sharing, I’m told that the new design “makes users more comfortable with sharing.” I’m not quite sure what this means, but any of the sharing that I’ve done with people outside the organization has worked smoothly.

Microsoft has also overhauled the Manage access dialog to help people manage the set of users and groups that have access to a file or folder (Figure 4).

Microsoft says that customer feedback about the new dialog is very positive. It’s worth noting that this is not the first time that Microsoft has revamped the sharing dialog. The last major overhaul was in 2020-21 when Microsoft rationalized on a common sharing dialog for all apps, notably for Teams.

The Importance of Sharing

Getting sharing right is clearly important. When Microsoft launched the Delve app in 2015, it resulted in a crescendo of protest from tenants who suddenly found that Delve suggested documents to users when the organization thought that Delve should not. Of course, the software did nothing wrong. Delve respected the access rights given to users when it computed the set of interesting documents to suggest (using an early version of Graph document insights). The problem was entirely down to poor management and access control, often at the level of complete SharePoint Online sites. Users might not have realized that they had access to the documents in poorly-protected sites, but software can’t be blamed if it goes looking for documents to suggest to a user and finds some that are available.

We’re heading for a similar situation with Microsoft 365 Copilot. The Copilot software depends on finding information with Graph queries to help satisfy user prompts. Like Delve, Copilot will find files that are available to the user who prompts for help, and the results generated for the user might include some confidential. And if the user doesn’t bother to check the content generated by Copilot, the information might then be revealed with people who shouldn’t have it. This is the danger of oversharing, and it’s certainly an issue for organizations contemplating Microsoft 365 Copilot need to resolve before implementation.

Simplified Sharing Experience One Step Along the Path

The new sharing dialog won’t solve oversharing. It’s just one step along the path to help users share information with the right people in the right way.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Clipchamp for Work, but Only for Microsoft 365 Commercial Customers

In a July 31 announcement, Microsoft says that they will make Clipchamp for Work available to Microsoft 365 commercial customers through the targeted release program in the coming weeks. The news is covered by Microsoft 365 roadmap item 124826.

Although the advent of Clipchamp for Work is good news, it’s tempered by Microsoft’s statement that

“Clipchamp will be added to the following Microsoft 365 SKUs: Microsoft 365 E3, Microsoft 365 E5, Microsoft 365 Business Standard, and Microsoft Business Premium. Clipchamp will not be added to Office 365 SKUs.”

Ignoring the Office 365 enterprise SKUs is part of Microsoft’s tactics to force customers to upgrade to Microsoft 365 SKUs to drive the average revenue per user (ARPU) and increase the profits flowing from its cloud business past the $111.6 billion mark announced for Microsoft’s 2023 fiscal year.

The same method is being used with Microsoft 365 for Copilot, which only supports the Microsoft 365 E3 and E5 enterprise SKUs. Moving from Office 365 E3 to Microsoft 365 E3 costs an extra $13/user/month. Although a case can be argued that features like Windows 11 Enterprise and Enterprise Mobility and Security which are bundled into the Microsoft 365 enterprise SKUs represent good value for the extra cost, the fact remains that some customers don’t want to be forced to upgrade. Adding a very capable video editor to the Microsoft 365 enterprise SKUs probably won’t make any difference when it comes to deciding whether to upgrade, but it is one more factor to consider.

Adding on to Stream

Making Clipchamp for Work a new Microsoft 365 service plan makes sense. Like Stream for SharePoint, Clipchamp for Work stores its videos in SharePoint Online and OneDrive for Business, so it’s very different to the Clipchamp consumer version. Microsoft won’t provide a migration mechanism to move videos from the consumer to commercial version.

Stream for SharePoint added features steadily since its release, recently adding inline playback in Teams, automatic transcript generation, and a teleprompter when recording videos. However, Stream for SharePoint lags behind Stream classic in terms of video editing capabilities. Stream classic never offered much, but at least it could trim some excess from the start and end of videos.

Clipchamp fills the void and adds a lot more functionality besides such as a gap remover (merge videos together seamlessly) and a video cropper. Microsoft also promises that “Filters, effects, and text overlays give your editing a professional and personalized look. With the green screen filter in Clipchamp, you can adjust your backgrounds to suit your professional context and environment. Transitions can easily be added as a finishing touch to give your video that high-quality look.”

Clipchamp for Work includes an in-browser video editor. You’ll be able to create a new video by launching Clipchamp from the Microsoft 365 app launcher or from OneDrive for Business. Clipchamp can edit videos stored in OneDrive for Business and SharePoint Online, including Teams meeting recordings. The point about using SharePoint storage for videos is to take advantage of integration with the rest of the Microsoft 365 suite such as sharing, compliance, data lifecycle management (retention), and information protection.

Clipchamp Brings Proper Video Editing to Microsoft 365

When Microsoft bought Clipchamp in September 2021, the speculation was that Microsoft would add Clipchamp quickly to Office 365. Well, that hasn’t happened. Apart from needing to get used to the Microsoft 365 ecosystem so that single sign-on works and Clipchamp can process videos stored in SharePoint Online, the delay in reaching this point is likely due to waiting for Stream on SharePoint to mature and the migration from Stream classic to complete, which it now mostly is.

I’m glad to see a proper video editor in Microsoft 365. Video communications are becoming more important all the time. It’s just a pity that Microsoft won’t make the Clipchamp functionality available to their loyal Office 365 enterprise customers. We’ll just have to keep on using Camtasia instead.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across Office 365. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.

More Details Revealed About Microsoft 365 Backup and Microsoft 365 Archive During Inspire session

A week or so after the launch of Microsoft 365 Backup and Microsoft 365 Archive at their annual Inspire conference (for partners), I took the time to listen to the recording of the session covering the topic delivered at the event. It’s hard to get much detail from a 14-minute session after filtering out all the marketing messages delivered by the avuncular Chris McNulty, but I found some interesting points to ponder.

As a reminder, these products are not yet available. They might be toward the end of 2023. Then again, product dates have an unfortunate habit of slipping, especially when they’re for solutions in a new area. This is doubly true when dealing with solutions targeted at backup and restore and touted as a great solution for ransomware because of their “unprecedented speed and scale.”

McNulty started with some statistics:

• Microsoft 365 users add two billion documents and emails daily. I assume this figure includes Office documents, PDFs, Loops, OneNote notebooks, emails, Teams messages, and everything else that can be stuffed into SharePoint Online, OneDrive for Business, and Exchange Online. In September 2022, Microsoft said that Exchange Online processes 9.2 billion messages daily, 2.4 billion of which are spam. However, it’s unclear if these figures include system messages that are transient and not stored.
• Microsoft 365 user activity consumes 200 petabytes of storage monthly. Much of the data is unstructured. I assume that imports from SharePoint Server and other non-Microsoft 365 sources consume some of this storage. While providing such a large amount of storage is a heavy expense for Microsoft, its existence inside Microsoft 365 creates opportunities. For instance, it is the raw material for Microsoft 365 Copilot.

Microsoft also said that the estimated annual cost of ransomware is $20 billion (2021). They also noted a 74% increase in password attacks in one year, which is yet another good reason for Microsoft 365 tenants to make better use of multi-factor authentication even if attacker tactics like password sprays are less effective due to the removal of basic authentication.

Microsoft 365 Backup

The basic value proposition for Microsoft 365 Backup is simple: the ability to backup and restore data more rapidly than any other backup solution. This is because the data remains within Microsoft 365 and therefore doesn’t have to be copied across an internet connection. Partners have access to the Microsoft APIs for backup, restore, and archiving to allow them to integrate Microsoft 365 in their solutions. In this context, Microsoft will take care of the background processing and the partner looks after the user interface and integration with backup and restore solutions that handle other non-Microsoft workloads to create a single pane for all backup and restore operations.

Of course, keeping backups of your SharePoint Online, OneDrive for Business, and Exchange Online data within the Microsoft trust (security) boundary is a double-edged sword. Keeping all your data eggs in the one Microsoft basket is convenient, enables fast restore, and easy to use because operations are integrated in the Microsoft 365 admin center.

Jacklynn Hiranaka’s demonstration of backup and restore showed how easy it is to configure full backup for a tenant (Figure 1). She made the point that once backup is enabled, it becomes effective immediately. This is likely because Microsoft can utilize techniques like capturing SharePoint changes in the Preservation Hold Library or Exchange changes in Recoverable Items to generate backup items. You can imagine how restores operate like a supercharged version of the SharePoint Restore this library feature or Exchange’s Recover deleted items.

Even more impressive was the assertion that Microsoft 365 Backup can perform parallel restores for SharePoint Online, OneDrive for Business, and Exchange Online to restore information very quickly.

Microsoft 365 Archive

Brad Gussin covered details of Microsoft 365 Archive. This is a SharePoint Online option (Exchange Online has its own archiving). You can already archive Teams and put the associated SharePoint Online sites into a read-only mode. Microsoft 365 Archive puts inactive SharePoint sites into a state where administrators can still manage the sites (to bring them back into an active state) but the data is no longer “hot” (available for immediate user access).

The major advantage gained by moving sites to an archived state is that the storage they consume is no longer charged against the tenant’s SharePoint storage quota. The data is still in SharePoint, but just like the storage consumed by Syntex Repository Services to hold Loop app data, it’s not accessible in the normal way.

Administrators will be able to search for inactive sites and decide which sites to archive. Site owners can protest this action and negotiate with administrators to keep their sites online. Once the final decision to archive, the process to archive sites takes a couple of hours. Actions to archive or reactivate sites are available through the SharePoint Online admin center (Figure 2) or PowerShell. Microsoft hasn’t specified how the PowerShell option will work, but it could be through an updated Set-SPOSite cmdlet or perhaps dedicated cmdlets to archive and reactivate sites. Long-term, Microsoft plans to enable finer granularity by supporting archival at the file level.

Microsoft 365 features such as data loss prevention, data lifecycle management (retention processing), information protection, and search remain in place for archived sites. eDiscovery can find items in archived sites (using the search indexes) and retrieve items using search exports.

A cynic might say that Microsoft created the need for an archive solution by restricting the amount of storage made available to tenants (1 TB plus 10 GB per eligible license) and the way that retention processing consumes quota. The more intelligent versioning planned for document libraries might help restrain storage consumption, but overall it’s still true that SharePoint Online storage is expensive when compared to the abundant storage made available to OneDrive for Business accounts.

No Pricing Available

Microsoft hasn’t revealed how much Microsoft 365 Backup and Microsoft Archive will cost. I’ve been surprised by some recent Microsoft pricing decisions (like the $7/user/month demanded for slightly more intelligent Entra ID access reviews). The good thing is that backup for Microsoft 365 is a competitive market. Microsoft has some strong advantages, but if it goes too far in terms of inflated pricing, customers will vote with their wallets and go elsewhere.

Learn about using SharePoint Online, Exchange Online, and the rest of Office 365 by subscribing to the Office 365 for IT Pros eBook. Use our experience to understand what’s important and how best to protect your tenant.

Sensitivity Label PDF Support Increases Coverage for Protection

In my review of sensitivity labels for 2023, I noted that the only way to apply a sensitivity label direct to a PDF was with:

• The paid-for versions of Adobe Acrobat.
• Generating PDFs from Office documents (subscription apps only).
• Applying a label through the unified labeling client.

Unlike retention labels, it wasn’t possible to apply a sensitivity label to a PDF using the SharePoint Online browser client. Now it is, and it’s an important update given the widespread use of PDFs within Microsoft 365. Between Office documents and PDFs, sensitivity labels can now protect over 90% (my estimate) of all files stored in SharePoint Online and OneDrive for Business. It’s another step to making PDFs a fully functional format within the Microsoft Information Protection ecosystem.

What Sensitivity Label PDF Support Means for SharePoint Online

In an update announced by principal program manager Sanjoyan Mustafi on LinkedIn, the preview of SharePoint Online support for PDFs is available to all commercial tenants worldwide. Support extends to sensitivity labels with predefined permissions. Labels with user-defined permissions or those that use Double Key Encryption (DKE) are unsupported.

Supporting sensitivity labels for PDFs means that people can use SharePoint Online and OneDrive for Business to:

• Apply sensitivity labels to PDFs through the browser interface (Figure 1) and amend or remove the label afterwards, including forcing the user to provide justification if required by policy. This includes applying the default sensitivity label defined for a document library to PDFs as users load them into the library (requires the SharePoint-Syntex advanced management license).
• Apply sensitivity labels to PDFs stored in SharePoint Online and OneDrive for Business through auto-label policies. This feature is covered in message center MC644060 (14 July, 2023).
• Apply sensitivity labels to PDFs using the assignSensitivityLabel Graph API (if your app has permission to do so).
• Display the names of sensitivity labels for protected PDFs in document libraries.
• Index the content of PDFs protected by sensitivity labels. This supports Microsoft Purview solutions like Data Loss Prevention, content searches, and eDiscovery.

Like Office documents protected by a sensitivity label with encryption, SharePoint Online can’t display a thumbnail of a protected PDF (Figure 2). I believe that this has something to do with the inability to fetch the necessary use license to decrypt the file. Thumbnails are shown for PDFs assigned a sensitivity label with no encryption. To open a document, use the Edge browser (which supports reading protected files) or download the file and use an app that understands how to open protected PDFs (like Acrobat).

I hear that Microsoft is working on the viewing issue and expects to have a fix by the end of 2023.

Enabling Sensitivity Label PDF Support for SharePoint Online

By default, SharePoint Online support for PDFs is disabled. To enable support, load the SharePoint Online administration PowerShell module and run the Set-SPOTenant cmdlet. You’ll need a recent version of the module (use this script to update your Microsoft 365 modules to the latest version):

Set-SPOTenant -EnableSensitivityLabelforPDF $True

To revert, run the command to update the setting to $False.

Set-SPOTenant -EnableSensitivityLabelforPDF $False

Disabling SharePoint support for PDFs has no effect on PDFs with sensitivity labels. It will stop users being able to assign or update labels through the SharePoint Online and OneDrive for Business browser interfaces and SharePoint Online will cease indexing protected PDF content.

If you don’t want to use PowerShell, check the Information protection section of the Purview compliance portal, and go to Auto-labeling. You might see a message inviting you to turn on support for PDFs. If you do, select Turn on now and the job is done.

More information about PDF support for sensitivity labels in SharePoint Online is available in Microsoft documentation.

Sensitivity Label PDF Support is an Important Step Forward

I don’t think it is an exaggeration to say that some organizations have been waiting years for PDF support to arrive in SharePoint Online. Given the widespread use of PDFs in many organizations, this is an important step forward for those wishing to protect their most sensitive information stored in SharePoint Online and OneDrive for Business.

Support the work of the Office 365 for IT Pros team by subscribing to the Office 365 for IT Pros eBook. Your support pays for the time we need to track, analyze, and document the changing world of Microsoft 365 and Office 365.

SharePoint Intelligent Versioning Based on Usage Coming Soon

Update: 14 June, 2024: The SharePoint version history feature is rolling out in preview.

In a recent article describing changes SharePoint Online made to how it stores retained files in the Preservation Hold Library, I mentioned the effect of retention on SharePoint storage and hoped that the changes would reduce this impact. Now it appears that Microsoft plans further changes to help.

Microsoft 365 roadmap item 145802 posted on June 30, 2023 discusses proposed changes for version history limits in SharePoint Online document libraries. Today, SharePoint Online requires versioning to be enabled for document libraries and lists and uses the following values to control major versions:

• Default: 500 versions.
• Minimum: 100 versions.
• Maximum: 50,000 versions.

Versioning is a critical feature for SharePoint Online. It underpins capabilities such as being able to restore a document library to a point in time, the autosave feature used by the Office apps to make sure that people don’t lose work, and updating of local file copies by the OneDrive sync client. It’s easy to accumulate a large number of versions. For instance, the PowerShell chapter for the Office 365 for IT Pros (2024 edition) eBook already has 81 versions (Figure 1) since its creation in early May. A single editing session to create this article created seven versions.

For these and other reasons, SharePoint Online sets the default number of versions to 500. It’s therefore not a good idea to reduce the number of versions for a document library by editing the number in Versioning Settings (Figure 2). On the other hand, increasing the number of versions retained by a document library can increase the storage consumed by the Preservation Hold Library.

Moving to Automatic Versioning Management

Microsoft says that they plan to “increase version history limits” for SharePoint Online and OneDrive for Business document libraries. Site administrators will be able to choose two types of version limits:

• Automatic mode: SharePoint Online “intelligently” adjusts the versions kept for files based on age and the probability that a version will be required for a restore.
• Manual mode: Site administrators set version expiration and count limits for document libraries.

Tenant administrators will be able to set version limits that apply to newly created document libraries.

The big change is a movement away from simple count-based version limits (i.e., SharePoint Online keeps up to 500 versions of files) to a system where SharePoint Online manages version counts automatically depending on the usage of documents and the site.

Different Update Patterns

For instance, if people edit a document daily, it might generate ten versions every business day or fifty-plus versions a week. Under the present scheme, SharePoint Online begins to discard versions after ten weeks or so. The new mechanism might note the update pattern and decide that it should keep more than 500 versions to allow for a longer restore window than 10 weeks and remove versions after six months.

On the other hand, a static document that’s edited twice a year might have a much lower version count. And SharePoint Online might dynamically adjust the version count downwards after a document moves from the phase where people actively work on its content to when the file becomes stable and is no longer being actively edited.

All of this is speculation based on the description in the Microsoft 365 roadmap item. We won’t know the exact details about how automatic versioning management works until we see the new mechanism in practice. More will become known when the preview appears (currently scheduled for November 2023). General availability is scheduled for March 2024.

Update: According to the Microsoft 365 roadmap item, intelligent versioning won’t appear until August 2024.

The Manual Alternative

As noted above, if site administrators believe that a document library needs to use a specific version count (and a new expiration limit), they can opt for manual management instead of automatic versioning.

Intelligent Versioning Needed

Features like Autosave mean that SharePoint Online makes heavier use of versions than the on-premises servers. This factor plus (I assume) pressure on SharePoint storage means that it makes sense to employ a more intelligent management system for versions. No file is worked on in the same way, so taking usage into account seems like the right approach. We’ll see when the preview starts in November.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Preservation Hold Library Now Holds Files With Version History

During a recent content search for some documents, I noticed that the search found far fewer versions of SharePoint Online files than was the case in the past. This is the effect of the change introduced in mid-2022 (or rather, delayed until August 2022) documented in message center notification MC288633 (Microsoft 365 roadmap item 82062). I discussed the potential for the change when Microsoft first announced their intention to implement it in October 2021. Now it’s time to see how things work in practice.

In a nutshell, from the time Microsoft deployed the change to a tenant, SharePoint Online stops storing multiple copies of retained files in the Preservation Hold Library. Instead, SharePoint follows the same approach as taken for regular files stored in document libraries and stores a single file containing its full version history. Files end up in the Preservation Hold Library when SharePoint must retain them because of a retention policy applied to the site, a retention label applied to individual files, or eDiscovery holds. In all cases, files remain in the Preservation Hold Library until the hold applied by retention or eDiscovery lapses.

Site owners don’t have to create the Preservation Hold Library as SharePoint creates it automatically when required. To view Site items in the Preservation Hold Library, add /PreservationHoldLibrary to the site URL in the browser bar.

Difference in File Storage

Figure 1 shows how SharePoint used to store Office documents in the Preservation Hold Library. Each version is stored as a separate file with no versions. If the user updates the file, SharePoint creates a new version in the Preservation Hold Library to capture the changes and make them available for eDiscovery.

The example in Figure 1 comes from January 2022, before Microsoft deployed the change. Examining a more recent entry in the Preservation Hold Library, we see that different versions of the file are available (Figure 2).

It’s easy for changes like this to pass by without being noted, especially when Microsoft delays the deployment of an update for one reason or another (in this case, it was to give customers more time to prepare).

A practical effect of the change is that eDiscovery searches find the latest version of retained files. If investigators want to see previous versions, they can view or restore that version.

Storage Quotas

One of the issues with the way that SharePoint Online captures files for retention is the large amount of storage consumed for this purpose. Given that SharePoint storage is expensive, this is bad news.

In Figure 3, we see that SharePoint uses 20.6 GB or 21.87% of the overall site storage. Even though I know that this site holds many copies of large chapter and book files for the Office 365 for IT Pros eBook, 20.6 GB of retained content is quite a chunk.

The good news is that the advent of multi-version storage seems to have reduced the storage used by retention by a couple of percentage points over what it was in 2021. It’s probably too early to be definite on this point, but the signs are good. My expectations are that the overall storage used by the preservation hold library should reduce over time as older files reach the end of their retention period and SharePoint removes them from the library. We’ll see.

It’s taken me too long to comment on the Preservation Hold Library change. Sometimes life gets so busy that good changes go by ignored. This is a good change.

Support the work of the Office 365 for IT Pros team by subscribing to the Office 365 for IT Pros eBook. Your support pays for the time we need to track, analyze, and document the changing world of Microsoft 365 and Office 365.

Find SharePoint Documents to Decrypt Before Tenant Divestiture

A reader wanted to know the best way to find a bunch of files protected by a sensitivity label. The scenario is that the organization had divested an operating division. Sites used by that division had protected files that needed to be decrypted before they moved to a new tenant. If this failed to happen, the protected files would be inaccessible in the new tenant because the users signing into that tenant didn’t have the right to access their content. The question therefore is what’s the best way to find SharePoint documents protected by sensitivity labels so that administrators can remove the labels before the divestiture.

Office documents store label information in their file attributes, so the basic task is to search those attributes to find files protected with one or more specific labels. You could try and do the job with PowerShell and the Graph API. For instance, I have a script to report the files in a SharePoint document library, including the labels assigned to files. Another script uses the Unlock-SPOSensitivityLabelEncryptedFile cmdlet from the SharePoint Online management module to remove labels from documents. The two could be combined to find and remove labels from protected files.

The PowerShell approach is viable if the exercise spans several thousand documents in a few sites. Things become more problematic as the numbers scale up. For instance, sites with document libraries configured to apply default sensitivity labels to new documents (requires Office 365 E5 licenses) could accumulate thousands of protected documents in each library.

Using eDiscovery Searches to Find SharePoint Documents Protected by Sensitivity Labels

eDiscovery searches could solve the problem. Microsoft Purview eDiscovery (Premium) supports finding protected content. The documentation says that files “located on a SharePoint or OneDrive account are searchable and decrypted when the search results are prepared for preview, added to a review set in eDiscovery (Premium), and exported.” Figure 1 shows search preview displaying a protected document found by eDiscovery (Premium).

eDiscovery Premium can’t process documents protected by sensitivity labels with user-defined permissions (permissions assigned by the document author when they apply the label to the document) or when user access granted by the sensitivity label has an expiration date. In addition, eDiscovery Premium can’t decrypt files protected by the Azure Information Protection unified labeling client that are subsequently uploaded to SharePoint Online or OneDrive for Business.

Purview eDiscovery (Standard) and content searches can also find items protected with sensitivity labels. However, these solutions do not decrypt the content unless an unprotected document is an attachment for a protected email. That’s OK, because if you find and export the protected files, an Azure Information Protection (AIP) super-user can remove labels from files using the Set-AIPFileLabel cmdlet from the Azure Information Protection module. Although this is feasible, if you’re contemplating processing thousands of documents, I would buy some Office 365 E5 licenses and use Purview eDiscovery (Premium).

Configuring Content Searches to Find SharePoint Documents with Sensitivity Labels

To search for SharePoint files through Microsoft Search or a Purview content search, you use sensitivity label identifiers (GUIDs). The SharePoint Online search schema includes a managed property called InformationProtectionLabelId, which holds the GUID (identifier) for the sensitivity label assigned to a document. You can use this property to search for documents with a specific sensitivity label in SharePoint search or content searches by using the form InformationProtectionLabelId:GUID. For example, InformationProtectionLabelId:2fe7f66d-096a-469e-835f-595532b63560. The search results are trimmed and only display documents whoever performs the search can access.

An alternative approach is to remap the Sensitivity property, which stores the local language value of the label, to one of the 200 customizable RedefinableString managed properties available in SharePoint Online. This approach allows users to search using label names like “Public” and “Confidential,” but the downside is that it’s possible to assign multiple local language values for sensitivity label display names. If this happens, the searches would need to look for all defined values. By comparison, the identifier is unique and immutable, so using label identifiers is a better choice for search criteria.

To find the label identifiers, connect a PowerShell session to the compliance endpoint and run this command:

Get-Label | Format-Table ImmutableId, DisplayName

ImmutableId                          DisplayName
-----------                          -----------
2fe7f66d-096a-469e-835f-595532b63560 Public
8b652c9a-a8b7-40ec-bb1a-c5334b1b7fef No Encryption
a49e1277-93db-4a2f-8105-43c5196b4fef Non-business use
fb0975b2-1ea1-4c3c-850c-e859e690d282 Partner-Accessible Content
e42fd42e-7240-4df0-9d8f-d14658bcf7ce General Access

Now create a content search and input the label identifier into the search conditions, prefixed with InformationProtectionLabelId, just like shown in Figure 2:

To search for documents with different sensitivity labels, separate the label identifiers with OR. For example, here’s the Keyword Query Language (KQL) query to find documents with either of two labels created between 19 May 2023 and 23 June 2023:

InformationProtectionLabelId:1b070e6f-4b3c-4534-95c4-08335a5ca610 OR InformationProtectionLabelId:2fe7f66d-096a-469e-835f-595532b63560(c:c)(date=2023-05-19..2023-06-23)

Dealing with Protected Content

Searching for protected files isn’t difficult. The real question is what you do with the files that the search uncovers. Having a bunch of encrypted files (with or without the new and improved encryption cipher) isn’t much good unless you can decrypt them. That’s where most of the problems lie, which is why Microsoft might have included the feature in Purview eDiscovery (premium).

Learn about using sensitivity labels, eDiscovery, and the rest of Office 365 by subscribing to the Office 365 for IT Pros eBook. Use our experience to understand what’s important and how best to protect your tenant.

OneDrive File Type Exclusions Control Synchronization for Sync Clients

Microsoft 365 message center notification MC597037 (updated June 27, 2023) brings news that the OneDrive sync client will display information about files blocked by synchronization by tenant administrators. Worldwide deployment of the updated sync client should finish by mid-July. In the past, users have been left in the dark when they discovered that some files wouldn’t synchronize, but now they can go to the Advanced settings section of the client to see what file types the tenant doesn’t allow them to synchronize.

Oddly, the description for Microsoft 365 roadmap item 124868 takes a different perspective and says:

“This feature will allow you to configure OneDrive Sync Setting to exclude selected files and selected file types from syncing to OneDrive. When available the configuration settings will be located in the OneDrive admin center.”

It seems like a little copy and pasting mistake because it’s long been possible for tenants to exclude file types from synchronization. Microsoft’s documentation explains how to achieve the goal using group policy. It’s also possible to impose a block by running the Set-SPOTenantSyncClientRestriction cmdlet from the SharePoint Online administration module. For example, this command blocks three file types:

Set-SPOTenantSyncClientRestriction  -ExcludedFileExtensions "mp4;rar;zip"

TenantRestrictionEnabled   : False
AllowedDomainList          : {}
BlockMacSync               : False
ExcludedFileExtensions     : {mp4, rar, zip}
OptOutOfGrooveBlock        : True
OptOutOfGrooveSoftBlock    : True
DisableReportProblemDialog : False

Running the Set-SPOTenantSyncClientRestriction cmdlet is the same as blocking file types through the Settings section of the SharePoint Online admin center (Figure 1). Both update the same configuration, which the OneDrive for Business sync client downloads and applies when it synchronizes files from the user’s OneDrive for Business account and whatever SharePoint Online document libraries are synchronized locally.

The Effect of OneDrive File Type Exclusions on Synchronization

Introducing a block on a file type isn’t something to do without thinking. After I ran the cmdlet to block the MP4 file type, my OneDrive for Business client complained bitterly because it could no longer synchronize any Teams meeting recordings and other videos stored in OneDrive (Figure 2).

Teams meeting recordings are possibly a bad example. According to Microsoft, few people go back and view a meeting recording after it is made, which is the reason why Teams applies an expiration tag to recordings after creating the files in OneDrive for Business or SharePoint Online. But I have many other MP4 files for which I want to keep a local copy, so maybe MP4 shouldn’t be on the file exclusion list.

It took the OneDrive for Business sync client several days to recover after updating the SharePoint policy to allow the synchronization of MP4 files, but eventually everything settled down and the client is now happy to process MP4 files again.

OneDrive File Type Exclusions for Personal Sync Client

In any case, file exclusions for OneDrive for Business are old news. What’s new is that Microsoft allows OneDrive Personal users to set their own exclusion list in the latest version of the client (I am using version 23.124.0613.0001). Because the client is for personal use, there’s no system-provided values. Instead, it’s up to the user to input the set of file types they want to exclude through the Advanced Settings section of the client (Figure 3).

File type exclusions are specific to a device rather than an account. You’ll find the information you enter in the client in a text file at

c:\users\<user>\AppData\Local\Microsoft\OneDrive\settings\Personal>odignore.txt.

If you run OneDrive Personal on multiple workstations, you’ll need to configure the settings on all workstations.

OneDrive FIle Type Exclusions are Client-Specific and Don’t Affect the Browser

Th summary is that both the OneDrive for Business and OneDrive Personal sync clients now display details about file types excluded from synchronization. OneDrive for Business users can’t do anything to affect the set of excluded files (except persuade an administrator to change the tenant configuration) while OneDrive Personal users can make their own minds up. In both cases, remember that these settings only affect the OneDrive sync clients. They have no effect on the OneDrive browser client, meaning that users can upload and download whatever OneDrive content they like using a browser.

Learn about using OneDrive for Business and the rest of Office 365 by subscribing to the Office 365 for IT Pros eBook. Use our experience to understand what’s important and how best to protect your tenant.

Managing OneDrive Storage Quotas Through Groups

A reader asked if it is possible to control the assignment of OneDrive for Business storage quotas using groups using a mechanism like group-based license management. The simple answer is that Microsoft 365 doesn’t support such a feature, but like many administrative operations, it’s relatively easy to automate with PowerShell.

Another article covers the basics of reporting and assigning OneDrive storage. OneDrive for Business accounts are personal SharePoint Online sites. Assigning a new storage quota to a user’s OneDrive account is done using the Set-SPOSite cmdlet from the SharePoint Online administration module. This is one of the Microsoft 365 modules that receives frequent updates, so make sure that you use the most recent version. It’s a good idea to check for updates monthly, either manually or using a PowerShell script to process the Microsoft 365 modules typically used by tenant administrators.

Creating a Script to Update OneDrive Storage Quotas

The steps required in the script to update OneDrive storage quotas based on group membership are:

• Connect to SharePoint Online and the Microsoft Graph PowerShell SDK.
• Read information about the target OneDrive storage allocations from some source. I used a CSV file with columns for the group name, group identifier, and storage allocation in megbytes, The names of the columns are group, groupid, and allocation.
• Figure out the service domain for the tenant to calculate the root of OneDrive account URLs. This will be something like: https://office365itpros-my.sharepoint.com/personal/. Later, we combine a modified version of user principal names (replacing dot and @ characters with underscores) to form the URL for each account. An example is https://office365itpros-my.sharepoint.com/personal/James_Ryan_office365itpros_com.
• For each group, get the group members. For each member, figure out the user’s OneDrive account URL and run the Get-SPOSite cmdlet to check its current storage quota. You can use any of the group types supported by Entra ID including dynamic Microsoft 365 groups. With some adjustments to the code, it would also be possible to use an Exchange Online dynamic distribution list.
• If the assigned quota is less than the desired quota, run the Set-SPOSite cmdlet to increase the quota.
• Create a report about what happened (Figure 1).

The script includes nothing complicated in terms of code. You can download the script I wrote from GitHub. Remember that the script is not bulletproof in terms of error handling. Its intention is to prove the principle of what is possible. The script should run without a problem if you sign in with a tenant administrator account. I have not tested the code in an Azure Automation runbook (to run the script on a schedule), but I think that adapting the code for Azure Automation would not be difficult.

Use Azure AD Administrative Units Instead of Groups

Azure AD administrative units are the current flavor of the month in Microsoft Purview with many solutions, including Data loss prevention (DLP) and Data lifecycle management (retention) supporting the use of administrative units to scope policies. If you have the necessary Azure AD Premium licenses, you could use administrative units as the basis for storage assignment.

This article explains how to use PowerShell to retrieve information from administrative units. Instead of fetching a set of user principal names for group members, you’d fetch the same information for the members of an administrative unit, like this:

[array]$GroupMemberUPN = (Get-MgBetaAdministrativeUnitMember -AdministrativeUnitId 150dccad-f8b8-4e54-9246-89834b8b5a25).AdditionalProperties.userPrincipalName

PowerShell Automation Scores Again

It would be nice if Microsoft included group-based OneDrive storage management in SharePoint Online. However, this functionality is probably not high on their priority list for new development. This is yet another example of how PowerShell fills in the cracks and gaps left in Microsoft 365 management and underscores why tenant administrators should have the ability to perform at least simple tasks with PowerShell.

Support the work of the Office 365 for IT Pros team by subscribing to the Office 365 for IT Pros eBook. Your support pays for the time we need to track, analyze, and document the changing world of Microsoft 365 and Office 365.

SharePoint Embedded is a Special Form of SharePoint Online

Updated 18 March 2024

Last March, I wrote about the preview of the much-awaited Loop app, and noted that the objects generated by the Loop app “are stored in hidden SharePoint Online containers with names like https://office365itpros.sharepoint.com/contentstorage/CSP_GUID.” By comparison, the content of Loop components created in apps like Teams chat and Outlook are stored in .loop files in the author’s OneDrive for Business account.

At the time, Microsoft pointed to the support SharePoint Online gave to the Loop app but were non-specific about any detail. Microsoft revealed more information at the BUILD conference, where they said that the Loop app uses Syntex repository services, described as “the fastest way for developers to build and manage file and document centric apps that leverage the rich content platform services of Microsoft 365.” Syntex repository services are in private preview at present. Subsequently, Microsoft renamed Syntex Repository Services to be SharePoint Embedded.

Powered by SharePoint

The developer blog post on the topic gives more information. It seems like Syntex repository services (“powered by SharePoint”) is a document management services Microsoft sells to app developers. Apps developed on the platform store their files in “repository containers,” a SharePoint Online storage partition within a customer tenant (you could think of OneDrive for Business as a SharePoint partition). Repository containers created by apps remain under the control of the app that create containers and cannot be accessed through regular Microsoft 365 interfaces, including administrative interfaces like the SharePoint Online admin center. Apps access their files through Microsoft Graph APIs and present the information through their own GUIs.

The big advantage put forward by Microsoft is that by managing their files through Syntex, app developers get access to Microsoft 365 capabilities like sensitivity labels and eDiscovery in addition to the scalability of the Microsoft cloud platform. In other words, don’t worry about storing files, compliance, and all that stuff because Syntex will do that for you – focus on delivering app functionality to end users. It’s a good example of creating new capabilities off a proven platform.

Loop and Syntex Repository Services

Coming back to Loop, Microsoft asserts that the Loop app is “100% powered by Syntex repository services.” What I thought were hidden SharePoint containers are repository containers that the Loop app accesses to store and manage its workspaces and pages. Microsoft noted that the Designer app also uses Syntex repository services (Figure 1).

We’re still in the early days and everything doesn’t work as promised. Loop is in preview and Microsoft 365 content searches cannot find its content, nor can retention policies target app repository containers. The Loop app doesn’t support assignment of sensitivity labels to workspaces or pages, and data loss prevention policies don’t intervene of someone shares a Loop page link outside the organization. All of this is expected when apps and underlying services are evolving.

SharePoint Embedded is an Interesting Choice for App Developers

If Microsoft delivers what it says will happen, app developers have an interesting choice to make. Should they create everything from scratch and have total control over a solution or use Syntex repository services and get a lot of off-the-shelf functionality. Cost will obviously be part of the equation. There’s no information available about who pays for Syntex repository services when it’s used to host app content.

Backup is another aspect to consider. Microsoft 365 tenants are accustomed to backing up documents and other elements stored in SharePoint Online. How will they backup documents and lists stored in app repository containers? Will backup vendors be able to read a list of app containers in a tenant and stream their contents out to a backup target? This is an example of an issue that needs better understanding before a new service becomes 100% operations-ready.

In passing, I note that the Syntex backup solutions for SharePoint Online, Exchange Online, and OneDrive for Business announced in November 2022 still haven’t made an appearance. Developing and bringing new cloud services into production is hard, even for Microsoft.

Learn more about how the Office 365 applications really work on an ongoing basis by subscribing to the Office 365 for IT Pros eBook. Our monthly updates keep subscribers informed about what’s important across the Office 365 ecosystem.

Similar but Different to Request Files in OneDrive for Business

In January 2020, I wrote about the feature that allows OneDrive for Business users to ask people to upload files to a folder. Time moves on and message center MC495329 (7 January 2023) announced the arrival of a similar feature for SharePoint Online document libraries. According to Microsoft 365 roadmap it 103625, rollout started in February. It’s taken a while for it to show up in my tenant, or maybe I just haven’t looked hard enough.

In any case, Microsoft says that the feature is “an easy and secure way to request and obtain files from anyone.” Essentially, you select a folder in a document library that you want to use as a target for uploads. You then create a request files link that you give to the people who have the information you want. For instance, these might be professional advisors working on some documents relating to a project. They use the link to upload the files to the target folder, which site members can then interact with as normal.

Any site member can generate a link by selecting the target folder and choosing the Request files option from the […] menu. SharePoint Online generates a link (Figure 1), which the user can share using whatever method they like.

People who upload files don’t have any visibility into site contents and can’t see the files once they upload them to the site. This is a one-way transmission.

Getting SharePoint Online Ready for Request Files

The support documentation for the Request Files feature is available online. I don’t intend to repeat it here. However, some points from the feature documentation deserve emphasis.

First, the Request Files feature depends on Anyone sharing links. If your tenant doesn’t allow people to create Anyone links, they won’t be able to request external people to upload files to a folder. The permissions allowed for the link must include upload rather than just view and edit.

Second, Microsoft checks if Anyone links are enabled in a tenant when they deploy the software update for the Request files feature. If the tenant allows Anyone links, Microsoft enables all sites to support the feature. Originally, my tenant blocked Anyone links, which meant that the default condition applied (disabled) for all sites. After enabling Anyone links, I had to explicitly enable Request files for sites to make the option available.

Other restrictions can interfere with the ability of users to create Request Files links. For example, if you apply the file download block policy to a site, the option to request a link is unavailable.

Apart from enabling Anyone links (through the SharePoint Online admin center), control over how the Request files work is via PowerShell. The Set-SPOTenant cmdlet enables or disables the feature across the entire tenant. This command makes sure that the feature is enabled for the tenant and sets the expiration for request files links to seven days:

Set-SPOTenant -CoreRequestFilesLinkEnabled $False -CoreRequestFilesLinkExpirationInDays 7

While this command disables the feature for a specific site:

$SiteURL = "https://office365itpros.sharepoint.com/sites/SecureSite"
Set-SPOSite -Identity $SiteURL -RequestFilesLinkEnabled $False 

To check the site settings, run:

Get-SPOSite -Identity $SiteURL -Detailed | Select-Object Request*

RequestFilesLinkEnabled RequestFilesLinkExpirationInDays
----------------------- --------------------------------
                  False                                7

Like any change to SharePoint Online settings, it can take up to a day before updates are effective.

By default, the site inherits the value for the link expiration setting from the tenant configuration, but you can define a more restrictive expiration period if you like. You can’t override the tenant configuration and define a less restrictive expiration period for a site. The link expiration period can be anything from 0 (zero) to 730 days (two years). Usually, the more secure the site, the lower the link expiration period.

OneDrive for Business Settings

As noted above, OneDrive for Business also supports the Request Files feature. The OneDriveRequestFilesLinkEnabled setting in the tenant configuration controls if the feature is available in OneDrive for Business accounts while the OneDriveRequestFilesLinkExpirationInDays sets the expiration period for the sharing links. You can’t prohibit Request Files for selected OneDrive for Business accounts. The feature is either enabled or disabled for all.

Set-SPOTenant -OneDriveRequestFilesLinkEnabled $True –OneDriveRequestFilesLinkExpirationInDays 7

Using Request Files

When someone uses a Request Files link, SharePoint redirects them to a special page where they can select files to upload together with some personal details (First and Last Name) to let the requestor know who uploaded files to the folder (Figure 2).

The person who created the request files link receives email from SharePoint when someone uses the link to successfully upload files to the document library (Figure 3).

Figure 4 shows a set of files uploaded to a folder in a document library. SharePoint Online doesn’t validate the details of a person who uploads a file, so the name recorded as a prefix for the filename could be incorrect or false. That’s not important because it’s assumed that the person who requests file uploads will process whatever comes in afterward to decide what’s useful (or not), rename files, and so on.

In terms of tracking the use of the Files Request feature, SharePoint Online captures when a link is used and a file is uploaded in the audit log. This PowerShell code finds the events for the last 14 days and reports them.

Connect-ExchangeOnline
[array]$Records = (Search-UnifiedAuditLog -StartDate (Get-Date).AddDays(-14) -EndDate (Get-Date) -Operations FileRequestUsed, FileUploaded -ResultSize 1000)
If (!($Records)) {Write-Host "No File upload records found - exiting" ; break}

$Report = [System.Collections.Generic.List[Object]]::new()
Write-Host "Processing" $Records.Count "audit records..."
ForEach ($Rec in $Records) {
  $AuditData = ConvertFrom-Json $Rec.Auditdata
  Switch ($AuditData.Operation) {
    "FileUploaded" {
       $FileName  = $AuditData.SourceFileName.SubString(7,($AuditData.SourceFileName.Length-7))
    }
    "FileRequestUsed" {
       $FileName = $Null 
    }
  } # End Switch
  $ReportLine = [PSCustomObject]@{
      TimeStamp    = Get-Date $AuditData.CreationTime -format g
      UploadedBy   = $AuditData.UserId
      Action       = $AuditData.Operation
      ClientIP     = $AuditData.ClientIP
      Folder       = $AuditData.SourceRelativeUrl.Split("/")[1] 
      FileName     = $FileName
      SiteURL      = $AuditData.SiteURL
      Site         = $AuditData.SiteURL.Split("/")[4]           }
  $Report.Add($ReportLine)
  
} #End Foreach Record

# Remove normal uploads
$Report = $Report | Where-Object {$_.UploadedBy -notlike "*@*"}
$Report | Select-Object Timestamp, Site, Folder, FileName  -Unique

Control Over Files Request

Some people might be cautious about using a feature that allows external people to upload files to SharePoint Online. It could, after all, be a vector that an attacker could abuse to upload infected files. On the other hand, is it any more dangerous than asking external people to email attachments to an internal user so that they can upload the files to SharePoint Online.

Control is available by

• Limiting the number of sites that support Files request.
• Limiting the validity of file request links.
• Training users to use the Files Request feature sparingly, and if they use it, they should take the responsibility of restricting access to the upload link and checking whatever files external people upload before making those files available more broadly within the tenant.

Like any new feature, it will take time for tenants to operationalize Files Request. Happy uploading!

Support the work of the Office 365 for IT Pros team by subscribing to the Office 365 for IT Pros eBook. Your support pays for the time we need to track, analyze, and document the changing world of Microsoft 365 and Office 365.

Block Download Policy covered by Syntex-SharePoint Advanced Management License

Microsoft launched the Syntex-SharePoint Advanced Management license into preview in late January 2023. The license is now generally available and cost $3/user/month. Since news about the license emerged, people have been figuring out if the features covered by the license are worth the cost by examining details of the features it enables. Now a new block download file policy is available for Teams meeting recordings.

Blocking Downloads and Teams Meetings

In February, I covered the Block Download Policy for SharePoint Online, a feature in Syntex-SharePoint Advanced Management to limit users to browser access when interacting with content stored in sensitive sites. Blocking downloads for Teams recordings is a similar feature that’s now available in preview. The big difference is that the block download policy applies tenant-wide for all Teams recordings created after the block comes into force in both SharePoint Online sites (for channel meeting recordings) and OneDrive for Business (for personal meeting recordings).

Clearly Microsoft is responding to a customer need to make Teams meeting recording more secure. Blocking downloads removes the worry that someone with access to a recording of a sensitive meeting can download it before the meeting file automatically expires.

Site-Wide Block Download Policy Applied With PowerShell

As noted above, the block is tenant-wide. No GUI is currently available in the SharePoint Online admin center, so management of the block is by running the Set-SPOTenant cmdlet from the SharePoint Online management module.

Make sure that you run an up-to-date version of the module (I used 16.0.23408.12000) as otherwise the Set-SPOTenant won’t support the necessary parameters. Keeping modules like Exchange Online management, Teams, SharePoint Online, and the Microsoft Graph PowerShell SDK up to date is an important task. Ideally, you should check and update modules monthly. As it’s always nice when PowerShell looks after PowerShell, here’s a script to automate the process, including tidying up by removing old module files afterward.

To impose the block, use Set-SPOTenant to set these parameters:

• BlockDownloadFileTypePolicy from $False (the default) to $True.
• BlockDownloadFileTypeIds to “TeamsMeetingRecording.” This is the only value currently supported by the cmdlet.
• ExcludedBlockDownloadGroupIds to the identifiers of security groups whose members you want to exclude from the block download policy. You can’t use Microsoft 365 groups to exclude accounts. This parameter can be left blank if you want the policy to apply to all accounts. If you want to specify multiple security groups, do so in a comma-separated list.

Here’s the command I ran in my tenant to enable the block policy and check its settings afterward:

Set-SPOTenant -BlockDownloadFileTypePolicy $True -BlockDownloadFileTypeIds TeamsMeetingRecording -ExcludedBlockDownloadGroupIds "dc637020-4b0f-4f65-bdf0-3c7dbe8a83e7"

Get-SPOTenant | Format-List BlockDownLoadFile*, ExcludedBlock*

BlockDownloadFileTypePolicy   : True
BlockDownloadFileTypeIds      : {TeamsMeetingRecording}
ExcludedBlockDownloadGroupIds : {dc637020-4b0f-4f65-bdf0-3c7dbe8a83e7}

It can take up to a day before a policy update becomes effective across SharePoint Online. Before it is effective, anyone can download a Teams meeting recording (Figure 1).

When the block download policy is effective, users don’t see the download options for recordings created after the effective date (Figure 2).

It’s important for users to understand that they are only blocked for new recordings. At least, while the feature is in preview. However, when the block download policy is generally available, a background agent will search for older Teams meeting recordings stored in SharePoint Online and OneDrive for Business and mark the files as blocked for download. Although I can see why customers would want this to happen, the fact is that many of the Teams recordings will age out and disappear in a relatively short period unless users take explicit action to retain the files.

Available in Preview Now

SharePoint Online’s block download policy for Teams recordings is available in preview. After Microsoft makes the block download policy generally available, you’ll need to buy some Syntex-SharePoint Advanced Management licenses to continue using the policy or the block download policy will stop working (perhaps much to the relief of some users!).

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Azure AD B2B Collaboration and Guest Accounts for SharePoint Sharing

Two recent message center notifications highlight closer integration between SharePoint Online and Azure AD. MC526130 (11 March) says that new tenants created after March 31, 2023 will automatically enable the SharePoint Online integration with Azure B2B integration. Existing tenants aren’t impacted by this change. The associated update, also scheduled for roll-out in late March, is MC525663 (10 March). The news here is that SharePoint Online site sharing will use the Azure B2B Invitation manager instead of the legacy SharePoint Invitation Manager (Microsoft 365 roadmap item 117557).

Rationalization Around Azure AD

The two updates rationalize existing sharing methods with external users and focus on Azure AD as the driving force for managing invitations. The journey toward Azure AD B2B Collaboration started in 2021, so it’s been a while coming. The project makes a lot of sense for both customers and Microsoft (their gain is through reduced engineering expenses).

Ten years ago, it was reasonable for SharePoint to manage site sharing invitations. Today, when the site collection-based architecture is replaced by single-sites and most sharing occurs through Microsoft 365 groups and Teams, it’s illogical for SharePoint Online to have its own mechanism. 280 million monthly active Teams users create a lot of work for SharePoint.

Another factor is that site sharing with external users is a relatively uncommon action today. Most external users join groups or teams and gain access to the group-connected site. Although non-group connected sites do exist, they’re in the minority and some of those sites (like hub and communication sites) aren’t candidates for sharing with external people. And of course, even site owners might be blocked from sharing sites by a sensitivity label.

Time to Review Applicable Policies

Overall, I don’t think the change will disrupt many organizations. As Microsoft notes “You may want to review your Azure B2B Invitation Manager policies.” Two policies are worthy of note. The first is the Azure B2B Collaboration policy, which includes an allow or deny list (but not both) of domains.

The policy is now found under Collaboration restrictions in the External Identities section of the Azure AD admin center (Figure 1). It is commonly used to block sharing with consumer domains (deny list) or to restrict collaboration to a set of known domains belonging to partner organizations (allow list). If the organization already supports guest accounts, it’s likely that the collaboration policy already exists. Even so, changes like this are useful reminders of the need for regular review of any policy that affects how external people access tenant resources.

Azure AD cross-tenant access policies are a more powerful and flexible mechanism to control external access through both Azure B2B collaboration and Azure AD direct connect (used for Teams shared channels). Cross-tenant access policies are still relatively new and don’t need to be implemented unless required for a specific reason, so your tenant might not use them yet.

Although the Azure AD B2B Collaboration policy is likely to dominate for the immediate future, over time, I expect a slow transition to take advantage of the granular control available in cross-tenant access policies. When an organization changes over, SharePoint Online will take advantage. Leveraging advances made in Azure AD is an excellent reason for SharePoint Online to embrace Azure AD more fully.

Review Guest Accounts Too

Azure AD B2B collaboration works but that doesn’t mean that you don’t need to manage guest accounts. As more sharing happens, more guest accounts end up in your Azure AD. Some guest accounts are used once to share a document. Others are in ongoing use as guest members of groups and teams access shared documents. It’s a good idea to keep an eye on guest accounts and remove them as they become obsolete.

Support the work of the Office 365 for IT Pros team by subscribing to the Office 365 for IT Pros eBook. Your support pays for the time we need to track, analyze, and document the changing world of Microsoft 365 and Office 365.

SharePoint Block Download Policy Licensed by Syntex-SharePoint Advanced Management and Managed with PowerShell

One of the features covered by the new Syntex-SharePoint Advanced Management license blocks users from being able to download files from a SharePoint Online site or OneDrive for Business account. The idea is to protect sites that store very confidential material by forcing users to work with the files stored in the site using browsers. Users can’t even use the Office desktop apps because those apps download a temporary copy of files to work on them locally.

The block files from download feature is currently in preview. To enable a block download policy for a site, you’ll need to use the Set-SPOSite cmdlet from the latest version of the SharePoint Online management PowerShell module.

Restricting Download Access

I tested the feature by creating a new team called Project Aurora. I then configured the SharePoint Online site belonging to the team by running these commands to find all sites, select the URL for the Project Aurora site, and use it to configure a block download policy with an exclusion for site owners. In other words, site members can’t download files from its document libraries, but site owners can.

[array]$Sites = Get-SPOSite -Limit All
$Site = ($Sites | Where-Object {$_.Title -eq "Project Aurora"}) | Select-Object -ExpandProperty Url
Set-SPOSite -Identity $Site -BlockDownloadPolicy $True -ExcludeBlockDownloadPolicySiteOwners $True

The preview documentation says that site owners can grant exclusions to groups by passing the group identifiers in the ExcludedBlockDownloadGroupIds parameter. I see some issues here because Microsoft has long coached customers not to update membership of group-connected sites through SharePoint Online. In addition, adding a Microsoft 365 group to site membership creates an unsupported condition of nested Microsoft 365 groups. For now, I would avoid using group-based exclusions and concentrate solely on site owner exclusions.

After populating the default document library with some documents, I signed into the site with a member account. The site flagged the restrictions in place and removed the options to download files (Figure 1).

The Teams Files channel tab also removes the download option but doesn’t display a banner to inform the user about the restrictions. The Files channel tab does remove the option to use an Office desktop app to open a document. Before restricting downloads by policy, Microsoft recommends that you check any potential effect that the block might have on other applications, including Power Apps and Power Automate.

The file download restrictions are the same as when using a conditional access policy to limit access when users attempt to access SharePoint content from an unmanaged device. That’s the point of this feature: you don’t need to deploy conditional access policies to get equivalent protection. Although conditional access policies are a good way to control what people can do after they connect to a Microsoft 365 tenant, there’s no doubt that organizations can end up with many different policies to manage. Replacing a conditional access policy with a relatively simple download block applied at the site level might be a good thing to do, especially if you want to have finer-grained control over what sites block file downloads.

Applying the SharePoint Block Download Policy to Multiple Sites

As a practical example of how you might deploy block download policies, let’s assume that you want to stop downloads for all sites assigned the most stringent sensitivity label. In my tenant, that’s a label called “Confidential Access.” The important thing is to know the label identifier (GUID) because that’s how Microsoft 365 workloads connect to sensitivity labels. In this case, the GUID is c99e52c6-f5ff-4050-9313-ca6a3a35710f.

This script applies the SharePoint block download policy to all sites assigned the Confidential Access sensitivity label. First, we find the set of sites associated with Microsoft 365 groups. Because the Get-SPOSite cmdlet does not return all site properties when it processes multiple sites, we need to loop through the site of sites to check the sensitivity label for each site and apply the policy after detecting a matching label:

# Process sites and set the SharePoint block download policy
[array]$Sites = Get-SPOSite -Template "GROUP#0" -IncludePersonalSite:$False -Limit All
Write-Host ("Scanning {0} sites to find those with the Confidential Access label" -f $Sites.count)
[int]$i = 0
ForEach ($Site in $Sites) {
   $SiteData = Get-SPOSite -Identity $Site.Url
   If ($SiteData.SensitivityLabel -eq "c99e52c6-f5ff-4050-9313-ca6a3a35710f" -and $SiteData.BlockdownloadPolicy -eq $False ) {
      Write-Host ("Applying site download block policy to {0}" -f $SiteData.Title)
      Set-SPOSite -Identity $Site.Url -BlockDownloadPolicy $True -ExcludeBlockDownloadPolicySiteOwners $True; $i++
   }
}
Write-Host ("Finished processing. {0} sites updated with a block download policy" -f $i)

Remember Your Syntex Licenses

Remember that every member of a site that uses a block download policy to restrict downloads to site owners or groups must have a Syntex Advanced Management license. Given that you’ll probably only apply this kind of restriction to a limited number of sites, that shouldn’t be a big issue.

Support the work of the Office 365 for IT Pros team by subscribing to the Office 365 for IT Pros eBook. Your support pays for the time we need to track, analyze, and document the changing world of Microsoft 365 and Office 365.

Syntex-SharePoint Advanced Management Covers Secure Collaboration for SharePoint Online

Updated 2 March 2022

I know that many Microsoft 365 organizations don’t use sensitivity labels, even if they have the necessary licenses to use labels to protect content. All Office 365 licenses allow users to read protected content, but you need Office 365 E3 or above to apply labels to files, and Office 365 E5 or Microsoft 365 Compliance E5 for auto-label processing. At least, that’s been the case up to now.

Applying a default sensitivity label for a SharePoint Online document library (Figure 1) counts as automatic processing. Apparently, Microsoft considers the fact that new and modified documents in the library pick up the sensitivity label (unless previously labeled) as reason enough. In late January 2023, Microsoft revealed that this feature was one of the set to be licensed through a new Microsoft Syntex-SharePoint Advanced Management license.

Features Enabled by the Microsoft Syntex-SharePoint Advanced Management License

The new license is in preview and includes other elements to improve secure collaboration based on SharePoint Online and OneDrive for Business, including:

• Using sensitivity labels with Azure AD authentication contexts to limit access to SharePoint Online sites. This feature has been in preview since 2021.
• Restricting access to a SharePoint Online site to members of a Microsoft 365 group. This restriction blocks users who have received access to a file in the site.
• Blocking the download of files from SharePoint Online sites or OneDrive for Business accounts without the need to use Azure AD conditional access policies. In other words, users are forced to use a browser to access the site or account and cannot download, print, or synchronize files. The restriction also blocks access to the Office desktop apps because these apps need to download files to work on them locally.

In addition, Syntex-SharePoint Advanced Management includes some management and governance features. The three examples cited appear to be instances where it’s possible for administrators to do the same thing with some effort. Microsoft is making it easier. For example, the ability to limit access to OneDrive for Business to those who are members of a specific security group stops people licensed to use OneDrive but who aren’t members of the security group from using the app. The same effect is possible by simply removing the OneDrive service plan from their assigned licenses.

I haven’t seen what actions are included in the feature to export recent SharePoint site actions, but it might be possible to replicate the functionality by fetching SharePoint management events from the unified audit log.

My assumption is that any user who takes advantage of a feature licensed by Syntex advanced management requires a license. For instance, site members of a site where a document library uses a default sensitivity label all require Syntex-SharePoint Advanced Management licenses.

I can’t find a public announcement by Microsoft about the Syntex-SharePoint Advanced Management license. Cynics will say that this is another example of how Microsoft creates licenses for new functionality to generate additional revenue from its installed base. A more benign view is that the new license allows people with Office 365 E3 licenses to use the security and governance features enabled by Syntex Advanced Management. When I find out more details about licensing, including if some features covered by Syntex Advanced Management are also available through other licenses, I shall share the information.

Viewing Metadata for Protected Files

On an associated topic, I was asked why the metadata of documents protected by sensitivity labels remains visible to people who have no right to access these files. It’s a good question because some get confused when they notice an interesting document in a library but can’t open it because they’re blocked by the rights assigned in the label. For instance, who wouldn’t want to open a document with a title like “Proposed Pay Rises for Staff”?

When you enable SharePoint Online and OneDrive for Business to support sensitivity labels, it allows the workloads to deal with protected (encrypted) content. SharePoint Online stores protected files in an unencrypted format to allow functions like indexing and data loss prevention policies to work. Any access to a document, such as a user opening or downloading a file, causes SharePoint Online to encrypt the document so that the application used to open the file (like Word) can apply the rights assigned to the user. Everything works very nicely and those who have access to files can work with that content and those who don’t cannot.

When browsing items in a document library, site members can see metadata like the titles and authors of protected documents. Attempts to open these documents fail if the user doesn’t have the necessary rights. Because SharePoint Online doesn’t encrypt or obscure the metadata, those users know that documents with potentially very interesting content are available.

How SharePoint Online Stores Documents

The reason why document metadata is visible to all site members is rooted in how SharePoint Online stores documents. SharePoint Online uses Azure SQL as its storage platform. Blob storage holds documents and other files while metadata is in a separate table (list). The Azure SQL data is heavily protected against illegal access. Once a user has access to a document library, the assumption is that SharePoint can show them all the items, which is what they see in the list shown in a browser or the Teams files channel tab. It’s only when a user attempts to access a protected document that SharePoint Online validates their right to open that content.

You can argue that SharePoint Online and OneDrive for Business should hide the existence of protected documents that the user can’t open, but this would require SharePoint Online to check that access before displaying documents in a library. Such a check would incur a huge performance penalty because SharePoint Online cannot assume that the rights assigned in a sensitivity label are the same as the last time it checked.

New Functionality, New Costs

Although the news about the Syntex-SharePoint Advanced Management license will disappoint some, it’s reasonable that Microsoft should charge extra for security and management features that not every Microsoft 365 tenant will want or need. Those that need the functionality will simply have to pay the $3/user monthly cost. Hasn’t that always been the way?

Stay updated with developments across the Microsoft 365 ecosystem by subscribing to the Office 365 for IT Pros eBook. We do the research to make sure that our readers understand the technology.

Document Management Par Excellence

Browsing through Twitter (as some still do), I came across a “What is SharePoint” article. Given that I’ve used SharePoint since the initial release of SharePoint Portal Server in 2001, I opened and read the content. (Fun fact: SharePoint originally used the Exchange ESE database engine. The move to SQL happened with SharePoint 2003. Around the same time, the Exchange “Kodiak” project dabbled with the idea of moving to SQL. That project never proceeded).

In any case, the article sets out to explain what SharePoint is and how people use it, which is a worthy purpose. Some good points are made, especially about the transition from the old-style SharePoint to the new UX and architecture. Inevitably, a couple of points of contention exist, so here’s what I think about the role played by SharePoint Online today inside the Microsoft 365 ecosystem.

SharePoint Online Has Always Been Part of Office 365

First, the article asserts that SharePoint Online joined Office 365 in 2012 following the release of SharePoint 2013 Server. This is inaccurate. SharePoint Online has always been part of Office 365 and was included in the beta released in April 2011 and the initial version released on June 28, 2011. Microsoft based the initial release of SharePoint Online on SharePoint 2010 Server. There’s no doubt that the subsequent upgrade to the Wave 14 servers (Exchange 2013 and SharePoint 2013) helped Office 365 enormously, but that came later.

SharePoint’s Toolbox

The article covers the attempts of SharePoint to be all things to all customers by providing features like task management and conversations. One undoubted truth for SharePoint is that it failed to be the “Swiss army knife of collaboration.” That’s a good thing because we learn through failures, and I think SharePoint learned that its strengths are in content management and not collaboration or workflow.

Then again, you can argue a good case that other developments in the Microsoft 365 ecosystem left the capabilities available in SharePoint behind. The big difference between on-premises and the cloud is that on-premises servers are often the fulcrum of a complete ecosystem. Once servers like SharePoint and Exchange become part of a cloud solution, they are no longer at the center and must instead function as a productive part of the ecosystem. Teams, Yammer, and Outlook are better points for collaboration (each with its own strengths). Planner and Project are better at task management, and Power Automate offers better workflow capabilities. A common point is that all these apps contribute to and use services from other apps and Azure, including SharePoint Online. All contribute to the ecosystem, as does SharePoint Online.

Once Teams gathered speed, there was no stopping it, especially after the acceleration in demand for its services during the pandemic. SharePoint Online wisely dropped working on solutions that were never going anywhere and concentrated on what it does best, which is to deliver an enterprise-class document management service to Microsoft 365. After SharePoint focused, its developers were able to exploit other areas based on existing capabilities, like what is now Microsoft Lists.

SharePoint and Teams

I fundamentally disagree with the article’s assertion that SharePoint is the backbone of Microsoft Teams. You could say the same about Azure (Teams uses many Azure services, including Azure Cosmos DB for its message stores), or Exchange (Teams uses Exchange Online for its calendar and to store compliance records). It’s true that every new team comes complete with a new SharePoint Online site. The same is true for private and shared channels, each of which has a site associated with the site belonging to the host team. But this simply reflects an app’s use of SharePoint Online for document management. It’s just like the way Yammer stores documents for its communities.

This brings me to the true backbone of Teams: Microsoft 365 groups. Without the identity management, membership model, and resource provisioning of Groups, Teams wouldn’t work the way the app does today.

In December 2017, I wrote an opinion piece saying that Office 365 Groups saved SharePoint Online. I was wrong: although Outlook groups demonstrated how users could have easy access to SharePoint without having to navigate SharePoint’s browser interface, it was Teams saved SharePoint Online by providing users with a reason to use SharePoint Online. I said “People don’t think about using SharePoint. They think about using Teams, or Planner, or Yammer, or Outlook” and “if they have a file to store, they put it wherever the application dictates, like in the Files section of Teams. It is a natural and easy way for people to use document management and it is the engine driving SharePoint usage. That assertion is truer now than it was in 2017. Accessing SharePoint Online files through the Teams Files channel tab (Figure 1) is an area that Microsoft has improved over the years and is now as functional as the SharePoint browser interface in practical terms.

The growth in Teams to 270 million monthly active users (likely higher now because Microsoft hasn’t updated the figure since January 2022) propelled SharePoint usage to new heights. When Microsoft announced the new Syntex backup solution at Ignite 2022, they said that “Every workday, on average, our customers add over 1.6 billion documents to Microsoft 365.” Those documents go into SharePoint Online sites and OneDrive for Business accounts, and users create many of those files using the connection between Teams and SharePoint Online (here’s Microsoft’s description of that connection).

OneDrive for Business

SharePoint Online deals with business users. OneDrive for Business is the personal side of SharePoint Online. Microsoft uses the consumer version of OneDrive as the document management solution for consumer apps, including Teams Personal.

Microsoft didn’t break out the percentage of the 1.6 billion documents added daily so we don’t know how many ended up in OneDrive for Business. I suspect that the proportion is roughly half and half. OneDrive for Business stores files shared in Teams chat and Outlook messages, including Loop components. It stores user files created in the Documents folder on Windows desktops, and so on. OneDrive for Business is everywhere.

One of the reasons why OneDrive for Business does so well is its excellent sync client. I would not have said that some years ago because the original OneDrive sync client was awful. Synchronization challenges have been encountered and overcome since and the current sync client does a wonderful job of keeping files synchronized across devices. The addition od differential synchronization in 2020 was an important step in this process. I depend on OneDrive synchronization and document auto-save to preserve my work.

SharePoint is a Basic Microsoft 365 Workload

Microsoft considers three workloads to be the foundation of Microsoft 365: Exchange, SharePoint, and Teams. SharePoint Online is the critical document management service for Microsoft 365 and it fulfils that role extremely well. As time passes, the connections and dependencies between the base workloads grow and deepen, something that never happened in the on-premises world.

It’s been interesting to observe the development of SharePoint from a small department-level server to a massive worldwide service for hundreds of millions of users. Many people never realize that they use SharePoint Online because they interact through other apps. That’s just fine. No application is the center of anything these days. Services are what’s important and SharePoint Online delivers a great service, and that’s what’s important.

Move Videos from Azure Storage to SharePoint

Publicized in message center post MC437552 (September 23), the Stream migration tool to move videos from the classic Azure-based storage to SharePoint Online and OneDrive for Business became available in public preview in early October. Microsoft says that they expect the migration tool to be generally available in early 2023.

 Microsoft says that the migration tool will “transfer audio and video files to Stream (on SharePoint) while also bringing over all critical metadata and permissions associated with your Stream (Classic) content. Additionally, the tool will ensure that links and embeds of Stream (Classic) content will be redirected to the same content on Stream (on SharePoint).”

Originally launched in June 2017, Stream Classic took over from the original Office 365 Video portal in early 2020. That lengthy transition testified to the difficulties involved in moving content from one repository to another. The Stream transition to SharePoint Online is quite a strategic move for Microsoft as it further consolidates Microsoft 365 application content within SharePoint Online (and OneDrive for Business). Using a common repository for multiple data types makes it easier to engineer and maintain services like search. Microsoft says that they expect to announce a retirement date for Stream classic in early 2023 and plans to keep the service available for a year afterward.

The Migration Tool

Microsoft uses the Migration Manager tool to migrate videos from Stream classic. The tool is available in the SharePoint Online admin center (Figure 1). It’s the same technology Microsoft uses to move other data sources into SharePoint. Levering a proven migration framework rather than building a bespoke tool is an intelligent approach.

The Migration Process

I don’t intend to go through Microsoft’s step-by-step documentation for the Stream migration process. Instead, I’ll highlight the major phases and note my experience of running a migration. The migration process is divided into three stages:

• Scan: Look for Stream classic videos stored in Azure storage. The videos are categorized as Stream group (a channel), a Microsoft 365 group, or user content. The scan automatically populates the known locations in Stream classic. You can decide to migrate some or all of the locations (Figure 2).
• Prepare migration: After all the Stream classic videos are found, they can be added as migration targets in either SharePoint Online sites or OneDrive for Business accounts. User content videos move to OneDrive while group content moves to SharePoint.
• Migrate: This stage processes the videos by moving them from the Stream classic repository to the nominated targets. During this process, any links and embeds pointing to the old classic locations are redirected to the new locations.

Background jobs perform the scanning to find video files. The time required depends on the load on the service and the number of videos in the various locations. Suffice to say that it can take some time before the scan results are available. Apart from reviewing the results on screen (Figure 3), you can download reports to analyze the information and decide what content to include in the migration.

Adjusting Settings Before Moving Content

One of the big advantages of moving Stream storage to SharePoint is that users can take advantage of larger storage quotas. However, this isn’t a good reason to migrate everything as there’s no doubt that some older video content can probably be left to rot in Stream classic until the service closes down. This is especially true for Teams meeting recordings. When Microsoft introduced automatic expiration for Teams meeting recordings, they noted that 99% of all recordings are not watched again 110 days after the meeting. Applying this rule to old videos of Teams meetings waiting to be migrated seems like a good idea.

Before migration, you can adjust the target location if necessary (Figure 4). For instance, you might decide to bring the videos belonging to multiple Microsoft 365 groups together in a single SharePoint Online site.

Migrating

After making whatever adjustments are required, you can migrate the content. Be sure that you’re ready because there’s no way to reverse course once the migration process moves videos from Stream classic to SharePoint.

Once again, this is a background process that will proceed at its own pace. I launched the job overnight and came back in the morning to find that everything had gone well with only one hiccup (Figure 5).

The hiccup seemed to be a glitch in the migration process. Selecting the line displays a flyout pane with details of the migration. This reported that the migration couldn’t download one video owned by a Microsoft 365 group. However, when I looked at the group through the classic Stream portal, no videos were present and a set of videos were in the destination SharePoint Online site. This is a preview and it is notoriously difficult for migration tools to process 100% of content right exactly right. Even with the glitch, the right outcome occurred as the process moved videos from Stream classic to SharePoint Online.

It’s important to realize that once the migration tool moves videos from Stream classic, it soft-deletes the files to hide them from the Stream classic portal. The soft-deleted files do not appear in the Stream recycle bin. Microsoft will remove the files permanently “180 days after admin disables the tenant but not before classic end of life,” implying that there might be some way to restore videos if necessary.

Don’t Forget the Pre- and Post-Migration Work

Migrating from Stream classic to Stream for SharePoint isn’t just a matter of moving video files. Up-front planning is needed to determine what the target locations should be. Just because a Microsoft 365 group owned some videos in Stream classic doesn’t mean that its SharePoint Online site is the right target location.

Critical metadata moves to SharePoint Online (title, description, thumbnails, transcripts, and permission), but the metadata for some videos might need updating after the transition. Equally, the need might exist to adjust other video settings (like add chapters or set an expiration date) or to edit a transcript. But if you just want to move videos across, the Stream migration tool does a good job.

Keep up to date with developments like the transition to Stream for SharePoint by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers understand the most important changes happening across Office 365.

Another Functionality Gap Plugged

Microsoft 365 notification MC400977 (updated August 31) covers the introduction of the Recording video feature in the Stream for SharePoint browser client (Microsoft 365 roadmap item 88522). This is part of the work to replace the old Stream classic browser interface by introducing a new Stream portal. In this instance, the upgrade allows users to create 15-minute videos by recording themselves or their screen.

Users in targeted release tenants should now have this functionality. General availability roll-out is ongoing and should be complete by the end of October.

In passing, it’s worth noting that the beta version of the Stream 2.0 for iOS and Android apps (Figure 1) are available for testing. This version allows users to play videos stored in Stream Classic and Stream for SharePoint. Although, the app doesn’t yet support recording, it’s good to see the ecosystem building out.

Recording a Stream Video

Getting back to the Stream for SharePoint browser client, Microsoft says “Users will now be able to use the new Stream camera to record their webcam, record their screen, add edits (think ink, text, backgrounds, and filters) and upload to their OneDrive. Future iterations of the camera will include more features, such as adding music clips.”

Update: the Stream browser app now offers two options for recording: camera and screen.

In other words, Stream can use the technologies built into a workstation to record video (webcam) and screen, and then do some basic editing (some applied before recording starts), before storing everything in OneDrive for Business.

To begin, select the big New recording button in the Stream client. This launches a new browser tab ready to record video. Like Teams, Stream supports background effects (referred to as a backdrop), and offers the set of default background images available in Teams along with background blur and the ability to upload an image. Unfortunately, there doesn’t seem to be a way to save a custom background the way you can with Teams, nor does Stream offer the chance to use any custom background images you’ve already uploaded for Teams. As shown in Figure 2, the same green-screen technique is used to place the user in front of the background image. Interestingly, grab handles are available for the user image to allow the user to drag and place their image anywhere on the recording canvas. They can also resize their image to make it larger or smaller as appropriate to the content being recorded.

You can have great fun playing with the effects built into the Stream camera. Anyone who’s accustomed to working with video apps on mobile phones or other platforms will find nothing challenging here. In my case, I limited myself to moving my picture to the bottom right of the backdrop and inserting some text (Figure 3).

When everything’s ready, click the big round record button. Stream starts a three-second countdown (to settle your nerves) and then starts to record. When you’re finished (or come to the end of the 15 minute maximum supported for recordings), hit the stop button. You now have an opportunity to review what Stream captured (Figure 4). If you’re happy to keep the content, click Publish.

If you have an app like OBS VirtualCam or Snap camera that appears as a valid device camera, you can use these devices instead of a standard webcam.

Recording Files

When it publishes a video, Stream writes it into the top-level of the user’s OneDrive for Business account. It would be nice if Stream allowed you to defined a folder to store these recordings. The files are named after the date and time of the recording, so you end up with files like 20220913_203811 (recorded on 13 September, 2022 at 20:38:11). Files have a .wbem extension, indicating that the files are saved in the WebM format.

Updating Recordings

Once stored in OneDrive for Business, you can update the properties of recordings to generate a transcript and captions, add some text to describe what the video is about, and allow or disable comments (Figure 5), or share the recording with other people.

One thing I do is rename the file to give the recording a title that’s more appropriate to its content. Renaming has a consequence. The Stream client caches information about videos and will continue to display the old file name for a while after the rename happens. Any attempt to access the video at this point will fail because Stream tries to open the file with the old name. However, after a few minutes (or a browser refresh), the cache should catch up with actuality and display the new name.

Relationship with Clipchamp

Microsoft acquired Clipchamp in late 2021. Since then we’ve been waiting to see how Microsoft will make Clipchamp available to Microsoft 365 commercial customers (it’s already included in the Microsoft 365 family and personal plans). It seems reasonable to assume that Microsoft will include Clipchamp Essentials in Office 365 SKUs at some point in the future to allow users to edit the videos they record with Stream (the trim feature available in Stream classic is unavailable for the new Stream) or import from other sources, or indeed stitch segments captured in individual files together to create a longer video.

Stream Continuing to Evolve

Microsoft is making steady progress on the transition to Stream on SharePoint. The new web player is 100% deployed to Office 365 commercial tenants (not yet GCC) to play videos stored in Teams, SharePoint Online, and OneDrive for Business. Being able to record videos is another important part of the puzzle and it’s nice to see that it’s available now.

Keep up to date with developments like the transition from Stream Classic to Stream for SharePoint by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers understand the most important changes happening across Office 365.

Document Label Mismatches With Sensitivity Labels of Different Priorities

Two years ago, Microsoft launched support for sensitivity labels in SharePoint Online, including the ability to detect a mismatch between the label assigned to an Office document and the label assigned to the site storing the file. The mismatch occurs when the document library has a higher priority than the site label. For instance, someone might upload a document labeled Highly Confidential to a site labelled General Access, or they might update a document to assign it with a label with a higher priority than the site label.

A document label mismatch mightn’t be a problem. Storing sensitive material in a site designated for less sensitive information could be exactly what the user intended to do. However, a mismatch might also create a potential issue when users with access to a site might see highly confidential material. In practical terms, the users might not be able to open the files because they don’t have the necessary rights, but they can see file metadata such as titles, authors, and so on.

Audit Record for Mismatch Missing Important Data

When it detects a document label mismatch, SharePoint Online generates a DocumentSensitivityMismatchDetected audit record in the Office 365 (unified) audit log. The audit record contains information about the:

• The file name.
• The site URL and relative location (full URL).
• Sensitivity label and priority for the document label.
• Sensitivity label and priority for the site label.

The big piece of missing information is the account name (user principal name) of the user who caused the document label mismatch. It’s not as if SharePoint Online doesn’t know who caused the problem. After all, SharePoint Online sends the miscreant an email notification (Figure 1) about the issue to prompt them to consider if a label change is necessary.

Dealing with Missing User Information

The solution exists in other audit data. When someone updates or uploads a document, SharePoint Online captures an audit event for the action. These events capture user information. Later, SharePoint detects the mismatch. SharePoint Online stores documents in lists, and each item in the list has a unique identifier. The identifier is in the audit event for the upload or change. It’s also in the event generated when SharePoint finds the mismatch. Therefore, we can reference the upload/change event to find who created the mismatch.

To illustrate the point, I wrote a PowerShell script to:

• Connect to the compliance endpoint to collect information about the labels used in the tenant.
• Build a hash table of the label identifiers and display names. The audit events log label identifiers, so we can use the hash table to find the display name.
• Search the audit log for FileUpdated, FileModified, and DocumentSensitivityMismatchDetected events. The script looks back over the last 80 days. Given the volume of FileUpdated events often found in tenants, you could reduce this period.
• Split the audit records into those for document mismatches and the other events.
• Create a hash table composed of list identifiers and usernames from the document upload and change events.
• For each of the document mismatch events, lookup the hash table to match against the list identifier and return the username responsible for the mismatch. Also resolve the sensitivity labels assigned to the document and site to the label display names.
• Report the results. Figure 2 shows typical results as viewed through Out-GridView.

The full script is available from GitHub.

Some people like to block the messages sent by SharePoint Online using an Exchange Online mail flow rule so that they can send their own notifications to users. It would be easy to take the report data generated by the script and use that information to create and send appropriate messages, perhaps using the Microsoft Graph PowerShell SDK.

Blocking Email Notifications

To stop SharePoint Online sending emails to advise users about label mismatches, you can update the tenant configuration:

Set-SPOTenant -BlockSendLabelMismatchEmail $True

The setting affects all sites. It isn’t possible to block the notification emails about mismatched labels for selected sites. Blocking emails also stops SharePoint Online writing audit events to record document label mismatches. Microsoft plans to break the link between the two actions so that a tenant can block emails without stopping the creation of the audit records, but no date is available for this update.

Audit Mystery

It’s a mystery why Microsoft decided that the DocumentSensitivityMismatchDetected shouldn’t contain the user information, I see no logic in that decision, but once you know about it, you can compensate. Isn’t PowerShell wonderful?

Learn how to exploit the data available to Microsoft 365 tenant administrators through the Office 365 for IT Pros eBook. We love figuring out how things work.

Default Sensitivity Label for SharePoint Document Libraries Now Rolling Out Worldwide

Update 2 April 2023

In January 2022, I explained the process to assign a default sensitivity label to a document library in a SharePoint Online site. At the time, Microsoft was in the early days of the feature’s development and the configuration was a very manual process. Now, the public preview software is generally available worldwide. Documentation to set up and use the feature also available.

Setting a default sensitivity label is very simple. Select Library settings from the cogwheel menu and choose the desired sensitivity label (Figure 1). Naturally, you can only select a label that’s configured for file and email protection rather than those set up for container management. If you have multiple document libraries in a site, each library can use a different default sensitivity label. That’s a nice touch because usually if a site has multiple libraries, the libraries serve different purposes, and the chosen label can reflect that purpose instead of being a one-size fits-all selection.

Licensing

Although Microsoft hasn’t confirmed this, assigning a default sensitivity label to a document library will follow the usual line of regarding anything that performs an automatic action as a premium feature. Accordingly, you’ll need Office 365 E5 or Microsoft 365 E5 Compliance licenses to use the feature when it is generally available.

Update: The feature is now available and requires one of these licenses:

• Microsoft Syntex – SharePoint Advanced Management
• Microsoft 365 E5/A5/G5
• Microsoft 365 E5/A5/G5/F5 Compliance
• Microsoft 365 F5 Security and Compliance
• Microsoft 365 E5/A5/G5/F5 Information Protection and Governance
• Office 365 E5/A5/G5

PDF Files and Existing Documents

As Microsoft’s documentation explains, the reference to support for PDF files in the UI is incorrect. New Office documents uploaded by users receive the default label within a few minutes, but PDFs are ignored for now. It’s likely that Microsoft will address this issue when the feature is generally available toward the last quarter of 2022.

New documents that have labels are ignored. Existing documents already present in the library are also ignored. In other words, SharePoint Online doesn’t scan all documents and apply the default sensitivity label to any without an assigned label. However, when users edit Office documents that don’t have an assigned label, SharePoint Online will apply the default sensitivity label defined in the policy applicable to the site. This change is due in mid-October 2022.

More Changes Coming for PDFs

In June, Microsoft announced that Office applications would maintain sensitivity label support when used to create PDFs. This is part of Microsoft’s work to remove the need for organizations to deploy the now-deprecated unified labelling client to apply sensitivity labels to PDFs. According to MC387639, the public preview for this functionality should be available around about now.

An associated message center notification (MC411677, August 10 2022) lets Visual Basic for Applications (VBA) developers know that soon PDFs generated when VBA scripts use Office features will also maintain sensitivity labels for the output files. This is Microsoft 365 roadmap item 93406. Microsoft is warning that some VBA add-ins will need to be updated when the change is effective in December 2022.

Meanwhile, Adobe is running a preview program to allow its Acrobat product to apply, remove, and update sensitivity labels to PDFs. The free Adobe Acrobat DC reader product has been able to read protected PDFs (if the user has the appropriate rights granted by the sensitivity label) for several years. The new functionality is currently understood to be limited to Adobe’s paid-for products.

Sensitivity Labels Increasingly Mainline

It takes time for a new technology to become mainline. Sensitivity labels are getting there. Native (built-in) support for encryption, decryption, and rights management within apps are important steps forward. Office and PDF documents are the most common formats used within Microsoft 365. Their increasing embrace of sensitivity labels makes it easier for people to protect their most sensitive information, and that’s a good thing, even if it makes it a little harder for ISVs to process encrypted user data.

Keep up to date with developments like the app support for sensitivity labels by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers understand the most important changes happening across Office 365.

SharePoint External Users From Guest Members and Sharing

A SharePoint external user is someone who doesn’t have an account in your tenant. Because of the influence of Teams, most SharePoint Online external users are guest accounts, created when external people join the membership of Microsoft 365 Groups (teams). If the organization uses the SharePoint Online integration with Azure AD B2B collaboration, SharePoint also creates guest accounts when people share files or folders with external people.

As discussed in this article, it’s reasonably easy to generate a report of the membership of all Microsoft 365 groups in a tenant. The report includes guest accounts and can be used to figure out if guests from the wrong places (like competitors) have access to information in your tenant. However, the script that creates the report relies on cmdlets like Get-UnifiedGroupLinks or Graph API requests to return details of group members, and these exclude any mention of guest accounts in a SharePoint site that aren’t members of the group which owns the site.

PnP Samples Repository

This brings me neatly to a script to report external users posted in the PnP Samples repository (a useful place to go for SharePoint-centric code examples). Reflecting that there are usually multiple ways to solve a problem, three versions are available (CLI for Microsoft 365, SharePoint Online PowerShell module, and PnP PowerShell).

Unhappily, there doesn’t appear to be a good way to retrieve the external users for a site using a Graph API request. You can certainly find the set of all guest accounts in a tenant, or the guest accounts for a team/group, but these methods exclude the guest accounts added for sharing purposes.

The Oddness of Get-SPOExternalUser

The lack of a better method is why the scripts found on the internet use the Get-SPOExternalUser cmdlet. It’s an odd cmdlet in some ways.

For example, Get-SPOExternalUser has a PageSize parameter to limit the number of external users returned. The maximum is 50, which means that if more than this number of external users exist for a site, you must continue fetching until all are retrieved (the Position parameter controls the start of the page of users to fetch). You end up with commands like:

[array]$users = Get-SPOExternalUser -SiteUrl $SiteId -PageSize 50 -Position 50

And after fetching a page of user data, you must combine it with the other pages to get a complete set. Although pagination is common with Graph API requests, it’s unusual to see it used like this with a cmdlet that could surely benefit from a parameter to fetch all matching items, like:

Get-SPOExternalUser -Limit All

Moving onto the output, here’s an example of the data returned for an external user (guest account):

Email         : vasil@michevxx.com
DisplayName   : Vasil Michev (MVP)
UniqueId      : 1003BFFD9AF15B76
AcceptedAs    : vasil@michevxx.com
WhenCreated   : 05/11/2018 18:46:40
InvitedBy     :
LoginName     :
IsCrossTenant : False

As far as I can tell, the InvitedBy and LoginName properties are not used. Across all the sites in my tenant, I found one instance of the InvitedName property being populated. In that case, the property held the user principal name of the guest account, and I couldn’t figure out how this happened.

The AcceptedBy property holds the name of the account that accepted the invitation to the site (to share a document or as a guest member). This property is not populated for sites belonging to shared Teams channels. Instead, a LoginName property captures the account used to connect to the channel site.

The WhenCreated property also deserves some comment. It seems like Microsoft reset this value for many accounts at around 18:46 UTC on 5 November 2018. Many accounts across multiple sites have this creation date. It’s an unnatural concentration of external users created at a specific time on that date. I can’t explain it.

Creating a SharePoint External Users Report

Your account needs to hold the Global tenant administrator or SharePoint administrator role to run this script and generate a SharePoint external users report. The steps are straightforward, which is probably why so many versions are available online. This version captures some extra information about the channel-connected sites used by Teams.

• Find all sites.
• For each site, get its external members.
• Create a report file.

Here’s the script:

$Sites = Get-SPOSite -Limit All | Sort-Object Title

$ExternalSPOUsers = [System.Collections.Generic.List[Object]]::new() 

#Iterate through each site and retrieve external users
$Counter = 0
ForEach ($Site in $Sites) {
    $Counter++
    Write-Host ("Checking Site {0}/{1}: {2}" -f $Counter, $Sites.Count, $Site.Title)
    [array]$SiteUsers = $Null
    $i = 0; $Done = $False
    Do {
      [array]$SUsers = Get-SPOExternalUser -SiteUrl $Site.Url -PageSize 50 -Position $i
      If ($SUsers) { 
        $i = $i + 50
        $SiteUsers = $SiteUsers + $SUsers }
      If ($SUsers.Count -lt 50) {$Done = $True}   
    }  While ($Done -eq $False)

    ForEach ($User in $SiteUsers) {
       $ReportLine    = [PSCustomObject] @{  
         Email        = $User.Email 
         Name         = $User.DisplayName
         Accepted     = $User.AcceptedAs
         Created      = $User.WhenCreated
         SPOUrl       = $Site.Url
         TeamsChannel = $Site.IsTeamsChannelConnected
         ChannelType  = $Site.TeamsChannelType
         CrossTenant  = $User.IsCrossTenant
         LoginName    = $User.LoginName }
        $ExternalSPOUsers.Add($ReportLine) }
} #End ForEach Site

Playing with PSWriteHTML

Now that we have some data to report, I’ll reveal that the real reason for this article is to mention the PSWriteHTML module. The module is maintained by Przemyslaw Klys and its job is to make HTML output easier to generate for PowerShell scripts. The ImportExcel module is another example of a community-created module to help people generate nicer output.

In any case, to create a HTML report, I used these commands:

Import-Module PSWriteHTML.psd1 -Force
$ExternalSPOUsers | Sort Email | Out-HtmlView -HideFooter -Title "SharePoint Online External Users Report"

Figure 1 shows the output, which at first glance looks like a nicer version of the output generated by the Out-GridView cmdlet. The important difference is that you can export the HTML report in different formats, including a nice PDF file.

Having different options to share information is a nice thing. If you create reports from PowerShell, consider having a look at the PSWriteHTML module. It might solve some problems for you. After all, it created a prettier SharePoint External Users report for me!

Learn more about how the Office 365 applications really work on an ongoing basis by subscribing to the Office 365 for IT Pros eBook. Our monthly updates keep subscribers informed about what’s important across the Office 365 ecosystem.

Introducing the Tenant Admin Namespace for SharePoint Graph Settings

Despite being the two basic Microsoft 365 workloads, one of the notable gaps in Microsoft Graph API coverage has been administrative interfaces for SharePoint Online and Exchange Online. A small but valuable step in the right direction happened with the appearance of the settings resource type in the TenantAdmin namespace. For now, the coverage for tenant settings is sparse and only deals with some of the settings that administrators can manage using the Set-SPOTenant PowerShell cmdlet, but it’s a start, and you can see how Microsoft might develop the namespace to handle programmatic access to settings that currently can only be managed through an admin portal.

Options to Manage SharePoint Online Settings

SharePoint Online tenant-wide settings apply to SharePoint Online sites and OneDrive for Business accounts. Like all Graph APIs, apps must have permissions to be able to make requests. The read-only permission is SharePointTenantSettings.Read.All while you’ll need the SharePointTenantSettings.ReadWrite.All permission to update settings.

Three methods are available to use the new API:

• The Graph Explorer.
• A dedicated app registered in Entra ID
• The Microsoft Graph PowerShell SDK.

The Graph Explorer is acceptable for testing or one-off commands. However, given that the Set-SPOTenant cmdlet is available, it’s unlikely that you’d use the Graph Explorer as your preferred method to update settings.

Creating a dedicated app just to manage SharePoint Online settings is unlikely too unless you use the same app to manage multiple tenants. This points to the most likely use of the TenantAdmin API, which is to allow MSPs to create apps to manage multiple tenants on behalf of customers.

The Microsoft Graph PowerShell SDK could be used to replace the SharePoint Online management module. An organization might want to do this to rationalize the number of PowerShell modules its developers work with and maintain. I can see this happening in the future when Microsoft has developed the TenantAdmin API to match the capabilities available today through the Set-SPOTenant cmdlet. For now, I’d stay with the SharePoint module and keep a close eye on what happens with the API.

Updating SharePoint Online Settings with the Microsoft Graph PowerShell SDK

As an example of using the new API, let’s update the setting controlling Loop components in Microsoft 365 apps. This seems appropriate given the recent appearance of Loop components in OWA. The setting controlling the availability of Loop components is IsLoopEnabled, which is True by default. Here’s the code to retrieve the current setting:

Connect-MgGraph -Scopes SharePointTenantSettings.ReadWrite.All
$Uri = "https://graph.microsoft.com/V1.0/admin/sharepoint/settings"
$SPOSettings = Invoke-MgGraphRequest -Uri $Uri -Method Get
$SPOSettings['IsLoopEnabled']
True

To change the setting to False (and disable Loop components), we use the same URI and run a Patch request. To make the command slightly more interesting, we’ll also update the SharePoint News feed setting at the same time and set a new default time zone for new sites created in the tenant. The time zone for new sites is an example of a setting that cannot be set using the Set-SPOTenant cmdlet. Currently, the time zone can only be set in the SharePoint admin center, so this is an example of how the Graph API will expose new settings.

First, we create a payload object.

$NewSettings = @{
    "isLoopEnabled" = "false"
    "isSharePointNewsFeedEnabled" = "true"
    "tenantDefaultTimezone" = "(UTC) Dublin, Edinburgh, Lisbon, London"
}

Then, we patch the settings.

Invoke-MgGraphRequest -Uri $Uri -Method Patch -Body $NewSettings

SharePoint responds by listing all the settings available to the API: You can see that the two settings have the values contained in the payload.

Name                           Value
----                           -----
isFileActivityNotificationE... True
isCommentingOnSitePagesEnabled True
sharingBlockedDomainList       {Gmail.com}
sharingAllowedDomainList       {hotmail.com, live.com, locklan.com.au, Microsoft.com...}
siteCreationDefaultManagedPath /sites/
deletedUserPersonalSiteRete... 60
isSiteCreationUIEnabled        True
isSyncButtonHiddenOnPersona... False
isSitePagesCreationEnabled     False
tenantDefaultTimezone         (UTC) Dublin, Edinburgh, Lisbon, London
isLoopEnabled                  False
personalSiteDefaultStorageL... 5242880
allowedDomainGuidsForSyncApp   {}
isSiteCreationEnabled          True
availableManagedPathsForSit... {/sites/, /teams/, /containers/}
isResharingByExternalUsersE... False
isSharePointMobileNotificat... True
sharingDomainRestrictionMode   none
sharingCapability              externalUserAndGuestSharing
isMacSyncAppEnabled            True
imageTaggingOption             basic
isUnmanagedSyncAppForTenant... False
isSitesStorageLimitAutomatic   True
isSharePointNewsfeedEnabled    False
excludedFileExtensionsForSy... {*.exe, *.zip, *.rar, *.pst...}
@odata.context                 https://graph.microsoft.com/beta/$metadata#admin/sharepoint/settings/$entity
siteCreationDefaultStorageL... 26214400

Something to Monitor

I suspect that the new API will not be heavily used for now and won’t until it attains feature comparability with the Set-SPOTenant cmdlet. But that’s not the important thing to take away. This is the start of the development of Graph API support for tenant administrative settings, and that’s certainly something to welcome.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Grid View and Details Now Available

One of the things people often forget about Teams is that the app is built from other apps. Some apps, like the Activity Feed and Chat, feature in the navigation rail. Others are limited to a channel, like the Wiki and Files channel tabs. The Wiki tab is losing its default status, meaning that Teams will no longer install it automatically for new channels, but the Files channel tab is going nowhere. In fact, Microsoft is steadily improving its functionality. Given that it’s one of the two default tabs added to all channels, this is a good thing.

The last overhaul was in early 2020, and recently the Files channel tab received another useful upgrade. Microsoft didn’t post an announcement in the Microsoft 365 message center, but the changes are clear and obvious. The possibility might exist that I totally missed the changes when they first appeared, but I thought I’d discuss them anyway.

The Role and Functionality of the Files Tab

The role of the Files channel tab (aka the FileBrowser app) is simple. It allows team members to view the files in the channel folder of the default document library in the SharePoint site belonging to the team. Each channel has its own folder used to store files shared in channel conversations.

The view presented by the Files tab has always lagged the range of options available to users in the SharePoint Online browser interface. In some respects, this didn’t matter. Although SharePoint aficionados know about the missing options, people using Teams might not know that SharePoint manages team files and often didn’t notice. The options available in the Files tab are enough to get the job done, and that’s all that mattered.

But the lack of some features like check-in/check-out bothered some people. The last overhaul addressed many issues, but some remained, like the inability to see the version history and apply retention labels. The latest version of the Files tab (Figure 1) brings some notable enhancements, two of which are particularly worthwhile.

Document Details Pane

First, the details pane is now available for a selected document. This allows you to update metadata for the document and assign a retention label. Apparently, the SharePoint browser interface might soon get the ability to assign a sensitivity label to a document. Today, this can only be done by the Office online and desktop apps (and according to MC395209, soon in the paid-for version of Adobe Acrobat), but when SharePoint gets sensitivity label support, it will probably appear in the details pane and so turn up in the Files tab.

Grid View

SharePoint’s Grid View used to be called Quick Edit. In February 2021, Microsoft renamed the capability to bulk edit items in lists and document libraries. Grid View appeared in the SharePoint browser interface, but not in the Teams Files tab. Now it has turned up, and it works very nicely.

One of the nice things you can do with Grid View is to add a new column easily. For example, the document view we use to organize the chapter files for the Office 365 for IT Pros eBook shows the person who last edited a file but doesn’t show the author responsible for the chapter. Using Grid View (Figure 2), I added a new text column called Author and populated it with the names. The nice thing was that the new column shows up with the populated values in the SharePoint browser interface too.

Not Much Else to Do

I typically use the SharePoint browser interface to work with files. However, the updates to the Files channel tab have made me reconsider whether I should use it. The only thing I use regularly that’s missing is version history, but I only use that feature very occasionally. There’s no sharing option in the view and you can’t add a link to OneDrive. Sharing can be managed through the Details pane (not as easily, but possible) and I seldom create a OneDrive link. So, It seems like Microsoft has not much else to do for the Files tab – except sensitivity labels, of course.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across Office 365. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.

Useful Spreadsheet to Understand Changes

Two weeks ago, I reviewed the new Stream (on SharePoint) client. The client is incomplete, and Microsoft still has lots of work to do to round it out. An online spreadsheet helps customers compare the functionality available in the new Stream client against the classic client, and it’s worthwhile checking out to make sure that your favorite feature is in Microsoft’s plans.

Before reviewing what Microsoft intends to do as it builds out Stream on SharePoint, it’s worthwhile considering the fundamental nature of the change that’s in motion. Microsoft designed Stream classic as a standalone app. The original idea was that Stream could function independently of Office 365 to allow Microsoft to sell it to customers who didn’t have a tenant or allow consumers to use Stream to store video analogous to the way Outlook.com and OneDrive consumer work. Inside Office 365, Stream would replace the old Office 365 Video portal.

Stream Services Rather than a Portal

The app idea is dead. Stream on SharePoint is really a set of services that manage video files stored in SharePoint Online and OneDrive for Business. The obvious manifestation of the services is revealed in the web audio and video player, while the new Stream client is really just an adaptation of the OneDrive client modified to manage video files.

The transition means that there is no video portal. People interact with videos they have access to through the new Stream client, but there’s no organization-wide portal to highlight selected videos or publish material to end users. If organizations want this kind of functionality, they need to use a different approach. For example:

• A SharePoint site tailored to highlight and feature selected videos.
• A channel in a team dedicated to the same purpose.
• Organization videos published through Viva Learning.
• Videos published through a Yammer community.

In other words, there’s more work to do to create an organization-wide video portal. On the plus side, you now have the option to select your preferred approach instead of being limited to a video portal conceived and delivered by Microsoft.

Video Organization

Another related change is in how Stream organizes videos. Stream classic can collect videos into channels and Microsoft 365 groups. These concepts don’t exist in the new Stream because SharePoint organizes its files into sites, libraries, and folders. However, something called SharePoint video collections page is coming soon that appears to take the place of channels. The notion of using Microsoft 365 Groups to organize videos is present because many SharePoint sites are backed by a Microsoft 365 group, but it’s a less direct connection than what happens in Stream classic.

Stream Audit Events

On the compliant front, the Stream-specific audit events logged when users uploaded and viewed videos are no more. Microsoft says that “Audit log schema will change and be logged against the file in ODB or SPO.”

In other words, audit capture for video uploads and other activity is treated in the same way as other SharePoint file operations. When someone uploads a video, SharePoint captures a FileUploaded event; when they modify a video, SharePoint captures a FileModified event. Some Stream-specific events remain, such as those for transcript generation (FileTranscriptCreated), but the majority of the logged events for Stream actions are likely to look like SharePoint file operations.

There’s no harm in using SharePoint file operations audit events until the time comes to extract Stream events from the audit log. You’ll now need to search for the standard SharePoint file operations and then extract the Stream events from that set, probably based on file type (here’s an example PowerShell script I wrote to report Teams meeting recordings).

The problem here is the sheer volume of SharePoint file operations, especially FileModified events. Office documents stored in SharePoint and OneDrive generate vast quantities of these events because of the way the AutoSave feature works, so 5,000 FileModified audit events might only include two or three relevant to Stream. Some won’t care about this change at all, but you will if you use audit events to track video uploads.

Lots of change is in the air. It will be interesting how Microsoft 365 tenants take to the new Stream.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

No Good Way to Clean up Unwanted ex-Teams Channel Folders

Some years ago, I wrote about why Teams leaves the SharePoint folder intact when it deletes a channel. In a nutshell, this is to avoid data loss. The deletion of a channel can be reversed for up to 21 days afterward, and it’s important to be able to restore the complete channel including its files in the Teams channel folder during the 21-day retention period.

But then the question of what happens after the retention period arises. The channel is gone and irrecoverable, but its folder lingers on in SharePoint as a reminder of a now-gone collaborative space. The natural thing for administrators is to clean up the unwanted folder, but that’s not possible because the Delete option is missing from the folder menu. Figure 1 shows an example. There’s no Delete or Move to options in the menu for a folder connected to a Teams channel.

In a Microsoft technical community discussion, some suggest using the Move to option to move the complete folder to somewhere else, like a personal OneDrive account, and delete it from there. Others put their faith in the Move-PnPFolder cmdlet, and some other innovative solutions are offered.

The point is that administrators want to remove the unwanted folders belonging to deleted channels and don’t understand why Teams makes this difficult to do.

Teams Takes Control

After playing around with several channel deletion scenarios in my tenant, it seems to me that when Teams creates a channel folder, it updates the folder properties to remove the options to delete and move the folder. This wasn’t always the case. An unpublicized change seems to have made the change sometime late in 2019. I’ve been able to delete folders belonging to old channels, even immediately after deleting the channel in Teams by following the same approach as used to delete SharePoint folders that have no relationship with Teams:

• Removing all subfolders (delete or move the items).
• Deleting the channel folder.

An example is in Figure 2. In this case, the deletion was of a channel created in November 2016, which is right at the start of the Teams era. The connection with SharePoint Online was looser and this is probably what allowed the deletion to happen. You can see that the menu for this folder includes both Delete and Move to options.

As time went by, Teams became more proscriptive in how it dealt with SharePoint Online. For instance, you can’t modify the settings of the sites used for Teams shared and private channels because Teams will overwrite what you do with settings from the host team.

Prevent Accidents

But I think the reason why Teams doesn’t allow site administrators to delete these folders is to avoid the possibility of data loss both during normal operation and while a deleted channel is in a soft-deleted and restorable state. Removing the Delete option stops an accident happening that leads to data loss and removing the Move to option prevents someone moving files that might be required to restore a deleted channel.

Break the Connection with Teams

What’s missing is a step in the code Teams uses to permanently remove a deleted channel. When the 21-day retention period lapses, Teams cleans up by removing the channel from its soft-deleted cache. It would be good if it also reset the properties of the channel folder to break any connection to Teams and to allow site administrators to manage the folder as normal. In other words, restore the Delete and Move to options in the folder menu and stop telling people they must manage the folder through Teams.

I don’t mind Teams wanting to keep its channel folders under control, but there’s a time to let go, and it’s obvious that Teams hangs on too long in this instance.

Learn how to exploit the data available to Microsoft 365 tenant administrators through the Office 365 for IT Pros eBook. We love figuring out how things work.

Loss of Valuable Feature

In a surprising announcement, Microsoft said in MC394933 (June 24) that they plan to retire the SharePoint Inside Look part of the file preview card. Office 365 tenants will lose the feature starting in late July and the Inside Look will be gone by early August.

I think this is a pity. Inside Look estimates the time required to read a document and an extract of what it believes the author’s key points to be (at a glance – Figure 1). Some background process generated the key points and usually did a reasonable job, at least for documents written in English, which is all the feature supports.

Composing a Custom At a Glance

Sometimes, the process did not work so well, especially for larger documents and no inside look is available. Our main file for the Office 365 for IT Pros eBook is around 33 MB and SharePoint stays mute when it comes to the “at a glance” section. Fortunately, the option exists to create your own “at a glance” by composing three points of up to 100 characters each. The “Edit at a glance” option is available through the […] menu under “See details” in Figure 1, which and reveals an input form to compose the three points (Figure 2).

At a Glance Gone from Sharing Emails Too

In addition, Microsoft is retiring the insertion of the “at a glance” text in the email notification sent when someone shares a document (Figure 3). This is especially regrettable because the text gives recipients some immediate insight into the content within a document.

The Language Issue

Of course, I write documents in English and therefore get value from the feature. The problem might be because Microsoft says that the feature worked exclusively for Word documents written in English. This is surprising because Microsoft certainly has the translation capability to handle other languages. The obvious conclusion is that the issue lies in extracting the three “at a glance” points from the text of a document.

No doubt this is a machine learning task, probably based on something like creating points from sentences at the start of a document where summaries are most often located. I’m sure that scaling this capability up to handle the intricacies of non-English languages plus the resources needed to perform the processing are factors driving Microsoft’s decision to retire the Inside Look feature.

Removing Features is Hard

One thing that’s not clear is if the retirement covers the estimated time necessary to read a document. I’m unsure how Microsoft computes this number but can report that the Office 365 for IT Pros eBook (2022 edition) apparently takes 34 hours to read. Not all at one time, as that would leave you boggle-eyed and incapable of sensible conversation. I’m sure that the computation is based on factors such as the number of pages and words with other influences like the number of paragraphs and headings probably thrown in for good measure.

SharePoint Online has added some good features recently, like the document library drop-down menu and the ability to set a default sensitivity label for a document library (just like you can set a default retention label). It’s a pity to see something like the Inside Look disappear, even if it is English-only and only works for Word documents. Removing features is hard, but the cloud can take away functionality as quickly as new capabilities appear.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across Office 365. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.

Fills Gap in Current Implementation

Updated: August 15, 2022

Message center notification MC393822 (18 June – Microsoft 365 roadmap item 93209, and updated in MC412375 on August 11, 2022) informs tenants about an important change to the way sensitivity label policies apply default sensitivity labels. Up to now, if you define a default label in a policy to apply to documents (Figure 1), SharePoint Online and OneDrive for Business assign the label to new documents created in sites that come within the scope of the policy. MC412375 says that the public preview for the change will roll-out in late August and be available everywhere by mid-October.

Change Applies to Modified Files

The change Microsoft is rolling out in public preview from mid-June is to make sure that when people edit unlabeled (existing) Word, PowerPoint, or Excel files, SharePoint and OneDrive stamp the default label on the file. The functionality already works for the Office web applications and is now extending to Office on Windows and macOS.

Obviously, this is a good change for organizations that want to ensure that all documents have at least a default sensitivity label. Until now, the default label guaranteed that new documents received sensitivity labels, but this left a huge gap in terms of all the files created prior to the implementation of sensitivity labels.

Auto-label policies help close the gap because background processes can scan sites for documents and apply labels to the files if they don’t already have a label. The problem is that auto-label policies are a premium feature. However, if you have the necessary licenses, auto-label policies are a good way to achieve coverage of a large number of preexisting files.

Another change that’s coming soon is the ability to configure a default sensitivity label for a document library, much like you can do with retention labels. Again, this is a premium feature and it’s likely to require Office 365 E5 or Microsoft 365 Compliance E5 licenses.

API to Bulk Apply Sensitivity Labels

One missing piece in the puzzle is the lack of an API to allow organizations and ISVs to create applications to apply sensitivity labels in bulk. Microsoft’s AIP Scanner is an example of such an application. The scanner can apply sensitivity labels to protect information found on file shares or SharePoint on-premises sites. Other use cases include tenant-to-tenant migrations where the need might exist to apply sensitivity labels to a set of documents inherited from a tenant belonging to a company being acquired. There’s nothing off-the-shelf that can handle such a scenario today, and the prospect of having to apply labels manually is unattractive.

Apparently, an API is coming, but it will be a paid-for consumption-based API like that available for Teams Export. In other words, you’ll be able to build an application to apply sensitivity labels to a bunch of files (probably throttled at a certain level to reduce strain on the service), providing you have an Azure subscription to pay the bills.

Making Sensitivity Labels Mainstream

Sensitivity labels are still relatively uncommon inside Office 365 tenants. Microsoft is the only source that can definitively say what percentage of tenants use sensitivity labels or how much of their content have labels. Changes to allow tenants apply sensitivity labels more effectively by default, or to spread sensitivity label support more widely (like the work done to make it easier to protect PDFs) help to encourage more organizations to consider sensitivity labels to be a mainstream part of their overall information protection strategy. However, it’s still going to take time before sensitivity labels become the norm inside Microsoft 365.

Stay updated with developments across the Microsoft 365 ecosystem by subscribing to the Office 365 for IT Pros eBook. We do the research to make sure that our readers understand the technology.

Update the App Tile to Point to the New Stream

Message center notification MC381948 appeared on May 18, but I confess to not having paid much attention to it due to other more important topics. In any case, the notification informs tenants that they can configure the target for the Stream tile in the Office 365 app launcher to direct users to Stream powered by SharePoint rather than the classic Stream. Targeted release tenants should have the update now, with standard release tenants getting it in early July. By late August, all tenants should be able to update the app tile.

Classic Stream stores its video files in Azure blob storage. The plan of record is to move everything to SharePoint-based storage. Personal videos will be in OneDrive for Business while shared (group videos) will be in the document libraries of the SharePoint Online team sites belonging to Microsoft 365 groups (teams). Progression is already obvious as all newly-created Teams meeting recordings are now in the new location. Using SharePoint-based storage means that many features become available for videos, like the application of expiry dates for meeting recordings. In addition, OneDrive for Business offers a lot more storage quota for videos.

Migration a Work in Progress

The big piece of work remaining for Microsoft to do is the migration of old video content from classic Stream. The latest information is that Microsoft has the migration tool in private preview with some customers. Dates for when the migration tool will become generally available are unavailable.

Diverting the App Tile to the New Stream

In the interim, tenants might find it useful to divert users away from the old Stream and have them upload any new video content into OneDrive for Business. And that’s where MC381948 comes in. A new setting is available in the SharePoint admin center to control the behavior of the app tile for Stream. Three values are available

• The default option is to Automatically switch to Stream (on SharePoint). Microsoft controls this option and will set it after the migration of existing Stream content is complete.
• Stream (on SharePoint) directs users to the preview GUI for the new Stream. The user can switch to the classic Stream GUI if they want.
• Stream (Classic) forces people to use the classic Stream GUI.

In Figure 1, I chose to switch to the new Stream. After saving the choice, it takes about ten minutes for the option to ripple across the tenant (and maybe a browser refresh, just in case).

In effect, the target URI for the app tile changes from https://web.microsoftstream.com/ to https://www.office.com/launch/stream. The new GUI (Figure 2) displays any video files found in the user’s OneDrive for Business plus any video attachments for Outlook messages. This ability to highlight video attachments leverages the new messages search vertical and highlights the role of the new Stream in managing video content stored anywhere in Microsoft 365 instead of just in a dedicated repository.

Feel the Power of an Updated App Tile

There’s not much more to say about the new option (nor any nuggets to glean from the Microsoft documentation). On the one hand, it makes sense to begin using the new video storage and management platform for new content. On the other hand, you can argue that it’s best to keep all video content in one place until the migration is ready. The fatal flaw in that argument is the storage of Teams meeting recordings in OneDrive for Business. I switched to embrace change. What will you do with your app tile?

Keep up with the changing world of the Microsoft 365 ecosystem by subscribing to the Office 365 for IT Pros eBook. Monthly updates mean that our subscribers learn about new developments as they happen.

View Teams and Outlook Messages in Search Results

Microsoft Search and the results it delivers to users are in a state of constant flux. This is usually a good thing because it means that Microsoft is upgrading search capabilities to help users find information more effectively. Sometimes, things get out of step, and you can see extra results in one place that don’t appear in another. A little consideration usually comes up with a reason why this is so.

Take the example of the Messages vertical that Microsoft has added to Office.com. When you search from Office.com, the results include Teams and Outlook messages (Figure 1). In search parlance, the set of results exposed by the messages tab is referred as a “search vertical.” You can add custom search verticals to SharePoint search, but not to Office.com.

The Teams messages come from both chats and channel conversations. Selecting a Teams or Outlook message uses a deeplink to bring you to the source loaded in the Teams client or OWA.

Microsoft Search trims the search results so that users only see information from resources they have permission to access.

Why Messages from Deleted Teams Appear in Search Results

Sometimes search results resurrect messages from deleted groups. Take the second message listed in Figure 1, which comes from a conversation in the Project Athena group (a team). Selecting this message does nothing because it doesn’t have a deeplink to bring it to the source conversation.

Some investigation found that the team doesn’t exist anymore. I deleted the team since the conversation happened in 2018. However, the messages persist because the team came within the scope of a hold imposed by a retention policy. Microsoft Search relies on the compliance records the Microsoft 365 substrate captures for Teams chats and channel conversations, and these records remain in mailboxes until the retention period for the policy lapses. Therefore, the conversation remains available for search to find while the deeplink pointing to the source conversation is unavailable.

Microsoft Search in Bing

The interesting thing is that the ability to return messages in search results isn’t available in SharePoint search. You might expect this to happen because it’s a search for Microsoft 365 data. However, it’s a search of SharePoint resources, so the results only cover the information available to SharePoint Online and OneDrive for Business. Personally, I think Search should deliver the same results in SharePoint Search as it does in Office.com, even if SharePoint Online doesn’t manage the items found. The lines between applications continue to blur and it seems strange to have artificial barriers where they’re not needed.

Where messages do turn up is in search results from Bing.com if you configure Microsoft search in Bing through the Search & Intelligence section of Org settings in the Microsoft 365 admin center. In effect, when you do this, you connect Microsoft 365 content to Bing to expose “work” results alongside results for internet sources. Accessing the work tab exposes results from different Microsoft 365 sources, including messages (Figure 2).

This capability has been available for at least six months. At least, we updated the coverage about Microsoft Search in the Office 365 for IT Pros eBook about six months ago to report its availability!

Loop Components in Search Results

While looking at the various results now available through Microsoft Search, I noticed that Loop components show up. I probably missed this in the past but felt that it’s worth noting that even though Loop components pose some eDiscovery challenges, the information in the components is fully indexed and discoverable as evident in the first two search results shown in Figure 3.

There’s nothing surprising here because the Loop components in Teams chats (and soon in OWA messages) exist as files in OneDrive for Business.

Nice to See Messages in Search

Given the amount of data people now store in the cloud, effective search facilities are increasingly important. Adding the new search vertical for messages to Office.com is very useful. It’s just a pity that the same capabilities aren’t available elsewhere.

Storage and Sites Grow Strongly

At the recent Microsoft 365 Conference (April 5-7, Las Vegas), Microsoft CVP Jeff Teper spoke about Hybrid Workplace Innovations (see the video posted on YouTube). This event is rooted in the SharePoint Conference and the event is heavily supported by the SharePoint development group because of the audience it attracts. In that respect. Teper was talking to base, which he labeled as the “best community in tech.” I guess other communities would debate that point, but when you play to your audience, you need to make them feel good.

In any case, two interesting points about SharePoint Online growth are in the video. First, Teper said that SharePoint Online ingests 100 petabytes of new content monthly (Figure 1). That’s a lot of new storage for Microsoft to add to their data centers every month. But it’s not all documents, and it’s not all for SharePoint Online because OneDrive for Business is in the mix.

Storage Buckets

Microsoft doesn’t say what activity consumes storage. Thinking about where such a large quantity of new content comes from is an interesting exercise. I think the content can be divided into these buckets:

• User activity to create and update documents stored in SharePoint Online sites and OneDrive for Business accounts. This kind of growth is evidence that organizations have moved from older file shares and that people are comfortable with storing their personal files online rather than on local disks.
• Retention policies and labels keeping copies of documents for set periods. Unlike Exchange Online, SharePoint Online charges storage used for retention purposes against organizational storage quotas. In some cases, tenants might use over 20% of their SharePoint storage for retention (Figure 2).
• The effect of Microsoft Lists is probably small on such a large number, but Microsoft has put considerable effort into publicizing Lists and this effort will have some impact, especially for lists storing graphic elements.
• Transition of application storage from Azure to SharePoint Online and OneDrive for Business. Stream is still in mid-transition and Whiteboard is the latest application to move. Teams meeting recordings are the big application consumer of storage at present.
• The storage of Loop components in OneDrive for Business. Because Loop components are available only in Teams chats (soon in OWA), this has minimal impact.

Teams Meeting Recordings

In April 2020, I looked at how much storage a Teams meeting recording consumes and concluded that a recording generates about 7.65 MB per minute. At the time, the videos were in Stream’s Azure-based storage. More recent videos stored in OneDrive for Business appear to consume approximately the same amount of storage.

Microsoft hasn’t given numbers for Teams meeting minutes and users since April 2020 when 75 million Teams users generated 4.1 billion minutes per month, or 55 minutes per user.

The last number for Teams users is 270 million. Applying 55 minutes per user, we get current generation of 14.76 billion. Not every Teams meeting is recorded. Let’s say that 30% are, which gives 4.428 billion minutes requiring 33,210 million MB, or 33.82 petabytes (using this calculator).

Based on these numbers, Teams meeting recordings might account for a third of the monthly storage growth in SharePoint Online. With that kind of consumption, it’s no wonder that Microsoft has implemented a default 120-day expiration period for Teams meeting recordings.

Growth in Sites

The other statistic that took my interest is the creation of 8 million new sites per month (Figure 3). Teper was careful to emphasize that these are active sites and don’t include sites created for testing, development, etc.

Eight million new sites is a big number, but again it’s very understandable in the context of the size of Office 365. The latest Microsoft number for Office 365 users is 345 million paid seats, so eight million is not a big number when placed against the size of the overall base.

The ongoing growth in Teams and the way that Teams creates new sites for each team and for new private and shared channels drives a lot of site creation. Again, Microsoft didn’t give a detailed breakdown of the types of sites that Office 365 tenants are creating so we can only make guesses about what’s happening. What’s for sure is that demand for SharePoint Online services and storage is rising strongly to reflect its status as one of the core workloads in the suite.

Learn about SharePoint Online and the rest of Office 365 by subscribing to the Office 365 for IT Pros eBook. Use our experience to understand what’s important and how best to protect your tenant.

Just in Time for Outlook

Updated: March 22, 2023

Microsoft Loop components have been available in Teams chat since November 2021. I haven’t heard about widespread usage, but that might be because people need time to adjust their collaboration habits. Access to Loop components in other applications is also a gating factor, but availability in OWA and Outlook for Windows (current channel preview) should help to address this concern. According to MC360766 (April 18, Microsoft 365 roadmap item 93234), Microsoft will roll out this feature to tenants configured for targeted release in early May.

Update: It took a little longer than predicted, but Loop components are now available in OWA.

So far, there’s no sign of Loop components in Outlook desktop, but I’m sure the components will arrive in my email any day now to deliver the same kind of functionality as available in Teams chat (Figure 1). In a nutshell, if an email contains a loop component, it exists as a file in the sender’s OneDrive for Business account that is shared with the email’s recipients. We’ll report more when the software is available.

IsLoopEnabled

This brings me to MC371268 (May 2), where Microsoft announces that “in response to customer feedback,” they’re retiring the existing settings to control the availability of Loop components and introducing a new control called IsLoopEnabled.

The control is part of the SharePoint Online tenant configuration and is set using the Set-SPOTenant cmdlet. You’ll need to upgrade the SharePoint Online management module to version 16.0.22413.12000 or later. Microsoft posted this version in the PowerShell Gallery five days ago. You can install or update the module from the PowerShell gallery or download an MSI file from Microsoft.

The replaced control is IsFluidEnabled, which enables the Fluid Framework within a tenant. Microsoft plans to retire the IsFluidEnabled setting on November 25, 2022. Going forward, the relevant settings in the SharePoint Online configuration are:

• IsLoopEnabled: Controls if Teams can use Loop components. The default is True (Enabled).
• IsCollabMeetingNotesFluidEnabled: Controls if fluid components are available in OneNote collaborate meeting notes.

Update: Following the availability of the preview version of the Loop app, the control for the Loop app, Outlook, Whiteboard, and the Office Online apps is via settings in the Cloud policy.

eDiscovery and Compliance Issues

Although eDiscovery searches can find Loop component files stored in OneDrive for Business, Microsoft acknowledges “limited eDiscovery workflow support.” With the additional of Loop support in Outlook, this aspect might become more problematic. For example, today, the preview feature for search results can render the full content of emails. This isn’t possible when an email contains a loop component because the preview window needs a software upgrade to fetch the content from OneDrive and display it inline within a message.

Another issue is with exports of search results. Today, Microsoft Purview can export emails (and the compliance records captured for Teams chats) found by searches as individual message files or in PST files. Microsoft says that the export format is “not consumable by existing tools,” and that they’re working on “an offline consumable export format.” Taken together, these statements make me think that the exported emails contain references (links) to OneDrive files that aren’t accessible to investigators working offline or independent experts who review eDiscovery results without access to the source tenant.

Making the content of search results available offline probably involves replacing the embedded link in messages containing Loop components with a static version of the content extracted from OneDrive.

This topic deserves a more comprehensive test, which I will get to once Outlook support for Loop components is available. In the meantime, organizations that don’t want to run into potential eDiscovery problems should strongly consider disabling Loop components for both Teams and Outlook by setting the IsLoopEnabled control to False.

Set-SPOTenant -IsLoopEnabled $False

Stay updated with developments across the Microsoft 365 ecosystem by subscribing to the Office 365 for IT Pros eBook. We do the research to make sure that our readers understand the technology.

Advanced Setting Manipulated by PowerShell

For the last year, Microsoft has steadily added to the ability of sensitivity labels to manage different aspects of SharePoint Online sites. Possibly because of a desire not to clutter up sensitivity label settings in the GUI, the developers chose to manage the settings via PowerShell. Adding to the ability to manage the external sharing capability and default link settings, administrators can now control site sharing permissions (a preview feature) via a new advanced sensitivity label setting.

In the SharePoint Online browser interface, this option is available through Site Permissions – Site Sharing (Figure 1).

It’s possible to set site sharing permissions to block all but site owners with PowerShell by running the Set-SPOSite cmdlet with the DisableSharingForNonOwners switch. For example

Set-SPOSite -Identity https://office365itpros.sharepoint.com/sites/Office365Adoption -DisableSharingForNonOwners

However, the Set-SPOSite cmdlet doesn’t allow administrators to enable site sharing for non-owners. It’s a very simple off switch that cannot go back or set site sharing permissions to the option where only site owners can share the site. The new capability for sensitivity labels delivers a way to address these shortcomings, but only for sites assigned sensitivity labels with the advanced setting defined.

Available Site Sharing Permissions

Three site sharing permissions settings are available (the descriptions are from the GUI shown in Figure 1):

• MemberShareAll: Site owners and members can share files, folders, and the site. People with edit permissions can share files and folders. This is usually the default setting assigned to new sites.
• MemberShareFileAndFolder: Site owners and members, and people with edit permissions, can share files and folders, but only the site owners can share the site.
• MemberShareNone: Only site owners can share files, folders, and the site.

Updating the Site Sharing Permission

To assign a new site sharing permission, connect to the compliance endpoint by first connecting to Exchange Online (Connect-ExchangeOnline cmdlet) and then running the Connect-IPPSSession cmdlet. You then have access to the compliance cmdlets and can run the Set-Label cmdlet to update the MembersCanShare advanced setting. For example:

Set-Label -Identity 'General Access' -AdvancedSettings @{MembersCanShare= 'MemberShareFileAndFolder'}

To ensure that the update worked, run the Get-Label cmdlet:

Get-Label -Identity "General Access" | Select-Object -ExpandProperty Settings

[contenttype, Site, UnifiedGroup]
[tooltip, General access to information in a team, group, or site that's available to anyone in the organization plus guest members.]
[displayname, General Access]
[memberscanshare, MemberShareFileAndFolder]

Note that the Get-Label cmdlet only lists advanced settings that apply to a sensitivity label. For instance, the external sharing capability setting doesn’t appear here because it is not set for the General Access label.

Wait and Verify

The new label setting must propagate to SharePoint Online before it applies to the sites assigned the sensitivity label. The synchronization process usually takes about 24 hours, but it can take longer. After waiting for a day or so, to verify that the change worked, select a site with the sensitivity label you updated and check its site sharing permissions. Because we selected ‘MemberShareFileAndFolder’ as the value for the setting, you should see permissions as shown in Figure 2.

If the permission doesn’t show up as expected, check that the label settings are correct and wait another day before checking again. If nothing budges after a week, it’s time to seek assistance from Microsoft Support.

GUI Updates Take Time

Some will ask why Microsoft doesn’t expose advanced sensitivity label settings in the (now renamed) Microsoft Purview compliance portal. After all, many settings are managed through sensitivity labels in the GUI, including external sharing capability (Figure 3). This setting was originally only settable through PowerShell.

Although I don’t know for sure, I suspect that the answer is “development time.” In other words, after a new sensitivity label setting becomes generally available, extra development effort is necessary to update the GUI and make sure that everything works as it should. Patience is a virtue…

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across Office 365. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.

And the Joined Teams Feature That Should Have Been Available Sooner

Sometimes Microsoft releases a feature that is so obviously useful that you wonder why it never existed earlier. SharePoint’s document library dropdown menu is in this category. The unfortunate thing is that the feature has arrived too late.

Announced in MC301473 (updated February 3, Microsoft 365 Roadmap item 81990), the dropdown menu is generally available. When a site contains multiple document libraries. You see a dropdown menu to navigate between the different libraries. Take the example in Figure 1. We’re positioned in the default Documents library, but the site has four other document libraries. The dropdown menu makes it very easy to navigate from one library to another.

Will People Use the Document Library Dropdown Menu?

Despite the dropdown menu being generally available, my guess is that many people will never use the menu. This feeling is based on the experience that most SharePoint Online sites created today are linked to Teams and only have the default document library. Although the dropdown menu is available in these sites, it’s not very useful (Figure 2). It would be better if SharePoint Online hid the menu in these sites.

The other thing is that if a large proportion of SharePoint activity is via Teams, people don’t ever go near the browser interface and therefore won’t see the document library dropdown. MC301473  says that Microsoft is planning work to bring the dropdown to Teams, probably with an update to the Files channel tab. They haven’t shared a date for this work, but when it is delivered, Teams users will be able to switch between the available document libraries in a team-enabled site.

Teams Admin Center Shows User-Joined Teams

The Teams feature announced in MC332869 (February 15, Microsoft 365 roadmap item 87969) is another example of something that should have been in a product a long time ago. In this case, the message center notification tells us that the Teams admin center can now show the list of teams a user account is part of (Figure 3).

This is welcome news because there are many examples where tenants have written their own reports to list what teams (or Microsoft 365 groups) people belong to (here’s my version of a PowerShell script to create such a report). In fact, some like the idea of being able to print off the membership of a specific group or team (so here’s a script to do that job).

My point is that the necessary support to list the set of groups or teams a user belongs to has been around in the Graph transitiveMemberOf query for quite a while. Shown below is a simple code example which uses the Get-MgUser and Invoke-MgGraphRequest cmdlets from the Microsoft Graph PowerShell SDK to retrieve the set of groups a user belongs to. A filter extracts the set of teams, which we can then list. In the Teams admin center implementation, the code uses the team identifier to fetch details like its privacy, active state, and so on for display.

$User = Get-MgUser -UserId James.Ryan@office365itpros.com
$Uri = "https://graph.microsoft.com/v1.0/users/" + $User.Id + "/transitiveMemberOf"
[array]$UserGroups = Invoke-MgGraphRequest -Uri $Uri -Method Get
[array]$UserTeams = $UserGroups.Value | ? {$_.resourceProvisioningOptions -eq "Team"}

$UserTeams | ForEach{Write-Host $_.Id, $_.DisplayName}
34d68904-9d7c-4ef7-b715-eed283e80243 Industry News
c055da06-f21d-4381-9c51-f5a239d36329 Plastic Production (Team)
204e3211-4d07-4fde-95f9-227a8827742d Organization Planning (Team)
18aa8f1b-3bdf-41f7-b14b-a3be217478e8 Baden Workers
5348781d-52a8-490f-b75b-a72e702114d1 PL Test

In any case, the feature is now rolled out and available in your closest Teams admin center.

Fit and Finish Features

Microsoft will likely say that they work on features in priority order and items like a document library dropdown menu are not high on that list. That’s true, and it’s unfortunate when features show up long after they would have been useful.

Small but important features like the document library dropdown menu and revealing the set of teams someone belongs to are fit and finish items. No one will decide to use SharePoint Online or Teams because these features exist, but current users of these products will like that they do.

Keep up to date with new features in SharePoint Online and Teams by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers understand the most important changes happening across Office 365.

Perhaps Not the Biggest Problem for OneDrive to Solve

Featured in the set of OneDrive announcements at the Microsoft Ignite conference in November 2021, the ability to move a OneDrive shortcut from the Files root to a public or shared folder is now rolling out. The change is described in message center notification MC316147, first published on January 19 and updated on March 4.

The original announcement limited movement to private folders, and this is also the case in Microsoft 365 roadmap item 82166. However, something obviously changed since November because MC317147 explicitly states “when moving a shortcut to a folder into a shared folder, the short cut does not change its sharing permissions. People who don’t currently have access to the shortcut won’t be able to access its content but can rename or remove the shortcut.”

OneDrive Shortcuts

Originally launched in 2020, OneDrive shortcuts are a useful way to add pointers to folders that users commonly access so that they appear in OneDrive for Business. The shortcuts might be to folders in SharePoint Online document libraries or other OneDrive folders. When OneDrive shortcuts first appeared, I thought they were pretty good and used them for a while, but then I ran into a problem that still lingers today.

The OneDrive sync client is a critical component for users who keep files in the cloud. The sync client synchronizes files from cloud folders to local copies, and that updates made to the local copies synchronize back to the cloud. The original OneDrive sync client (Groove.exe) wasn’t very good, but a rewrite to create a new client fixed the problems and the current client is very stable. Interestingly, while the OneDrive sync client takes care of synchronization for non-Office files, to enable features like autosave and co-authoring, the Office apps perform the synchronization when actively working on documents.

The Office 365 for IT Pros eBook team depend heavily on the OneDrive sync client to synchronize changes made to the source Word documents used for book chapters. These and other files for the book are stored in a SharePoint Online document library. The OneDrive sync client makes sure that changes made by authors on Windows and Mac workstations synchronize with SharePoint Online.

Synchronization Problems with OneDrive Shortcuts

Which brings me to the synchronization problem with OneDrive shortcuts which stop me using shortcuts. Everything works well if you create OneDrive shortcuts and then set up synchronization with SharePoint Online. However, if you use the OneDrive sync client to synchronize both OneDrive for Business and SharePoint Online folders and then add a OneDrive shortcut to a folder in the same document library, it creates a sync issue.

Figure 1 shows a SharePoint Online folder in a document library. I don’t synchronize this folder to my workstation because it contains large book files. However, I synchronize other folders from the library. I also synchronize my OneDrive for Business account.

If I take the option to add a shortcut to OneDrive, SharePoint Online creates the shortcut and adds it to OneDrive for Business (Figure 2). Everything looks good and I can use the shortcut to access the files in the SharePoint Online folder.

However, the OneDrive sync client reports that it has a sync issue (Figure 3) saying that it cannot sync the shortcut because it conflicts with other folders. The client reports that the fix is to stop syncing two folders, both of which come from the same SharePoint Online document library.

The sync client offers to fix the problem by unsynchronizing the conflicting folder. Do not do this. The action breaks the connection between the local copy on the workstation and the cloud files, which means that you’ll have to re-establish synchronization afterwards, which could involve a lot of work to make sure that local copies are accurate.

However, the issue is only a warning about a single file (the OneDrive shortcut) and doesn’t affect synchronization for any other file. Changes made locally continue to upload to the cloud and updates made to cloud files by other workstations flow down to the local copy on my workstation.

The solution is simple. Go back to OneDrive for Business and remove the offending shortcut. The sync client is happy immediately and the warning disappears.

The problem doesn’t occur if you create a OneDrive shortcut to a SharePoint Online folder when no folders from that document library are synchronized. However, if you attempt to synchronize a folder from the document library, OneDrive fails and says that it can’t synchronize the folder because you’re already syncing a shortcut to a folder from this shared library (Figure 4).

I can’t imagine that this is the kind of experience that Microsoft would design into OneDrive shortcuts. What’s more, the problem has been in place since the introduction of shortcuts, so perhaps no one has complained too much.

Moving of Shortcuts Not The Biggest Problem

The clash between OneDrive synchronization and OneDrive shortcuts is the reason why I won’t use shortcuts. Although it’s great that Microsoft has done the work to make it possible to move shortcuts, it’s odd that they haven’t sorted out the obvious clash between two OneDrive components. When they do, I’ll consider using shortcuts again.

Make sure that you’re not surprised about changes which appear inside Office 365 applications by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers stay informed.

It’s All About the Substrate

I must be slowing down. At least, that’s the thought which ran through my mind as I tried to make sense of Microsoft’s post about SharePoint Online proxy addresses and Exchange Online mailboxes. Specifically, I couldn’t understand this sentence “To ingest SharePoint Online content into a mailbox, we establish SharePoint Online routing information to the mailbox.” This sounds awfully like the way site mailboxes worked, but thankfully those abominations are long gone. And then I realized that the text wasn’t as clear or precise as it could have been, despite discussing an interesting aspect of the Microsoft 365 ecosystem. Here’s what I think Microsoft meant to say.

The Microsoft Substrate and Digital Twins

As anyone who’s listened to Microsoft Fellow Jeffrey Snover talk about the Microsoft 365 substrate knows, the substrate plays a key role in making Microsoft 365 shared services work. The substrate is what captures compliance records for Teams, Planner, and Yammer. It handles the ingestion of audit records generated by multiple workloads. And the substrate creates “digital twins” of SharePoint Online and OneDrive for Business documents and lists. A digital twin is not necessarily a full copy of an item; it’s enough to allow shared processes to operate against the data. If access is required to the complete data, a link redirects to the owning workload.

The substrate does this work because assembling digital twins gathered from across Microsoft 365 workloads into one place makes it much easier for shared services like compliance processing or search to operate. Instead of a service needing to communicate with multiple repositories, it needs to deal with one. And the physical representation of that repository is a special form of Exchange Online mailboxes.

SharePoint Online Proxy Addresses

Which brings me back to the subject of the blog point: the SPO (SharePoint Online) proxy addresses stamped on user mailboxes. If you examine a mailbox, you see the proxy addresses assigned to the mailbox. For example, four proxy addresses exist for this mailbox:

DisplayName    : Steve Gippy (Operations)
EmailAddresses : {SPO:SPO_20876de2-3b1c-44ce-8773-34499caaa16c@SPO_a662313f-14fc-43a2-9a7a-d2e27f4f3478, 
SIP:steve.gippy@office365itpros.com, 
SMTP:Steve.Gippy@office365itpros.com, 
smtp:Steve.Gippy@office365itpros.onmicrosoft.com}

One is the primary SMTP address used for email routing (the one with capitalized SMTP), another is a secondary SMTP address belonging to the service domain for the tenant. Then there’s the SIP address used by Teams for calls and meetings. And finally, there’s SPO, the SharePoint Online proxy address, which means nothing to anyone because this address is created and maintained by background Microsoft 365 processes. The address includes a unique identifier for the user and the tenant identifier.

As the post says, administrators should leave the SPO addresses alone as “several internal cloud processes rely on them” not to mention that “Admins should never modify the SharePoint Online proxy address as it is an internal Microsoft service concept.” In other words, keep your greasy hands away from SPO proxy addresses. If you don’t, things break, and you won’t be able to fix them. In fact, you probably won’t know what broke and where it broke.

Without the SharePoint Online proxy address in place, the link between Exchange Online and SharePoint Online is broken, and the substrate can’t ingest digital twins from SharePoint Online into Exchange Online. In other words, the SharePoint Online proxy address stamped on user mailboxes is a connection back to SharePoint Online (and OneDrive for Business).

Hard and Soft Deletes

Now the opening of the post makes sense. It discusses why administrators see mailbox objects they believe are permanently removed (hard deleted) persist in a recoverable (soft deleted) state. After all, if you run the Remove-Mailbox cmdlet and use the PermanentlyDelete switch to tell Exchange Online to erase all trace of a mailbox, you’d like to think that the service would do your bidding.

But because Exchange Online is the foundation for the Microsoft 365 substrate, it has more to do than simply blow away a mailbox. In particular, because the search results generated by Microsoft search depend on mailbox content, some adjustment is necessary to reflect a mailbox deletion. That’s why Exchange Online signals SharePoint Online so that background processing can adjust the search results shown to users. While this processing proceeds, it’s possible to see erroneous results featuring a deleted user, but eventually processing completes and search is 100% accurate again.

Exchange Online keeps the mailbox in a soft-deleted state until the deleted mailbox retention period expires (183 days). By then, background processes have adjusted indexes and SharePoint Online is content. Exchange Online can then tidy up by hard-deleting the mailbox, unless of course it’s under the control of a retention hold (litigation hold or otherwise), in which case the mailbox is inactive and kept until all retention holds expire.

Life is More Complicated in the Cloud

All of this proves that cloud objects lead a more complicated existence than on-premises objects. The Microsoft 365 substrate connects objects together in a way that simply doesn’t exist on-premises, so when you remove an object, it might just have an effect elsewhere that must be dealt with. Which is why some mailboxes that you might want to hard delete have to stay soft-deleted until background processes can adjust connections.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Saturated Colors and New Designs for Teams Emojis – Windows Emojis Useful for Channel Naming Too

Last year, Microsoft refreshed the set of emojis designed for use in its applications, lauding the “bright, saturated colors and bold forms” available for more than 1,800 emojis. The new set is already available in Windows and OWA, with plans in place to bring the new emojis to Yammer and other places within Microsoft 365 during 2022. Of course, you don’t need an explicit insert emoji option to use the Windows emojis. In any editor and in most places where you can input characters, just press the Windows key and period and you should be able to browse the emoji set and insert whatever you like.

New Animated Emojis for Teams

Which brings me neatly to the latest Teams public preview feature: Fluent emojis (“a modern and delightful new version of the emojis we use every day” are available for use in Teams chat and channel conversations. The new emojis replace the set last refreshed in April 2021, which increased the number of available emojis to 800. Adding another thousand is an impressive graphical feat, and Microsoft says:

• Emojis in chat and channels messaging will update to the new Fluent style, along with delightful animations for applicable emojis.
• Reactions in chat, channels, and live meeting reactions will update to the new Fluent style.

Animations are the difference between the Windows emojis and those now available in Teams, which is why I guess some refer to 2D emojis (Windows) and 3D (Teams). With so many emojis to choose from (Figure 1), no doubt some conversations will now be conducted entirely through emojis (I’m not kidding).

Everyone will have their own favorite emoji. Mine is surely a candidate to become a new favorite to communicate a state of mind or opinion of a piece of work. The pile of poo emoji looks innocuous when inserted into a message (Figure 2), but the brown smelly material becomes animated when viewed by the recipient. It makes you think about the thought process necessary to come up with a suitable animation for such an object…

Not every emoji is animated, and some animations are very subtle (like the moving legs on the prawn emoji). You can see which emojis are animated by hovering over emojis when browsing the set in the picker.

The new emojis are available for Teams desktop, browser, and mobile clients, but user accounts must be enabled for Teams preview before they’ll be able to add the new emojis to chats and channel messages.

Using Emojis as Chat Reactions

According to MC296204 (updated February 8), people will soon be able to use the new emojis as reactions to chat messages. Microsoft 365 roadmap item 88080 says that users will be able to select any emoji as a reaction to a chat message (but not a channel conversation). The roadmap item lists review for April 2022 and general availability starting for standard release tenants in May 2022.

Update: According a May 26 update to MC296204, Microsoft is pausing plans for this update to make some changes.

Emojis in Channel Names

Another good use of emojis is to highlight important channels. Given that an individual team can have up to 200 regular channels and 30 private channels (with shared channels coming soon), there’s no doubt that it’s easy to overlook channels in a list, even when team owners pay attention to giving channels appropriate and helpful names.

Increasingly, I see people using emojis in channel names. Usually, the emoji is at the start of the name to make it more visible and catch the eye, but I’ve seen emojis placed at the end too. For instance, the channel for discussions about the current version of the Office 365 for IT Pros eBook has a jolly roger (skull and crossbones) emoji to display its important status (Figure 3).

You can’t use Teams emojis in channel names but the Windows emojis are available, so there’s no shortage of choice. In any case, if you use emojis in channel names, it’s best to put the emoji at the start of the channel name rather than the end to make sure that the selected highlight is always visible. As evident in Figure 4, channels highlighted in this manner stand out from other channels.

You can also include emojis in the display names of teams and Microsoft 365 groups, but maybe that’s going too far.

Ensuring SharePoint Online and Teams Agree

In the past, renaming a channel has wreaked havoc with the folder in the SharePoint document library associated with the channel. This month, Microsoft released the update titled Pairing naming convention between Teams channels and corresponding SharePoint folders (described in MC306666, December 18, Microsoft 365 roadmap item 72211).

Microsoft has been working on the issue for some time and originally planned to release the update last year. The fix makes sure that the SharePoint folder gets the same name following the rename of a regular channel. For private channels, both the site name and folder are updated. All of which means that you can include emojis in channel names without any qualms.

Keep up to date with even small changes by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers understand the most important changes happening across Office 365.

Search in Outlook Has Never Been Great

On January 11, MC313286 brought the news that Outlook searches might return no result if messages are stored in PST and OST files. I’ve zero sympathy for those who store email in PST files, but the loss of search in OST files handicaps offline operation for those of us who keep email in Exchange Online mailboxes. I realize that some persist in using POP3 and IMAP4 to access mailboxes (hopefully, the loss of basic authentication in October 2022 will stop this), but it’s time to move on use more modern messaging protocols.

In any case, the problem affects people who upgrade PCs to Windows 11 because the upgrade removes the search index. Over time, Windows rebuilds the search index, and all is well. At least, it’s as well as Outlook searches ever are. Over the years, my expectation that Outlook delivers reliable search results has never been high, so my level of disappointment is never severe. To be fair, searches performed by latest version of Outlook desktop (click to run) are better than before, but force of habit makes me depend on OWA when I need to search for something.

New Search Capabilities Include Outlook and Teams

Behind the scenes, Microsoft Search powers the search facilities in Outlook and OWA. Microsoft Search indexes and can search the Microsoft 365 substrate, meaning that it can find documents, email, tasks, and the compliance items for Teams, Planner, and Yammer. Recently, Microsoft upgraded the search UI in Office.com and SharePoint Online to add a “Conversations” tab to search results. This tab reveals Teams and Exchange Online messages (Figure 1) while other tabs deal with news, people, sites, files, and so on. The change is documented in MC299210 (last updated December 8) and Microsoft 365 roadmap item 68779.

If you select an item, a deeplink takes you to the original message in the underlying workload. For example, if you find a Teams message you want to see, the deeplink offers to open the Teams browser client but will open the item in the desktop client if that client is available. Outlook items open in OWA.

According to the roadmap item, the new search became generally available in January 2022. It should therefore be available in all tenants now.

Microsoft 365 Search in Bing Now Covers Outlook

The roadmap item refers to Bing.com too, which covers the scenario when Microsoft 365 results are integrated with results from Bing searches. It’s long been possible to see Teams and Yammer messages in Bing results. Now Outlook messages are included (Figure 2). As in other features powered by Microsoft Search, filters make sure that the person performing the search only sees the information they can access. This means that a search covers the user’s own mailbox but won’t reveal items in shared mailboxes or other user mailboxes they have delegate access to.

The presentation of Outlook content differs in Bing. In the past, Bing had a Conversations tab covering Teams messages and Yammer. Now, Teams and Outlook show up under Messages and Yammer is moved out to its own tab. I’ve heard speculation that this is because Yammer messages are slower to index. Curiously, the search results available in neither SharePoint Online nor Office.com include Yammer content, so perhaps Microsoft is doing some work to integrate Yammer better.

Integrated View is Best

The obvious advantage of using Office.com or SharePoint Online for searching is access to integrated results. OWA delivers good results for Outlook messages. However, given that we live in a world where communications aren’t restricted to email, the integrated search across SharePoint, OneDrive, Teams, and Outlook is very attractive. It’s now my favorite way to look for Microsoft 365 content.

Make sure that you’re not surprised about changes which appear inside Office 365 applications by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers stay informed.

Potentially a Play to Extract More Revenue from OneDrive Consumer

Being mostly concerned with the happenings in Office 365, our team doesn’t usually take much notice of developments in the consumer side of Microsoft. However, sometimes developments happen which are worth noting, especially when Microsoft marketing is excited about an announcement. Take January 31 for instance, when the avuncular Mark Kashman handcrafted text to announce the preview of Microsoft Lists for MSA. In other words, you can use your consumer Microsoft account to work with a “lightweight version of the Microsoft Lists app designed for small business and individual use.” All good, if you’re one of first 200,000 Microsoft account holders to head to the preview page to try out lightweight lists on a first-come, first-served basis.

Off I headed to lists.live.com to see what all the fuss was about. And I can report that it is possible to create a Microsoft list using a Microsoft Service account (Figure 1).

The process is painless, won’t kill any brain cells, and works like it does in the enterprise version. At least, it does from the user perspective. Those who do interesting and skillful things with Lists using Power Automate and other tools are likely underimpressed.

During the preview, Microsoft imposes a limit of 50 lists with up to 2,000 items per list. There’s also a 200 MB storage limit per list. That’s more than enough to test things out without doing anything more serious (always a bad idea with preview software).

The Teper View

On LinkedIn, Jeff Teper, who heads up ODSP (OneDrive, SharePoint, and Teams), had his say in another post. He asserts that making Lists available to consumer accounts is the next big technical bet for SharePoint. Under the covers, SharePoint has “user shards” (discrete segments of storage) to support consumer access and needed “a lot of engineering” to support authentication for MSA in addition to Azure AD. Lists for MSA uses a SharePoint MySite, which Teper notes is “just like we use in OneDrive for Business.” Microsoft suppresses the MySite UX, but the functionality is there, which Teper says “gives us a lot of flexibility for the future.”

A Premium Feature

Microsoft seldom undertakes large engineering efforts for zero return. In this case, I expect that, when it’s generally available, Lists for MSA will be a premium feature of OneDrive consumer, like the way that Outlook consumer is available in free and premium versions. In the same way, OneDrive consumers will use a common platform with some UX tweaking to hide or reveal features based on how much they pay. Lists is probably the first of these features, possibly coupled with Nucleus-powered offline capabilities and 100 GB storage (available today for $1.99/month).

Planner and Lists

In terms of Lists in SharePoint Online, an interesting post makes the case that Microsoft should replace Planner with Lists. Or perhaps, replace the underpinnings of Planner with Microsoft Lists (keeping the UX is easy). I don’t agree with the idea.

Planner and Lists are two different entities. In fact, Planner uses Tasks, one of the fundamental entities managed by the Microsoft 365 substrate shared across multiple applications. Lists are more complex objects, well suited for use as a development platform in many circumstances (including by Microsoft, such as the way Lists store Teams webinar information). Although a list can certainly manage a set of tasks, it’s a minor example of the kind of solutions people use Lists for today.

Moving Planner from Tasks (very simple items) to Lists is not straightforward, especially with the impact rippling across multiple applications and UIs. For instance, think of the way you can manage the same tasks through To Do, Tasks by Planner in Teams, and Outlook. If you moved Planner to use Lists, what impact would this have on To-Do and Outlook? The answer is “a lot.”

Over-engineering is as serious a problem as under-engineering, and it seems to me that any attempt to replace the fabric of Planner with Lists is an example of radical over-engineering.

Planner and Project – The One Development Group

There’s no doubt that Lists offers better support for customer-facing APIs today. The lack of application permissions for the Planner Graph API is regrettable, as is the slow pace of development in the Planner app overall when measured against the rest of Microsoft 365. That pace might be because the Project development team is responsible for Planner, and they want to keep clear blue water between Project and Planner.

Holding back Planner to enable Project to prosper might be regrettable but understandable in the context of the Microsoft 365 business. It’s no reason to jettison the Tasks underpinning for Planner and replace it with Lists.

Learn more about how Office 365 really works on an ongoing basis by subscribing to the Office 365 for IT Pros eBook. Our monthly updates keep subscribers informed about what’s important across the Office 365 ecosystem.

Feature Became Generally Available in July 2022

According to a LinkedIn post by Microsoft Principal Program Manager Sanjoyan Mustafi, administrators will soon be able to assign default sensitivity labels to document libraries in SharePoint Online and OneDrive for Business. The capability is in private preview at present, but Microsoft 365 tenants can sign up to join the preview here.

Update: According to message center notification MC391948 (June 13), rollout of the public preview of setting a default sensitivity label for a document library will roll out in late June. This is Microsoft 365 roadmap item 85621.

Update 2: On July 29, Microsoft announced that the roll-out for the public preview code had begun and that all tenants would receive the update within 90 days. The documentation is also available.

Today, you can require that users add a sensitivity label to documents and define a default label to use. This is done through settings of the sensitivity label publishing policy which makes labels available to users. Requiring documents to be labelled works, but you don’t know what labels users will choose. Sometimes, it might be necessary to ensure that every document in a library receives the same sensitivity label to reflect the level of confidentiality of the library, and that’s where the new capability comes in.

The Backend to Apply Sensitivity Labels

The preview includes the back-end code to define a default label and apply it to new Office documents uploaded or copied to or saved in a library. An asynchronous thread examines new items to check if they already have a sensitivity label. The stamping of the default sensitivity label on new items by the thread can take a few minutes.

If a new item already has a user-applied sensitivity label, the thread ignores the document based on the principle that explicit assignment by users always takes precedence over automatic assignment. If the item has a label of a lower priority (sensitivity labels have a priority order from 0 to n, with 0 being the lowest) received through automatic assignment (usually because a label publishing policy mandates the application of a default label), the thread replaces the label and applies the default label defined for the library.

For now, labeling only happens for new Office documents (support for PDFs will come later). Existing documents remain untouched, and you must apply labels manually if you want all documents to have the same label. However, in the future, Microsoft plans to update the code so that SharePoint will apply labels whenever a user opens an unlabeled document in a library with a default label.

Note that a user can remove the default label assigned for the library or replace it with a label of higher or lower sensitivity. In these cases, the user-assigned label remains, again following the principle of user precedence.

Update: Figure 1 shows the UX to configure a default sensitivity label for a document library. To access this screen, go to Library settings.

Configuring for Default Sensitivity Labels

Prior to Microsoft delivering the UX to configure a default sensitivity label for a document library, you had to update the configuration of the target document library using the SharePoint API. You can do this with Postman (the tool favored by Sanjoyan), but I prefer PowerShell, which is what I used. Sanjoyan explains the procedure in his post, but briefly is:

• Get a bearer token to authenticate with SharePoint Online. You can copy the token if you’re logged into SharePoint Online by using the developer tools (F12).
• Create a header structure to hold details of the transaction, including the bearer token.
• Create a body structure to define the GUID of the sensitivity label you want to add as the default for the library. Use Connect-IPPSSession to connect to the Compliance center endpoint and run Get-Label to find the list of labels. The GUID for each label is in the ImmutableId property.

Get-Label | Format-List DisplayName, ImmutableId

• POST to the URL for the document library using the header and body defined earlier.

The commands I used to update a document library were:

$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("Accept", "application/json;odata=verbose")
$headers.Add("Content-Type", "application/json;odata=verbose")
$headers.Add("X-HTTP-Method", "MERGE")
$headers.Add("If-Match", "*")
$headers.Add("Authorization", "Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsIng1dCI6IkRya21Mczl1akhnMkp1SE5CRm5vOERicXBJSSJ9.eyJhdWQiOiIwMDAwMDAwMy0wMDAwLTBmZjEtY2UwMC0wMDAwMDAwMDAwMDBAYjY2MjMxM2YtMTRmYy00M2EyLTlhN2EtZDJlMjdmNGYzNDc4IiwiaXNzIjoiMDAwMDAwMDMtMDAwMC0wZmYxLWNlMDAtMDAwMDAwMDAwMDAwIiwibmJmIjoiMTY0MzMxNTU5MiIsImV4cCI6IjE2NDMzMTkxOTIiLCJ1cG4iOiJ0b255LnJlZG1vbmRAcmVkbW9uZGFzc29jaWF0ZXMub3JnIiwicHVpZCI6IjEwMDNCRkZEODA1Qzg3QjAiLCJjYWNoZWtleSI6IjBoLmZ8bWVtYmVyc2hpcHwxMDAzYmZmZDgwNWM4N2IwQGxpdmUuY29tIiwidmVyIjoiYnJvd3NlciIsInRpZCI6ImI2NjIzMTNmLTE0ZmMtNDNhMi05YTdhLWQyZTI3ZjRmMzQ3OCIsImFwcGlkIjoiYzU4NjM3YmItZTJlMS00MzEyLThhMDAtMDRiNWZmY2QzNDAzIiwiYXBwX2Rpc3BsYXluYW1lIjoiU2hhcmVQb2ludCBPbmxpbmUgQ2xpZW50IEV4dGVuc2liaWxpdHkiLCJzY3AiOiJGaWxlcy5SZWFkV3JpdGUuQWxsIFNpdGVzLk1hbmFnZS5BbGwgU2l0ZXMuRnVsbENvbnRyb2wuQWxsIFRlcm1TdG9yZS5SZWFkV3JpdGUuQWxsIiwiaXBhZGRyIjoiNTEuMTcxLjIxMi4xMjkiLCJzZXNzaW9uaWQiOiIzYzdiMjUzYy0zNmJjLTQ1ZTctYmQ4OS1mNGRhZjdkZjZhNmEiLCJzaWduaW5fc3RhdGUiOiJbXCJrbXNpXCJdIn0.m0VNYiAPfu7GKuTcnAi0hc4ay7TAQ-KzlH1g3hRzRzJZccoLeRepey8k7ydNHsvdhO8N0E4mMEEz3dD8Tk-1qreBzNrqPkB6p2s8hGF1J04RaR6vkyTqJypFXLRXgmSsVrPsX1huNnkwZ0d_ShmPowUToZk_HN0MrDRIEleCks32pg1nQs2Umk63BkWAaUHJy_pLhYJOea0uzSc7iPeVpPaAQ8PbK8K4eRJX__DEByQueUSOd21V9O6KJ9ey-JasryPiqtncFUDGrofQ6EZztjwaCAjQubRv7RjOkMYeucgsgiI7cvfuvuCzcXjc6oqdosZwc-18Uurq_8r8ks9c4A")

$body = "{
`n `"__metadata`": {
`n `"type`": `"SP.List`"
`n },
`n `"DefaultSensitivityLabelForLibrary`": `"27451a5b-5823-4853-bcd4-2204d03ab477`"
`n}
`n"
$Uri = 'https://office365itpros.sharepoint.com/sites/Office365Adoption/_api/web/lists/GetByTitle(''Documents'')'
$Update = Invoke-RestMethod -Method 'Post' -Headers $Headers -Body $Body -Uri $Uri

Formatting of these commands must be precise, and the bearer token must be valid or the update will fail (I know, because I made many mistakes before doing it just right). The easiest way to make sure is to open the site you want to update in a private browser window to force a recent authentication and then copy the token (use F12 in Edge and access Local storage, then copy the value of the key for the identity for SharePoint Online as shown in Figure 2).

After configuring a default sensitivity label, it’s a good idea to change the default view for the library to include the sensitivity label to remind users that documents now have labels.

Steady Progress

Sensitivity Labels and SharePoint Online had a rocky start. There was a time when the content of protected Office documents was inaccessible to search and eDiscovery. That’s in the past (if you enable support) and Microsoft is busy filling out all the details that make software more useful. Adding a default sensitivity label to document libraries is a nice step forward but remember that using this capability will require Office 365 E5 or above, just like all the other auto-label application features in Microsoft 365.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across Office 365. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.

Now Available in SharePoint Online and OneDrive for Business

Message Center Notification MC302489 (December 8) brings news of yet another tweak made by Microsoft to the dialog used to create new Sharing Links. The update means that the settings for sharing links for “most video and audio” files now block download by default (Figure 1).

Previous tweaks to the dialog include making it easier to update sharing link settings and highlighting the edit setting. Because many workloads use the sharing link dialog, the benefit of the changes ripple across Microsoft 365.

Understandable Change in Line with Previous Updates

The change is understandable. Sharing a video or audio is often just an invitation to consume final content (using the recently-upgraded web viewer) and you don’t want people to be able to download the files. By comparison, sharing a document, spreadsheet, or presentation is often for review and editing purposes, and the recipient might need to download a local copy to edit the file offline.

Interestingly, Microsoft 365 roadmap item 82193 makes explicit reference to Microsoft Stream, probably reflecting the ongoing motion to move Stream away from its old Azure-based platform to storing videos in OneDrive for Business and SharePoint Online. This transition has already happened for Teams meeting recordings, and the migration for other Stream content is in preview. Teams meeting recordings restrict download access to the recording owner, so setting sharing links to no download by default is in line with that philosophy.

Not All Video or Audio Files

Noting the caveat that the change applies to most video and audio files, I checked the content of my OneDrive for Business account and discovered that OneDrive blocks downloads in sharing links created for Teams meeting recordings. The same doesn’t happen for other MP4 files that I uploaded to OneDrive where the download control is missing when creating sharing links (Figure 2).

The BlockDownloadLinksFileType setting for my tenant (managed through PowerShell with the Set-SPOTenant cmdlet) is WebPreviewableFiles, which means that download blocks are available for all supported files. Given that audio and video files are now in the supported category, something else is going on.

OneDrive recognizes both sets of files as MP4s, so the difference in behavior might be because the uploaded files didn’t have the same PROGID tags as the Teams recordings (these tags make it possible to apply an auto-label retention policy to Teams meeting recordings). Alternatively, it could be because some background job hasn’t yet processed the other MP4 files. Requiring extended periods to process files is not unknown in SharePoint Online and OneDrive for Business. In any case, I’ll keep an eye to see if things change.

Make sure that you’re not surprised about changes which appear inside Office 365 applications by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers stay informed.

Knowing When Sharing Happens

A natural question flowing from the discussion about implementing the SharePoint Online expiring access policy for external users is how administrators know if people use the feature. Equally naturally, the first place to look is the Office 365 or “unified” audit log to see if SharePoint Online generates any helpful events when users extend sharing links.

Unhappily, although SharePoint Online captures a UserExpirationChanged audit event when someone extends a sharing link close to its expiration, the information stored in the event is not enough to easily identify the content the sharing link grants access to. If you look at the sample audit event shown below, the SiteUrl property tells us that this event relates to sharing some OneDrive for Business content. Apart from that, we can see:

• The user principal name of the user who extends the validity of the sharing link (Jane.Sixsmith@office365itpros.com).
• The user principal name of the target user being granted access (Jsmith_yandex.com#ext#@office365itpros.onmicrosoft.com). The form tells us that this is a guest account (JSmith@yandex.com).

It would be nice if the name of the actual folder or document being shared was captured, but that’s not the case.

RecordType   : SharePointSharingOperation
CreationDate : 15/11/2021 13:17:04
UserIds      : Jane.Sixsmith@office365itpros.com
Operations   : UserExpirationChanged
AuditData    : {
                 "AppAccessContext": {
                   "AADSessionId": "bfe559aa-a811-488b-828d-a1fa90062133",
                   "CorrelationId": "b45e03a0-50df-3000-73a8-a6b7cbd31cc0"},
                 "CreationTime": "2021-11-15T13:17:04",
                 "Id": "5ee7b4d0-97ca-476d-c7ef-08d9a83a37aa",
                 "Operation": "UserExpirationChanged",
                 "OrganizationId": "a562313f-14fc-43a2-9a7a-d2e27f4f3478",
                 "RecordType": "SharePointSharingOperation",
                 "UserKey": "i:0h.f|membership|1003bffd805c87b0@live.com",
                 "UserType": "Regular",
                 "Version": 1,
                 "Workload": "OneDrive",
                 "ClientIP": "51.171.212.129",
                 "ObjectId": "https://office365itpros-my.sharepoint.com/personal/jane_sixsmith_office365itpros_com",
                 "UserId": "jane.sixsmith@office365itpros.com",
                 "CorrelationId": "b45e03a0-50df-3000-73a8-a6b7cbd31cc0",
                 "EventSource": "SharePoint",
                 "ItemType": "Web",
                 "Site": "cc191cff-670a-4740-8458-e6067537c747",
                 "UserAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/95.0.4638.69 Safari/537.36 Edg/95.0.1020.44",
"WebId": "551065f1-04a6-4979-8b19-2c8a0c16319f",
                 "TargetUserOrGroupType": "Guest",
                 "SiteUrl": "https://office365itpros-my.sharepoint.com/personal/jane_sixsmith_office365itpros_com",
                 "TargetUserOrGroupName": Jsmith_yandex.com#ext#@office365itpros.onmicrosoft.com

Investigating SharePoint Sharing Events

To see if it was possible to find some other information that would allow me to link the UserExpirationChanged events back to other sharing events, I wrote a script to extract the events from the audit log and parse their content. The results are not what I hoped. You can track the progress of sharing an item through:

• SharingSet: A user shares an item.
• SecureLinkCreated: A sharing link is created for the item. This is what is sent to the recipient.
• UserExpirationChanged: The expiration date for the sharing link is adjusted in line with policy.
• SecureLinkUsed: The recipient uses the sharing link to access the shared content.

The audit records for the first three events often have the same date and time because they occur close together (within milliseconds). For this reason, they can appear in a different order when viewing the report (Figure 1).

In due course, if the sharing link validity is extended further, SharePoint logs another UserExpirationChanged event. The cycle continues until the sharing link expires.

Download the Script

The script isn’t all that interesting. It finds the relevant audit events, extracts information, and reports its findings (you can download the script from GitHub). Unless you focus on UserExpirationChanged events which happen outside the initial creation of sharing links, I don’t think it helps much in terms of understanding the extent of sharing link extensions. However, someone who is smarter than I might be able to tweak the script to derive better results.

Learn more about how Office 365 really works on an ongoing basis by subscribing to the Office 365 for IT Pros eBook. Our monthly updates keep subscribers informed about what’s important across the Office 365 ecosystem.

SharePoint Expiring Access Policy Controls Sharing Links Issued to Guests

In the summer, Microsoft introduced an expiring access policy for external users in SharePoint Online sites and OneDrive for Business accounts. In a nutshell, a tenant can set a policy to control the number of days a sharing link lasts after a user shares some content with an Azure AD guest account (created automatically when sharing with an external user). The expiring access policy doesn’t apply to guest accounts who access content through their membership of Microsoft 365 groups (teams). Their ability to work with content in SharePoint Online is controlled by the guest’s membership instead of a sharing link.

By default, the expiring access policy is not set. A tenant or SharePoint administrator must enable it and define the sharing period in the Sharing section of the SharePoint Online admin center (Figure 1). The period can be from 30 to 730 days.

Once set, the policy applies to new sharing links. It also applies retrospectively to old links. The policy defined in the SharePoint Online admin center applies to all SharePoint sites and OneDrive for Business accounts. You can override the expiration period on a per-site basis.

Unlike other expiration policies used in Microsoft 365, like the Teams meeting recording auto-expiration policy or even retention policies and labels, content remains unaffected when an expiration period lapses. The only effect is on the sharing link which becomes invalid and unusable for access.

What Happens When Sharing Links Expire

As sharing links approach expiration, users receive warnings through two means. First, a banner appears in OneDrive for Business (Figure 2). The text could be better as it’s a sharing link which expires rather than a user. The Azure AD guest account will remain and can be used for other purposes, such as other sharing links or as a member of a group or team. The logic here might be that people manage sharing access on a user-by-user basis, so it’s appropriate to refer to users expiring.

The second method is email. SharePoint sends a note to people to advise them when sharing links are within ten days of expiration (Figure 3). In both cases, the Manage (or Manage access) link allows the user to update the soon-to-expire sharing links.

Clicking the link brings up the Access Expiration fly-out pane (Figure 4), which lists all sharing links created by the user subject to the expiring access policy. As you can see, some of the links are quite a long way off because the tenant has a 120-day expiration policy.

To extend the validity of a sharing link, select a user and click Yes, extend (Figure 5). SharePoint Online will then extend the sharing link by the maximum period allowed, in this case 120 days from the current date. You can also remove a sharing link if it’s no longer needed.

Good Practice to Implement Expiring Access Policy

It’s good practice and makes good sense for Microsoft 365 tenants to implement an expiring access policy. Many expiring sharing links will need no intervention by content owners when they expire. Other links will need an extension, which is a quick and low friction action. Overall, there’s nothing much to dislike about implementing an expiring access policy where links expire after a reasonable period, like 90 to 120 days. Organizations which store more sensitive content in SharePoint could reduce the expiration period and couple expiration with the targeted availability to content available with sensitivity labels.

Learn how to exploit the Office 365 data available to tenant administrators through the Office 365 for IT Pros eBook. We love figuring out how things work.

Joins the Controls for Teams Meeting Recordings

Now that Microsoft has created the transition of storage for Teams meeting recordings (TMRs) from Stream (classic) to OneDrive for Business and SharePoint Online (ODSP), attention is focused on how to manage these files. Microsoft plans to introduce an auto-expiration policy for TMRs in January 2022 to allow organizations dictate how long these files exist in ODSP. The auto-expiration policy will work for any Microsoft 365 tenant which has licenses for Teams.

If you have Office 365 E3, users can apply retention labels to TMRs to gain more control over their retention, and if you have Office 365 E5 or Microsoft 365 E5 licenses, you can deploy an auto-label retention policy to find and label TMRs (and track the success of the policy in finding and labeling TMRs). In short, over time, organizations are gaining ways to exert compliance control over TMRs.

Blocking Sharing with Data Loss Prevention

Data Loss Prevention (DLP) for SharePoint Online and OneDrive for Business is included in the Office 365 E3 SKU. The value of DLP is that you can use a policy to protect against inadvertent data leakage caused when someone shares a TMR outside the organization. Imagine what would happen if a competitor got hold of a recording of a discussion, complete with slides, about the development of a new product!

Using much the same approach as taken to identify TMRs for the auto-labeling retention policy, we can build a DLP policy for TMRs which looks for recording files and stamps them with metadata to stop sharing happening. The DLP policy to block external sharing for TMRs is very simple. It is a custom DLP policy (i.e., not created using a template) consisting of:

• A name and description.
• Target locations. For maximum coverage, choose all SharePoint Online sites and OneDrive for Business accounts. This will stop any sharing of TMRs created for personal meetings (OneDrive) and channel meetings (SharePoint).
• A single rule. The rule looks for any file with the property value ProgId:Media.Meeting that is shared with someone outside the organization. The rule action blocks sharing with people outside the organization. Figure 1 shows what the rule conditions look like. Optionally, the rule can allow users to override the block by providing a justification to explain why they need to share a recording with an external person.

Other rule settings which you might consider include creating a custom policy tip to explain why users can’t share TMRs externally or generating an incident report to alert administrators or other people when a rule violation occurs.

The Effect of DLP

It can take up to an hour before a new DLP policy is effective. When the policy is active, the indexing process for new files detects that TMRs come within the scope of a policy and applies the policy settings to block external sharing. There might be a few minutes before the block is effective for a new file during which it’s possible to create and send a sharing link. However, once the block is in place, the sharing link is nullified.

The effect of the policy is obvious because any document which matches the policy conditions now has a small icon (circle with a line in the middle). In Figure 2, the icon is shown alongside all the TMRs in the Recordings folder. Other video files that don’t have the property set are not marked. Hovering over a TMR reveals information about the file, including a link to a DLP policy tip if set. In this case, the link reveals some custom text to explain that external sharing is not permitted for TMRs.

If the user ignores the warning and goes ahead to try and share the recording anyway, they won’t be able to do this because OneDrive for Business blocks the attempt to create and send a sharing link (Figure 3).

Easy Update

Even if internal users don’t often go back to relisten to what was discussed in a conference call, there’s no doubt that some external people might find that content interesting, perhaps even to the detriment of your company. The time required to create and deploy a DLP policy to block external sharing of TMRs is roughly ten minutes (including a pause to drink coffee). It’s a quick and easy update to make it easier to manage the security of information contained inside these files. This is a good example of the value of DLP.

Learn more about how Office 365 really works on an ongoing basis by subscribing to the Office 365 for IT Pros eBook. Our monthly updates keep subscribers informed about what’s important across the Office 365 ecosystem.

Investigating Unlabeled SharePoint Sites

Microsoft is fond of equipping its administrative consoles with cards containing insights which administrators might action. Yesterday, I noticed that the SharePoint Online admin center highlighted that my tenant had many sites had no sensitivity label (Figure 1).

As you might recall, Microsoft 365 uses sensitivity labels to apply settings to “containers” (teams, groups, and sites). Controlling the external sharing capability of SharePoint Online sites is a good example of the power of this approach. By default, I assign sensitivity labels to when creating new Microsoft 365 groups and teams, so it surprised me to discover the unlabeled state of so many sites.

Explaining Unlabeled Sites

Using the Manage unlabeled sites link, I examined the sites. Because I use sensitivity labels for the sites used for groups and teams, I expected to find that some sites in the tenant had no labels. These include:

• Hub sites.
• Communication sites.
• System sites (such as the one used to manage Viva Topics).

Knowing that teams created using templates didn’t ask team owners to assign a sensitivity label until Microsoft fixed the problem in October 2021 (MC281936, Microsoft 365 roadmap item 84232), I could account for some other unlabeled sites. However, stripping all the explainable sites from the 126 noted by SharePoint still left a bunch that I couldn’t explain except by concluding that at some points in the past, the synchronization of sensitivity labels didn’t work as well as it should between SharePoint Online and the other workloads. This is an important thing to fix because if SharePoint Online doesn’t know about a sensitivity label assigned to a site, it can’t apply the management controls defined in that label.

For the record, the synchronization of sensitivity labels for new groups works well. This might be the vestige of a long-solved problem.

Fixing Up Site Sensitivity Labels

To address the problem, I decided to write some PowerShell. The first stage was to find all the sites created for teams and Microsoft 365 Groups that didn’t have a label. To do this, the code:

• Runs the Get-SPOSite cmdlet to find all sites created using the team site template.
• Run Get-SPOSite against each site to find sites without a sensitivity label. You need to access each site to find if it has a label because Get-SPOSite doesn’t return this property when run against multiple sites.
• Store the unlabeled sites in a list.

Here’s the code I used:

[array]$Sites = Get-SPOSite -Limit All -Template Group#0
If (!($Sites)) { Write-Error "No sites for Microsoft 365 Groups found... exiting!" ; break}
   Else { Write-Host ("Processing {0} sites" -f $Sites.Count) }

$SitesNoLabels = [System.Collections.Generic.List[Object]]::new()
ForEach ($Site in $Sites) { #Check each site to see if it has a sensitivity label
        $SiteData = Get-SPOSite -Identity $Site.Url
        If ([string]::IsNullOrWhiteSpace(($SiteData.SensitivityLabel)) -eq $True) {
           Write-Host ("Site {0} has no label" -f $SiteData.Url) 
           $SiteInfo = [PSCustomObject][Ordered]@{  
              URL    = $SiteData.Url
              Title   = $SiteData.Title   }
           $SitesNoLabels.Add($SiteInfo) }
} #End ForEach Sites

The properties of a Microsoft 365 group store the GUID of the sensitivity label, if one is assigned to the group/team. The next step is to retrieve the sensitivity label information for all groups. It’s possible to match a group with a site because the group properties include the site URL. I therefore:

• Used the Get-UnifiedGroup cmdlet to find all Microsoft 365 Groups. This won’t be a fast operation in large tenants, but it’s acceptable because this is a one-time operation. In the largest tenants, consider replacing the Get-UnifiedGroup cmdlet with the Groups Graph API (see the call to fetch all Microsoft 365 groups in a tenant described in this article).
• Removed any group that didn’t have a SharePoint site URL in its properties (sometimes an error in the provisioning process leaves this property blank. Microsoft 365 will eventually synchronize the site URL from SharePoint Online to Exchange Online).
• Store the site URL and sensitivity label GUID in a hash table. A list would also do, but it’s much faster to lookup against a hash table.

Here’s the code for this segment:

Write-Host "Retrieving sensitivity label information for Microsoft 365 Groups"
[array]$Groups = Get-UnifiedGroup -ResultSize Unlimited 
$Groups = $Groups | ? {$_.SharePointSiteUrl -ne $Null}
$GroupsTable = @{}
$Groups.ForEach( {
       $GroupsTable.Add([String]$_.SharePointSiteUrl, $_.SensitivityLabel) } )

We now have a list of sites without labels and a table with the labels assigned to the underlying groups. The next step is to check each site against the groups table to see if we can find what label the site should have. If we find a match, we can update the site. The next code segment does the following:

• Loop to check each unlabeled site.
• Use the site URL as a lookup against the groups table.
• If the site URL matches, use the label GUID to update the site with the Set-SPOSite cmdlet.

This code applies sensitivity labels to sites using the information from Microsoft 365 Groups:

[int]$Updates = 0; [int]$NoUpdates = 0
ForEach ($Site in $SitesNoLabels) {
    $Label = $Null
    $Label = $GroupsTable.Item($Site.Url)
    If ($Label) { # Update the site with the label we find
       Write-Host ("Updating site {0} with label {1}" -f $Site.Url, $Label.Guid) 
       Set-SPOSite -Identity $Site.Url -SensitivityLabel $Label.Guid 
       $Updates++ }
    Else {
       Write-Host ("Can't find sensitivity label for site {0} - group might be deleted" -f $Site.Url)
       $NoUpdates++ }
} #End ForEach Sites

The complete script is available from GitHub.

A Better Card

Of the 126 unlabeled sites reported by SharePoint Online, 116 were team sites. The technique described above managed to apply sensitivity labels to 103 sites. The remaining 13 are deleted sites kept by SharePoint Online because of a retention policy (the associated Microsoft 365 group is gone). The card displayed in the SharePoint Online admin center looks better (Figure 2) and all the sites belonging to Microsoft 365 groups and teams have their correct labels. All is well.

Insight like this doesn’t come easily. You’ve got to know the technology and understand how to look behind the scenes. Benefit from the knowledge and experience of the Office 365 for IT Pros team by subscribing to the best eBook covering Office 365 and the wider Microsoft 365 ecosystem.

Discovering Some Nuggets from Microsoft’s Coverage

It’s been a busy week for anyone following the Microsoft 365 ecosystem as Microsoft released a slew of blog posts and announcements to support keynotes and other sessions at the Microsoft Ignite Fall event. You could spend hours reading about new features and functionality and wonder when the code will appear in your Office 365 tenant and if any additional licenses are necessary.

This post captures notes about several features available now that I noticed as I perused Microsoft’s coverage. By themselves, each is not enough to warrant a separate post, but they’re interesting all the same. These changes are examples of the stuff we track to maintain the content of the Office 365 for IT Pros eBook. All our chapter authors have been busy this week.

SharePoint Online and OneDrive for Business

Sharing links show who you’ve shared a document with. This feature was announced in June but seems to have taken its time to roll out. The idea is simple. When you send a new sharing link, SharePoint Online and OneDrive for Business tell you who the document is already shared with (Figure 1), including a thumbnail of each person (if available in Azure AD). You can hover over a thumbnail to see who the person is. The number of active sharing links also appears. It’s a small but useful change.

Easy to overlook, the SharePoint Online admin center now displays connected channel sites when a site used by Teams creates private channels (Figure 2). If you can’t remember which sites have private channel sites, connect to SharePoint Online PowerShell and run:

Get-SPOSite -Limit All -Template TeamChannel#0 | ? {$_.TeamsChannelType -eq "PrivateChannel"}

If you click the channel sites link, the admin center displays details of those sites. Teams manages the settings for these sites, but it’s nice to be able to have easy access to the information. Shared channels, which are delayed until early 2022, also use channel sites.

OneDrive for Business supports Known Folder Move (KMF) and Files on Demand on MacOS, which is nice if you’ve invested in a brand-new M1-powered Mac.

If your tenant uses sensitivity labels and has SharePoint Syntex, you can apply sensitivity labels to protect the document understanding models. The application of a label in this manner flows through to protect individual documents identified by models. It’s another way of automatically applying labels to sensitive content.

Sensitivity label control over sharing capabilities of SharePoint Online sites is now generally available. In addition, co-authoring and autosave of protected documents is generally available in the Microsoft 365 apps for enterprise (Word, Excel, and PowerPoint). We use protected documents heavily to store chapter files for the Office 365 for IT Pros eBook, so this is a welcome advance.

Exchange Online

Microsoft Scheduler can now dynamically adjust the scheduling of recurring meetings. This is message center notification MC295855 (November 2) and it’s a great idea. Static recurring meetings are all too often cancelled or rescheduled because someone is sick or otherwise unavailable. After a recurring meeting finishes, Scheduler looks for the best time slot for the next instance and books that time.

Everyone’s probably familiar with the Exchange Online campaign to remove basic authentication for email connection protocols (that October 2022 date is getting nearer!). PowerShell is on the list of protocols to be blocked for basic authentication, but the Exchange Online management PowerShell module still uses basic authentication to communicate with WinRM on a local workstation. Work is under way to remove the need to use WinRM. Microsoft has released a preview version (2.0.6-3preview) of the module to demonstrate how they will remove the dependency by using a REST API in the background. Exchange Online has many cmdlets, not all of which have been converted to use the new mechanism, but you can test the preview now.

On the downside, Microsoft didn’t say anything at Ignite about the next version of on-premises Exchange. This is strange given the September 2020 announcement said the next version of Exchange Server would be available in the second half of 2021.

Microsoft 365

Microsoft says that Visio web app is rolling out to Microsoft 365 commercial tenants (all tenants with Office 365 enterprise plans). The rollout goes through to the end of January 2022, so keep an eye on the app launcher to see when Visio web app (aka Visio in Microsoft 365) shows up in your tenant.

Microsoft Cloud App Security (MCAS) is now Microsoft Defender for Cloud Apps (surely MDCA?). The app governance add-on is now generally available. It’s a good way to chase down apps registered in Azure AD that are over-permissioned or not being used. If you don’t have MDCA or don’t want to pay for the add-on, use our DIY audit method for Azure AD apps.

Access to the knowledge available in topic cards created by Viva Topics has been restricted to some lesser-used applications up to now. Things will change when topic cards appear in OWA and Teams. Apparently, this will happen soon and should be a game changer for the organizations who have invested in the work needed to harvest organizational knowledge through Viva Topics.

Teams

Microsoft prioritized Teams at Ignite as the center of a new way to work (see my practical365.com article), so there were lots of Teams-related developments discussed, most of which can be left until they appear in a tenant near you. One snippet in a blog post about improving meeting quality is that noise suppression in Teams meetings will be available for iOS soon. Microsoft claims that they saw a “31% decline in comments about background noise distractions” after the launch of noise suppression. This sounds like a good thing, but a single statistic provided without any further context or detail is worthless. We don’t know the sample size, whether the clients were Windows or Mac. What kind of meetings, and what is meant by “comments” (good, bad, or indifferent). Like many Microsoft statistics, there’s plenty of room for fudging an issue.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across Office 365. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what’s happening.

Delving Into SharePoint’s Custom Properties

I’ve used SharePoint since the initial release of SharePoint Portal Server 2001, but I would never regard myself as being a SharePoint expert. I am perfectly happy to perform site management using the SharePoint Online PowerShell module or the admin center, but admit that the finer points of the client-side object module (CSOM) and the Patterns and Practice (PnP) library often surpass the limits of my knowledge. Given that much of SharePoint Online usage is generated by the sites used by Microsoft 365 Groups and Teams, less need exists to get down and dirty with CSOM or PnP than appears to be the case for SharePoint Server.

The Site Property Bag

However, sometimes no other option exists but to interact with SharePoint using PnP, which brings me neatly to the subject of the site property bag. This is a feature allowing the assignment of custom values to sites. If you come from the Exchange world, it’s analogous to being able to set custom properties for mailboxes. And just like custom properties are often used in Exchange as filters to identify specific mailboxes, the site property bag can refine searches by marking sites with custom values.

Custom values written into the site property bag are simple name/value pairs. For instance, the name might be “Test” and the value “Tony.” The idea is that users can then search for sites by looking for those where “Tony” is present in the “Test” property. Being able to find sites using a filter is important for functionality like adaptive scopes for Microsoft 365 retention policies. Custom values end up as crawled properties in the SharePoint Online search schema. The crawled properties can be linked to refinable strings to become searchable, which is how the property bag values can be used in filters.

Updating Values in the Site Property Bag

The standard Set-SPOSite cmdlet in the SharePoint Online management module doesn’t update the property bag, but cmdlets from the PnP PowerShell module do. To begin, I downloaded and installed V1.8.0 from the PowerShell gallery. The developers issue frequent updates for the module, so it’s wise to make sure that you use the latest (non-preview) version.

Before attempting to update the property bag for a site, you must disable the site’s DenyAddAndCustomizePages setting. By default, SharePoint Online blocks custom scripts, and to update the property bag, we need to lift the restriction temporarily. To do this, run the Set-SPOSite cmdlet to set DenyAddAndCustomizePages to 0 (zero). Before proceeding, make sure that the value of DenyAddAndCustomizePages is Disabled (the default is Enabled).

$Site = "https://office365itpros.sharepoint.com/sites/BallyconneelyBuglers"
Set-SPOSite -Identity $Site -DenyAddAndCustomizePages 0
Get-SPOSite -Identity $Site | Select DenyAddAndCustomizePages

DenyAddAndCustomizePages
------------------------
                Disabled

The updated setting is effective immediately. The next step is to connect to the site using the Connect-PnPOnline cmdlet. An account can connect to a site only if it has access to the site. In this instance, I used a global tenant administrator account.

Connect-PnPOnline -Url $Site -Credentials $O365Cred
Set-PnPPropertyBagValue -Key "OrgPrivacy" -Value "Restricted" -Indexed
Set-PnPPropertyBagValue : Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED)) Site might have NoScriptenabled, this prevents setting some property bag values. At line:1 char:1

You’d imagine that a global tenant administrator can update site properties. After all, we’ve just used the same account to update the site customization setting with the Set-SPOSite cmdlet. However, the PnP module imposes its own rules. Everything looked good, but the error surfaced each time I attempted to write a new value into the site property bag.

After some debugging, I discovered that it is possible to update the site property bag only if you connect with a site administrator account. After adding the global administrator account as a site administrator, the Set-PnpPropertyBagValue cmdlet ran without a problem. If we examine the contents of the site property bag with the Get-PnPPropertyBag cmdlet, the custom value is present.

Get-PnpPropertyBag

Key                              Value
---                              -----
GroupId                          ff168380-8f71-4419-980c-7f1e8e6ea83a
vti_sitemasterid                 e2ea95e2-b7be-484f-bb63-e2b0fd4b38b6
vti_categories                   Travel Expense\ Report Business Competition Goals/Objectives Ideas Miscellaneous Waiting VIP In...
vti_createdassociategroups       3;4;5
vti_defaultlanguage              en-us
HomepageProvisioned              1
contenttypessynctimestampversion 1
vti_approvallevels               Approved Rejected Pending\ Review
taxonomyhiddenlist               73396654-2d02-47d9-a078-6f0ffe401097
vti_associategroups              5;4;3
profileschemaversion             6
GroupDocumentsListId             2825b7cc-43f3-4eef-b970-f9789082f70d
disabledhelpcollections
SiteNotebookGuid                 ddb569bc-70b8-4eae-8e02-cd221f11d5d2
GroupType                        Public
contenttypesusagebackfillversion 3
vti_associatevisitorgroup        4
vti_extenderversion              16.0.0.21409
OrgPrivacy                       Restricted
GroupAlias                       BallyconneelyBuglers
LastGroupSitePrivacyUpdated      637612064800877337
vti_associateownergroup          3
enabledhelpcollections           VGSEndUser
ProvCorrelationId                9462025b-ebf9-468c-bbde-3729d938bdbf
FollowLinkEnabled                TRUE
vti_associatemembergroup         5
GroupDocumentsUrl                Shared Documents
vti_indexedpropertykeys          TwByAGcAUAByAGkAdgBhAGMAeQA=|

After writing the custom values into the site property bag, make sure that you replace the block on custom scripts for the site:

Set-SPOSite -Identity $Site -DenyAddAndCustomizePages 1
Get-SPOSite -Identity $Site | Select DenyAddAndCustomizePages

DenyAddAndCustomizePages
------------------------
                Enabled

Checking Custom Scripting Status for Sites

Some blogs say that the DenyAddAndCustomizePages setting reverts to the default setting after a period. I have not seen this happen, but this could be simply a case of not waiting long enough for a SharePoint Online background Some blogs report that the DenyAddAndCustomizePages setting reverts to the default setting after a period. I have not seen this happen, but this could be simply a case of not waiting long enough for a SharePoint Online background process to work. In any case, it’s best to be proactive and leave sites in the correct state. A quick check with PowerShell will reveal any sites which need to be updated and correct the situation. In this example, we check only for group-enabled sites:

$ScriptingSites = 0
[array]$Sites = Get-SpoSite -Limit All -Template Group#0 | Sort Url
ForEach ($Site in $Sites)  {
   If ($Site.DenyAddAndCustomizePages -ne "Enabled") {
      $ScriptingSites++
      Write-Host ("Site {0} has scripting enabled, so now disabling scripting..." -f $Site.Url)
      Set-SPOSite -Identity $Site.Url -DenyAddAndCustomizePages 1 }
}
If ($ScriptingSites -gt 0) { Write-Host ("{0} sites found with scripting enabled - now disabled." -f $ScriptingSites) }

If you’ve added a tenant administrator account as a site administrator to update the property bag, make sure that you remove the account afterwards. It’s not good to allow access to site contents to tenant administrator accounts unless this is intended.

Moving Forward

As it turns out, updating SharePoint Online site property bags isn’t difficult. That is, if you satisfy all the requirements. In this case, making sure that you use a site administrator account is the important point. It’s something that I didn’t see covered in any of the blogs which describe how to update the property bag (I’m sure this is documented somewhere). Now that I know how to assign custom values to SharePoint sites, the road is clear to use these properties in adaptive scopes.

Learn more about how Office 365 really works on an ongoing basis by subscribing to the Office 365 for IT Pros eBook. Our monthly updates keep subscribers informed about what’s important across the Office 365 ecosystem.

Making Compliance Work Better

As discussed last week, Microsoft is simplifying how retention processing works for SharePoint Online and OneDrive for Business. It’s a good initiative because this topic is like a black box for many tenant administrators. The latest step comes in MC289965 (7 October – roadmap item 82063) to align how the SharePoint Online and OneDrive for Business browser interfaces deal with user requests to delete a file assigned a retention label configured to retain items for a specific period. For instance, a file might have a retention label with a retain action for seven years. (A retention label can be set to don’t delete or retain items, which makes it a visual marker).

Deleting Files in SharePoint Online and OneDrive for Business

Up to now, the following happens:

• OneDrive for Business: User deletes file with retention label. OneDrive for Business moves the file into the Recycle bin and captures a copy in the preservation hold library for the user’s account. A OneDrive account is a personal space and it’s reasonable to allow the account user to delete files if they wish. Note that you can’t delete a file assigned a record label. To create a retention label as a record, you need to use the Records Management solution in the Microsoft 365 compliance center (requires E5).
• SharePoint Online: User attempts to delete file with retention label but is blocked because of the presence of the retention label (Figure 1).

You can argue a case that SharePoint Online does the right thing. By not allowing the deletion to happen and keeping the file in place until its retention period expires, SharePoint Online demonstrates that the file has some importance.

The Problem for Compliance

However, the problem is that the current Microsoft 365 group model allows group members full control over most items in the SharePoint Online team sites used by Teams and Groups. Therefore, if SharePoint Online blocks a user from deleting a file because of a retention label, they can simply remove the label and then delete the file (unless the retention label is a record label). Although most users might not realize that they can remove a retention label to delete a file, the fact that they can is a big problem in terms of compliance. In that light, it’s better to allow the deletion to proceed. SharePoint Online will capture the file in the preservation hold library to ensure that its content remains indexed and discoverable for retention purposes.

Earlier Attempt to Change Ran into Problems

Last June, Microsoft published MC264360 to notify tenants that they planned to change the way the SharePoint Online browser interface worked to bring it in line with OneDrive for Business. In other words, users would be able to delete files even if a retention label with a retention period was present.

After pushback from customers, Microsoft withdrew the proposed change to do some additional work. The result of that work will roll out in early November for completion by the end of the month. SharePoint Online users will be able to delete labelled files like they can in OneDrive for Business unless the organization decides that this is a bad idea and updates the SharePoint Online configuration to retain the existing behavior. SharePoint Online will continue to block deletion of Items labelled as records.

Update January 11, 2022: The controls over deletion behavior are available in the Records management section of the Microsoft 365 compliance center (Figure 2).

Changing Things Back

If an organization decides that they’d like to keep things as they are, administrators will have to crack open the SharePoint Client Object Model (CSOM) and use the SetAllowFilesWithKeepLabelToBeDeletedSPO function in the SPPolicyStoreProxy class to set the value to False. Quite why Microsoft didn’t add a new parameter to the SPO-Tenant cmdlet to update this setting like all the other SharePoint Online organizational settings is beyond me. Microsoft says that when the feature rolls out, the ”configuration will be available within the Records Management solution settings.” That’s all fine and dandy, but Records management requires Office 365 E5 or Microsoft 365 Compliance E5 licenses, so many administrators might avoid it. This setting should be in the SharePoint Online admin center and settable through PowerShell.

No doubt someone who knows their way around CSOM will create and publish the code necessary to update the setting with PowerShell so that people without deep knowledge of SharePoint object models don’t have to, but I think it is unacceptable for Microsoft to push a change out that cannot be easily controlled by tenant administrators. On the bright side, I think most tenants will like the new delete behavior for files with retention labels and can therefore ignore grappling with CSOM.

Change Based on Experience

Changing the way SharePoint Online works when deleting files with retention labels with retention periods is the right thing to do. It will make compliance work better and is more logical for users. It’s just a pity that the opt-out control is hidden.

Keep up with the changing world of the Microsoft 365 ecosystem by subscribing to the Office 365 for IT Pros eBook. Monthly updates mean that our subscribers learn about new development as they happen.

Making Retention More Efficient

Message center notification MC288633 (1 October) covers the topic of optimized behavior of file versions preserved in SharePoint Online and OneDrive for Business. It’s a title guaranteed to turn off most Office 365 administrators unless they’re interested in compliance. As it happens, I am, so I read the notification.

My reading of the situation is that Microsoft is replacing an old-fashioned implementation of the preservation hold library with a more modern approach. As you might know, the preservation hold library is the location used by SharePoint Online to keep information needed for retention purposes. It’s the equivalent of Exchange Online’s Recoverable Items structure, a place where updated and removed content stays until the retention period expires.

The Preservation Hold Library

Up to now, SharePoint Online has used the preservation hold library to retain multiple versions of changes made to documents and list items. If someone edits a document which comes within the scope of a retention policy, SharePoint captures a pre-change copy of the document in the library. If someone deletes a document that must be retained, it goes into the preservation hold library. The actual processing is more complicated, but that description is sufficient here.

The net effect is that a preservation hold library for a busy site can accumulate a bunch of items (Figure 1). Although users cannot access the preservation hold library, its content is indexed and discoverable and available for searching, which means that eDiscovery investigators can recover the full change record for documents and list items. Administrators can also recover files from the preservation hold library, so there’s lots of goodness available.

The Downsides of Retention

Except that a downside exists. Or rather, two significant downsides. The first is that capturing edits and deletions for a busy SharePoint Online site can consume a large percentage of the storage quota used for the site. The amount differs from site to site depending on the characteristics of site usage and the type of file stored. For instance, the site which I use to store the Word documents for blog posts has thousands of relatively small files (usually in the range of 1-5 pages), most of which are never edited after publication. The preservation hold library for the site holds 924 items of 292.6 MB, or 5.92% of the site storage.

The site used for the Office 365 for IT Pros book has completely different characteristics. The Word documents (and some Excel spreadsheets) are larger (some chapters are over 100 pages) and they receive frequent revisions. For example, according to its version history, the chapter covering Teams architecture and structure in the 2021 edition has 330 versions, most generated using the Office AutoSave feature. The combination of large files and multiple revisions drives storage consumption to 15.3 GB, or 21.8% of the site (Figure 2).

The problem is that SharePoint Online regards the storage consumed by the preservation hold library in the same manner as it treats other libraries. Everything counts against the tenant’s overall SharePoint storage quota, which seems a little unfair given that Exchange Online provides additional free storage per mailbox to handle retention. It’s easy to run a report to find the storage consumed by each site, but you’ll need to access the site to discover how much is consumed by the preservation hold library.

The second issue is that content searches find multiple copies of files stored in SharePoint Online sites. This might be what you want, but usually it’s confusing (Figure 3).

The Change

The change rolling out in mid-Novembers means that files with multiple versions deleted from a SharePoint Online site or OneDrive for Business account which must be retained will be preserved as a single file instead of multiple versions. Storing fewer versions should reduce the demand for storage, but I shall wait and see how things work before making a definitive statement on that point. Reducing the number of versions held for a file will also speed up deletions and eliminate errors caused when retained files had more than a hundred versions in the preservation hold library.

Existing files in the preservation hold library are not updated and behave as before. Eventually, after the retention period for items expire, the weekly background job to check and remove obsolete material from the preservation hold library will remove the older files and release storage.

The new approach applies to any file which ends up in the preservation hold library because of a retention policy or in-place eDiscovery hold.

Given the number of files now stored in SharePoint Online due to increased use by apps like Teams, the effect of AutoSave in generating multiple file versions, and the impact on tenant storage quota that retention can have, this is a good change. It also simplifies administration and might even make it easier for backup and restore scenarios (fewer files to deal with). Time will tell!

Learn more about how Office 365 really works on an ongoing basis by subscribing to the Office 365 for IT Pros eBook. Our monthly updates keep subscribers informed about what’s important across the Office 365 ecosystem.

Personal and Organization Document Management for Microsoft 365

I don’t know why Microsoft ever thought that it was wise or desirable to consider SharePoint Online and OneDrive for Business as two separate workloads. The decision might have made sense years ago, when Microsoft began to extract itself from the legacy of its on-premises servers and wanted to demonstrate that it had multiple services to offer within Office 365. It makes none in the context of today’s cloud services.

The simple fact is that OneDrive for Business is no longer an optional extra for Office 365 users. Teams uses OneDrive for Business to share files, including the components built using the Fluid framework, in chats. Recordings of Teams personal meetings also go into OneDrive for Business, and Whiteboard is about to make the transition to OneDrive storage too. If you save an email attachment from Outlook, OneDrive is the preferred target. Users are encouraged to move their files stored in well-known folders from local workstations to OneDrive for Business to take advantage of features like Autosave and differential synchronization.

Increasing Importance of OneDrive for Business

Microsoft makes large amounts of storage available to OneDrive for Business users to make it possible to store data online. All signs indicate that Microsoft will continue to move application and personal data to OneDrive for Business storage whenever possible because it makes it easier to index and search files, including eDiscovery support. In a nutshell, the central importance of OneDrive for Business to cloud users increases as time passes.

The Demise of the OneDrive Admin Center

Which brings me to the elimination of the OneDrive for Business admin center. Or at least, the move of OneDrive settings into the SharePoint Online admin center (Figure 1), which removes the need for the OneDrive admin center. The SharePoint Online admin center has always had settings which affected OneDrive for Business, like sharing controls. Now we have a single place to manage system and personal document and file management for Microsoft 365, which is what these products deliver.

Microsoft covered the move of the OneDrive settings in a July 2021 blog post. With so many blog posts, announcements, updates, and other information about different aspects of Microsoft 365 appearing each week, you might not have noticed the transition. If you go to the Settings section of the SharePoint Online admin center (Figure 2), you’ll find the OneDrive for Business controls.

Checking Sensitivity Labels and Sites

Another topic featured in Microsoft’s July blog is the new insight card to report the number of unlabeled sites. These are sites that don’t have an assigned sensitivity label. As you might notice from Figure 1, my tenant reports 128 of these sites. Given that I’ve invested lots of time working to implement sensitivity labels for container management, this seemed like a high number.

After checking the list of sites, I discovered that the set includes:

• Sites retained by a compliance policy after removal of the original Microsoft 365 group.
• System sites like the App Catalog site and the home site and its predecessor.
• Sites created for Yammer communities before the switch of the Yammer network to Microsoft 365 native mode.
• Teams created from a template (to close the gap, MC281936 describes an update rolling out soon to allow team owners to assign a sensitivity label when creating a new team from a template).
• The Viva Topics center site.
• The site created for the group used to control who can create custom templates for the Teams Approvals app.

In short, a bunch of sites turned up, some of which could do with a sensitivity label and others which don’t. In other words, a list that’s well worth reviewing.

Simplification is Goodness

I strongly approve of Microsoft’s move to incorporate OneDrive for Business management into the SharePoint Online admin center. There are still too many administrative consoles across Microsoft 365 and this step simplifies the tenant management landscape.

With the introduction of the new Exchange Online admin center and the transition of the old Security and Compliance Center to the Microsoft 365 compliance center, we’re also seeing rationalization of user interfaces. On the downside, the switchover from old to new consoles seems to be taking forever. Maybe it’s because it people need time to absorb change, but sometimes you’d wonder if it wouldn’t be better if Microsoft pulled the plaster off quickly and launched a family of new fully-functional administrative tools.

Make sure that you’re not surprised about changes which appear inside Office 365 applications (like updates to admin portals) by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers stay informed.

Search for What Participants Say During Teams Meetings

Microsoft message center notification MC260749 (last updated August 12) titled Microsoft Search: Find a meeting recording based on what was said is both technically interesting and important. Described in Microsoft 365 roadmap item 82003, the roll-out was delayed several times, but the way is now clear for Office 365 tenants to be able to search videos using spoken text along with a bunch of other changes to make Teams meeting recordings more accessible and useful. While it’s hard to say exactly when individual tenants will have all the functionality described here, I expect worldwide deployment to be complete by the end of October 2021.

Everything in OneDrive

Exposing the content of meeting recordings for search is important because it starts the process to close a major compliance gap. Up to now, transcripts for online meetings have not been searchable. The problem first surfaced when Teams stored its recordings in Stream. When the meeting finished, Stream processed the recording and created the transcript. However, the transcript remained in the Stream Azure service and was inaccessible to Microsoft Search. If something can’t be indexed by Microsoft Search, its content cannot be found by a search.

Microsoft completed the migration the storage of Teams meeting recordings from Stream to OneDrive for Business or SharePoint Online (ODSP) on August 16, 2021. All new meeting recordings from that date are in ODSP with the migration of older content from Stream to ODSP happening later. Microsoft is busy building out the rest of the Stream 2.0 platform to handle videos which don’t come from Teams. For instance, they’ve released a preview of the new Stream browser interface which supports access to videos stored in both ODSP and the original Stream store.

The move to ODSP removed the ability to create and replay transcripts for meeting recordings which exists in Stream classic. Starting September 20, Microsoft plans to remove some of the automatically-generated transcripts from older videos in Stream classic to prepare for the migration to Stream 2.0.

To fill the functionality gap, Microsoft introduced a transcription capability for Teams meeting recordings (a recent update means that if you record a Teams meeting now, you generate a transcript automatically). However, the issue of searchability remained. Because ODSP stores the recording files, Search could index file metadata like the name of the recording, but that’s about all.

The gap in indexing and searchability is now closed. Teams stores the spoken text captured during a meeting (including speaker attribution so you know who said what) and meeting metadata in the Exchange Online mailbox of the meeting organizer. Capturing the spoken text in mailboxes allows Microsoft Search to index the data and therefore makes it possible for searches to find this information. And as we’ll see, ODSP also holds a copy of the transcript to allow the words in the transcript to connect with segments in a meeting recording.

Exchange Mailbox Storage for Transcript Information

Teams stores transcript information in a folder called ApplicationDataRoot/93c8660e-1330-4e40-8fda-fd27f9eafe10/MeetingTranscriptCollection in the non-IPM part of the mailbox. Hidden means that the folder isn’t available to users through clients like Outlook, but its contents are available to administrative interfaces like Microsoft Search and programs like MFCMAPI.

Transcripts are captured as mail items. Examining the captured items with MFCMAPI, it looks like two properties for are most interesting:

• TranscriptJsonBlob: stores the spoken text captured during the meeting. In Figure 1, you can see some captured text, including the name of the speaker. When users view the transcript in Teams, the information is displayed in a nicer format. It’s also possible to download transcripts in VTT or Word (DOCX) format.
• TranscriptMetadataJsonBlob: stores metadata about the call.

Linking Words to Videos

The original implementation for Teams meeting recordings stored in Stream classic supported transcription, including the ability to edit the transcript to correct obvious errors. To allow Microsoft Search to find the MP4 file for a meeting recording based on words spoken during a meeting, a background process copies the transcript data captured in Exchange Online and indexes it against the recording to match segments of the video with the spoken words.

Replication of transcript data from Exchange Online to ODSP can take anything from 15 minutes to a day after the meeting ends. Once the process completes, you can search for text spoken in meetings and find recordings using the transcript (Figure 2).

Transcript Playback

Matching words in the transcript with meeting recordings (and eventually, any video stored in ODSP) allows concurrent playback for the two elements. Microsoft 365 roadmap item 82057, rolling out in September 2021, delivers a transcript pane for video playback (Figure 3). No ability is yet available for a video owner to edit the transcript.

Curiously, closed captions are available for only 60 days from the date of recording. In addition, Microsoft says that “Closed captions aren’t fully supported” if you move or copy a recording from its original location. Presumably, this is because the move might affect the link to the transcript data.

Making Transcription Available to More Teams Users

The option to transcribe meetings used to be restricted to accounts with enterprise E3/E5 and Business Premium/Standard. In early July, Microsoft made live transcription available (MC260564) for other licenses, including the E1, F1, academic, and Business basic SKUs, noting that this step improves the accessibility of Teams and makes meetings more inclusive for those who are hard of hearing. Microsoft followed up with MC280258 (August 24), to announce support for transcripts and captions in 27 additional languages (Figure 4) to join the previous support for U.S. English.

Another Compliance Gap Nearly Closed

All the information shared during Teams meetings is gradually coming within the scope of compliance policies. eDiscovery can already find chat, presentations, and documents, and the advent of indexed speech means that spoken comments should soon come within the scope of eDiscovery searches. This hasn’t happened yet, probably because of the work needed to export transcripts and videos in eDiscovery cases, but I am sure this capability is high on Microsoft’s agenda.

Although the captured text is sometimes inaccurate, capturing any record of spoken comments is better than nothing. As time goes by, the artificial intelligence technology used to analyze speech to create the transcript will improve in terms of accuracy and ability to handle accents.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across Office 365. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what’s happening.

Fixes a Very Old Bug First Reported in 2016

Updated: December 19, 2021

Every channel in a team has a folder in the default document library of the SharePoint Online team site associated with the team. When a new channel is created in Teams, SharePoint Online creates a new folder with the same name as the channel. The channel and folder continue to share the same name until you rename the channel, in which case the names of the channel and the folder diverge. Microsoft acknowledges that losing the naming connection between Teams and SharePoint is a problem.

The issue has existed since the earliest days of Teams. The first user voice request for Microsoft to remove confusion by making sure that the channel and folder continued to share the same name following a rename appeared on November 3, 2016. I wrote about the issue in June 2019, saying that renaming channels could be messy.

Microsoft says they will fix the problem (MC280294) and clean up the mess with an update in mid-September (Microsoft 365 roadmap item 72211). The heading for Microsoft’s is “Pairing naming convention between Teams channels and corresponding SharePoint folders,” which I think is a poor attempt at conveying the impression that the change is something good. It’s not. Instead, it’s a long (very long) overdue fix for something that Microsoft should have addressed in 2017.

Update: In message center notification MC306666 (December 18), Microsoft says that they will roll out the fix in “late February through late April (2022).” The delay in the roll-out is likely due to the need to fix bugs which came to light during testing.

The delay means that any channel renamed before the deployment of the fix will remain unpaired, unconnected, unsynchronized, and seemingly unrelated to its SharePoint folder. Given the massive growth in Teams to 250 million monthly active users and the consequent growth in SharePoint Online usage, one can only guess at how many disconnected channels exist.

One of Those Complex Software Engineering Problems

No one denies that Teams is a complex product. Teams has dependencies on and consumes many different Microsoft 365 services from Azure AD to Exchange Online. The Teams development group has done a terrific job in growing the feature set in the product and expanding its capabilities into areas like multi-geo support. As Teams development VP, Rish Tandon, explained to me last May, the engineers have faced and solved many challenging problems as they developed the product from initial idea to world-class service.

But from time to time, the Teams development group just doesn’t deliver detail as well as it should. The failure to fix the channel rename problem is a classic example. Rolling out version 2.5.0 of the Teams PowerShell module with a broken version of the New-Team cmdlet is another. Neither seem to appear to be one of the complex software engineering problems that slow products down.

The New Channel Rename

After Microsoft deploys the update to Office 365 tenants, performing channel name (Figure 1) updates the value in both Teams and SharePoint Online.

As you can see in Figure 2, the synchronization with SharePoint Online means that the channel folder has the same name as used in Teams. In the past, the folder would still be “Projects” instead of the new “Projects 2021” name.

The General channel is an exception because it cannot be renamed. This is because the General channel represents the team. In fact, because the General channel exists in every team, the Teams clients translate its name to show translated values. For instance, it’s called Général in French and Allgemein in German. The names given to other channels are not translated and keep whatever name is given when created or renamed.

Rename synchronization for channels and folders applies for standard, shared, and private channels. Channels renamed prior to the update are not adjusted. If you want the names of these channels to synchronize with SharePoint Online, you’ll need to rename them again in Teams.

Microsoft notes that the new channel name will not be used by the OneDrive sync client until the client fully processes the channel following the rename. This usually doesn’t take long.

The Long-Awaited Fix

It’s good that Teams and SharePoint are now on the same page when it comes to channel renaming. It’s taken too long to happen, but it’s better later than never.

Learn more about how Office 365 really works on an ongoing basis by subscribing to the Office 365 for IT Pros eBook. Our monthly updates keep subscribers informed about what’s important across the Office 365 ecosystem.

Cleanout Starts on September 20

As we all know, Stream is in the middle of a migration from its old platform based on Azure storage to ODSP (OneDrive for Business and SharePoint Online). All new Teams meeting recordings are now stored in ODSP and Microsoft is preparing for the next phase, which is to migrate videos stored in Stream classic to ODSP. A bunch of work is going on to prepare the way, like a new web player for video content (MC261352, last updated August 10, due for a complete deployment by the end of September) and a new way of viewing the transcript of Teams meeting recordings (MC274185, July 30). In other words, things seem to be progressing nicely on the Stream 2.0 front.

Clearing Out Old Video Transcripts

That is until you read MC279467 (published August 20. At first glance, the text seems inconsistent with the generally positive progress of Stream. The notification says that starting September 20, Stream users will be unable to access (view, search, or edit) the transcripts automatically generated by Stream for:

• New videos uploaded to Stream. However, transcripts will be generated once the newly uploaded video receives one view.
• Older videos which have not been uploaded or edited (for instance, to trim a video) in the last 3 months (U.S. East datacenter region) or 6 months (for other regions hosting Stream services like the other U.S. regions and EMEA).

In other words, Microsoft is stopping the automatic generation of transcripts for new videos and removing transcripts previously generated for some older videos. This doesn’t affect the storage of videos; it is all to do with their transcripts, which are a significant accessibility feature for people to follow what’s happening in a video. Figure 1 shows an example of the transcript viewed alongside a video recording for my “Talking Teams” interview with Rish Tandon, Microsoft VP for Teams Development.

If someone views an older video, Stream will regenerate the transcript. The same will happen if an owner edits video details and chooses to regenerate the captions (Figure 2) which make up the transcript.

Microsoft will not remove transcripts for videos when:

• The transcript has been edited (for example, to correct some of the phrasing generated by automatic transcription). Although the automatic transcript is OK for most videos, it can have problems with idioms, unclear speech, and when people talk over each other.
• Transcripts are uploaded manually.
• Videos are active. To qualify, videos are uploaded, edited, or viewed in the last six months.

Interestingly, closed captions remain available for all videos.

Why is this Happening?

It’s reasonable to ask why Microsoft is cleaning out old Stream transcripts. Cynics will say that it’s a cost-saving measure to drive profits and point to the recent announcement about increased monthly fees for Office 365 plans. I think the answer is a little more prosaic. Microsoft is preparing for a migration. The nature of migrations is that they are often painful, complex operations. Removing data that doesn’t need to be migrated makes sense, so the possible reason is that Microsoft wants to clean out transcripts for videos which haven’t been watched or edited in a while so that they don’t need to migrate the data to ODSP. Given that transcripts probably need some massaging on ODSP to be indexed and become searchable there, this is a plausible driver.

Microsoft isn’t saying why the clean-out is happening but given that the migration is expected sometime early in 2022, it really doesn’t matter. After all, if someone notices that a transcript is missing for an antique video, it’s easy to regenerate it.

Removing Friction from Sharing

External sharing of SharePoint Online and OneDrive for Business elements like documents, list items, and folders uses a technology called ad-hoc external sharing. When users share items with external recipients, SharePoint Online and OneDrive for Business use a one-time passcode to allow that person to verify their identity. A one-time passcode (OTP) is a way to authenticate the identity of people outside your Microsoft 365 tenant when Azure AD cannot verify their accounts using another method.

The ad-hoc sharing mechanism works but requires several steps before the user can open the shared item.

• User receives the email telling them that someone has shared an item with them.
• User attempts to access the item. SharePoint Online detects that it must verify their identity, so sends an 8-digit OTP to their email address.
• The user receives the email (or finds it in their Junk Email folder) and enters the code (or cuts and pastes the code) into the form (Figure 1). Passcodes are valid for 30 minutes. The Keep me signed in checkbox controls the saving of the authentication cookie to disk to allow the user to reuse it for authentication until the cookie expires.
• SharePoint Online verifies the code and if correct, allows access.

Integrating SharePoint External Sharing with Azure AD B2B

To improve external sharing, in October 2021, Microsoft plans to turn on Email one-time passcode authentication for Azure AD by default for all tenants. Like the current ad-hoc sharing, the new mechanism features one-time passcodes. The big difference is that successful authentication results in the automatic creation of Azure AD guest accounts for external users.

Microsoft is making the change because it will enable new functionality for external recipients. Among the advantages cited are:

• Because they will have Azure AD guest accounts, external recipients who redeem one-time passcodes won’t need to create a Microsoft (MSA) account.
• Administrators can manage details of guest accounts, such as assigning them user-friendly display names or photos.
• Other Microsoft 365 features, such as team membership or sharing of other SharePoint Online and OneDrive for Business resources, can take advantage of the guest accounts.
• Guest accounts are subject to conditional access policies.
• Tenants that configure Google Cloud federation with Azure AD can share resources with federated accounts.
• The Azure AD B2B Collaboration policy controls external sharing. In other words, you can whitelist or blacklist domains that you want to limit sharing with or stop sharing with (a tenant can choose to deploy either a whitelist or blacklist, but not both).

Configuring Email OTP Authentication for Azure AD

While they can wait until Microsoft enables Email OTP authentication for Azure AD in October (or opt to disable the capability), tenants can choose to use email OTP authentication for Azure AD today. To enable the feature, go to the identity providers section and configure the email one-time passcode provider as shown in Figure 2.

As you can see, this is where you can disable the feature, if that’s what you want to do.

Some configuration is necessary for SharePoint Online to integrate with Azure AD B2B and use email OTP authentication (or as Microsoft says in its documentation, Azure B2B Invitation Manager). Do this with the SharePoint Online management module by connecting and running the Set-SPOTenant cmdlet to update the necessary settings:

Set-SPOTenant -EnableAzureADB2BIntegration $True
Set-SPOTenant -SyncAadB2BManagementPolicy $True

Bizarrely, while you can use the Get-SPOTenant cmdlet to retrieve the value of the EnableAzureADB2BIntegration setting, it doesn’t report a value for SyncAadB2BManagementPolicy.

Using Email OTP Authentication for Azure AD

With Email OTP authentication for Azure AD enabled and connected to SharePoint Online, the following happens for external sharing.

The user creates a sharing link as usual (existing sharing links continue to work and there’s no need to recreate links).

• Azure AD checks the directory and creates a guest account if an account doesn’t already exist for the external recipient.
• The external recipient receives the email notification of sharing and clicks the sharing link.
• Azure AD enters a validation process. Users with Azure AD or MSA accounts enter their email address and, if this is valid for the sharing link, the Azure AD Invitations service invokes the consent process to allow it to sign in the new guest account (Figure 3). Users without Azure AD or MSA accounts sign in using the one-time passcode authentication procedure to validate their identity.
• If the external recipient grants consent, Azure AD signs them in and allows access to the shared resource.

The external recipient now has a guest account in the tenant. They can use this account to access other resources shared with them. And if the authentication token granted through a sign-in is still valid, they won’t have to sign in again to open other shared resources. When the guest account accesses tenant resources, Azure AD captures audit records (Figure 4).

The tenant can manage the guest account like any other account, including imposing conditional access policies to restrict access where necessary, like confidential sites marked with an authentication context with a sensitivity label.

Guest Accounts Need Management

Using guest accounts to manage external access to SharePoint Online and OneDrive for Business resources is a sensible move. It’s a lower friction mechanism for external people that’s easier for tenants to operate. That being said, guest accounts do need to be managed as it is all too easy to allow obsolete or unused accounts accumulate in Azure AD. Microsoft doesn’t provide any tools to clean up old guest accounts, but you can do the job with PowerShell.

Learn more about how Office 365 really works on an ongoing basis by subscribing to the Office 365 for IT Pros eBook. Our monthly updates keep subscribers informed about what’s important across the Office 365 ecosystem.

Highlighting Who Already Has Access to Shared Information

Message center notification MC263839 (updated August 6 – Microsoft 365 roadmap item 83725) is all about new “Shared with” information which now appears on the control used to create sharing links. Well, it will when the roll-out completes in mid-August. Although tagged for OneDrive for Business, this change applies to both OneDrive for Business and SharePoint Online.

The idea is that the control now lists the set of people whom a file, folder, or list is already shared with so that owners know (at a glance – if they bother) how many people already have access and who they are.

Viewing Sharing Information in Different Circumstances

Showing sharing information works better in some situations than others. For example, if you share a file from a site owned by a Microsoft 365 group (or team), the set of sharing information includes:

• The group
• Group owners
• Group members
• Group visitors

It seems like this information could be filtered so that only the group is shown. The full set (Figure 1) doesn’t add value as the three entries (for SharePoint groups used to manage access) are defunct in the context of a group-connected site.

The information is more valuable when sharing a file from a site that isn’t connected to a group or OneDrive for Business. For instance, Figure 2 shows that a file is shared with 2 sharing links plus five specific users (tenant and guest accounts). Although you can mouse over an avatar to see who has access, it’s obviously better if the tenant and guest accounts have photos as this allows the sharing dialog to include thumbnails for each person.

Several tests showed that up to six entries can appear in the dialog. If more people have access, you’ll see an ellipsis choice to go to the Manage Access menu to view full details of the existing sharing.

The mock-up used in MC263839 (Figure 3) uses larger thumbnails. It’s an interesting insight into the design decisions that must be taken to settle on the final look and feel for user interfaces.

Making Sharing More Transparent

This change is another to build out capabilities in the sharing control to make it more powerful and useful. Although some will probably say that it’s just window dressing or eye candy, I rather like seeing the set of people with access to a file, folder, or list highlighted in this manner. It’s the small things that often have the biggest impact!

Learn more about how Office 365 really works on an ongoing basis by subscribing to the Office 365 for IT Pros eBook. Our monthly updates keep subscribers informed about what’s important across the Office 365 ecosystem.

Only for Recordings of New Teams Meetings

Updated 21 May 2022

Announced in MC274188 (July 30), in late September, Microsoft planned to enable meeting recording auto-expiration for new Teams meeting recordings (TMRs) stored in SharePoint Online and OneDrive for Business (Microsoft 365 roadmap item 84580). The new feature will move the MP4 files used for TMRs to the site recycle bin when their expiration date lapses. For enterprise users, the expiration period is 120 days after the creation of the recording. A reduced period of 30 days applies for academic users with the Office 365 A1 license. Once in the recycle bin, the MP4 files follow the standard SharePoint file deletion cycle. Auto-expiration for TMRs is available for all Office 365 and Microsoft 365 licenses which contain Teams.

TMRs are the first workload to move video storage from the classic Stream Azure-based platform to SharePoint Online and OneDrive for Business (ODSP), From August 16, 2021, all new TMRs will be in ODSP. Even though tenants have a lot more storage quota available (especially in OneDrive for Business for recordings of personal meetings) than in Stream, the new policy aims to restrict the amount of storage occupied by TMRs (roughly 400 MB per hour).

Update: Following a series of earlier delays, on January 31, 2022, Microsoft pushed deployment out to late March 2022 to make sure that when they start to delete files, they remove the right files. At the same time, Microsoft increased the default retention period from 60 to 120 days for all tenants that haven’t configured a custom retention period. Eventually all the blocking factors were removed and Microsoft began to roll out the auto-expiration of Teams meeting recordings feature in early April.

Setting a New Expiration Period for TMRs

Microsoft says that 96% of TMRs are not watched again in the 60 days (and 99% after 110 days) following the original meeting, which is why they’ve chosen this to be the default expiration period. Users can change the expiration period for individual TMRs by updating file properties through the file details pane (selecting preset values of 14, 30, or 60 days, a custom date, or Never Expire). Organizations can set a default expiration period for newly created TMRs using the Teams meeting policy assigned to user accounts. For example, to set the default expiration period for recordings of meetings made by people assigned the VIP User Meeting Policy, run the command:

Set-CSTeamsMeetingPolicy -Identity "VIP User Meeting Policy" -NewMeetingRecordingExpirationDays 120

Originally, Microsoft’s documentation described a maximum expiration period is 99,999 days (273 years). Subsequently, problems emerged when tenants set such a high value and the safe limit was found to be 9,999 days, which should be more than enough to keep any normal recording (remember, you can apply a retention label to keep recordings for longer). The minimum is 1 day, and you can set the value (in PowerShell) to -1 to set meeting recordings to never expire. The expiration period for A1 users can only be reduced from the default 30 days.

You can also update the auto-expiration period for meeting policies through the Teams admin center (November 2021 update). Interestingly, the Teams admin center allows a range of between 1 and 99999 days! I’ve asked Microsoft to clarify whether the supported period is 9,999 or 99,999 days. If you want to go higher than 9,999 days, maybe the best approach is to set expiration to never expire.

Background processes run to evaluate TMRs in ODSP to check their expiration date. If the expiration process detects an expired file, the process moves the file into the recycle bin and clears the expiration date field. Recording owners receive email notifications when OneDrive moves expired recordings into the recycle bin (Figure 2). If necessary, they can rescue important recordings from the recycle bin for up to 90 days after deletion. Once moved back from the recycle bin, the recording has no retention date set and will therefore not be evaluated for deletion again.

To help users understand when a recording approaches expiration will see visual indications in:

• Beside the link to the meeting recording in the meeting chat. Anyone with view access to the recording sees the expiration notice.
• Two weeks before expiration, a red icon appears beside the MP4 files for TMRs in the Recordings folder of OneDrive for Business accounts (personal meetings) or SharePoint Online sites (channel meetings).

Auto-expiration applies only to new TMRs. Existing TMRs stored in either ODSP or Stream do not have an expiration period. Auto-expiration is only available for TMRs and cannot be used with other file types held in ODSP. Expiration dates are kept if users move recording files to a different site (it’s the same file). They are not when users copy recording files (it’s a different file). Downloading and uploading a recording creates a new file with no expiration date. If you want to be sure that the expiration process does not remove a Teams meeting recording, apply a retention label to the file.

Tenant administrators can track the creation of TMRs in OneDrive for Business and SharePoint Online by using PowerShell to extract and analyze audit events.

Auto-Expiration and Retention

Auto-expiration is a good housekeeping rather than a compliance feature. It will help organizations cope with a swelling collection of TMRs in user OneDrive for Business accounts and SharePoint Online sites but will do nothing to help with data governance. Two interesting developments due to arrive soon are automatic transcription for TMRs and indexing of transcripts. From a compliance perspective, this means that it will be possible to search for words spoken during a meeting and be able to put those words in the context they were spoken. Obviously, this is a big advance in compliance capabilities.

To take advantage of spoken word retrieval and make sure that transcripts and videos are available to eDiscovery investigators, you obviously need to retain TMRs. For this reason, a retention label on a TMR prevents the auto-expiration process removing recording files until the retention period assigned in the label lapses. Also, a retention label mandating deletion after a period takes precedence over auto-expiration, meaning that if the retention label has a shorter retention period than the auto-expiration date, that’s when SharePoint will remove the file.

Precedence applies for retention labels assigned manually or via an auto-label policy (available to tenants with Office 365 E5). Organizations which leverage retention labels to preserve the recordings of important Teams meetings might not see much change after Microsoft introduces the new auto-expiration feature.

Make sure that you’re not surprised about changes which appear inside Office 365 applications by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers stay informed.

On

July 12, Microsoft released version 16.0.21411.1200 of the SharePoint Online PowerShell module, installable from the PowerShell Gallery. The updated module is especially notable because the Get-SPOSite cmdlet boasts three new properties to inform administrators if sites are connected to Teams. The properties are:

• IsTeamsConnected: Set to True if the site is connected to a team.
• IsTeamsChannelConnected: Set to True if the site is connected to a Teams private or shared channel. These sites have IsTeamsConnected set to False.
• TeamsChannelType: Set to None for teams-connected sites and to Private for sites belonging to private channels or Shared for sites belonging to shared channels (due later this year).

New View of Teams Sites

The updated module aligns with the effort to make SharePoint Online more manageable for teams-connected sites through a UI refresh and by showing details of channel-connected sites in the SharePoint Online admin center with a new Sites connected to Teams view (Figure 1). This view is in preview at present and should become generally available later this year.

The new view lists all teams-connected sites and indicates how many of the channels in a team have a channel-connected site. Clicking the link for the channel sites exposes further information (Figure 2).

The net effect of the change is that SharePoint administrators will see information about teams-connected and channel-connected sites in the SharePoint Online admin center which isn’t available today. Access to information about channel-connected sites is read-only. This is because these sites inherit settings from the parent team site. It also ensures that management of the channel-connected sites remains with the channel owners.

Using the New Teams Site Properties in PowerShell

Coming back to PowerShell, the new properties make it easier to find and report details of Teams-connected sites. You can still do this using the Get-UnifiedGroup cmdlet, which offers the advantage of exposing group information more easily. Now you have the option to check if team-connected sites have private or shared channels.

Here’s some quick and dirty PowerShell to report channel-connected sites. The code:

• Creates an array of sites connected to Teams.
• Creates another array of channel-connected sites.
• Loops through the sites array to see if any matching channel-connected sites are present and reports these sites. Remember, a team can have up to 30 private channels.

# Find Teams-connected site
[array]$Sites = Get-SPOSite -Limit All | ? {$_.IsTeamsConnected -eq $True}
# Find channel connected sites
[array]$ChannelSites = Get-SPOSite -Limit All | ?{$_.IsTeamsChannelConnected -eq $True}

$SiteCount = 0
$ChannelData = [System.Collections.Generic.List[Object]]::new()
ForEach ($Site in $Sites) {
   [array]$MatchedSites = $ChannelSites | ? {$_.Url -Match $Site.Url}
   If ($MatchedSites) {
      $SiteCount++
      ForEach ($MSite in $MatchedSites) {
       $ReportLine = [PSCustomObject][Ordered]@{  
         Parent      = $Site.URL
         Title       = $Site.Title
         URL         = $MSite.URL
         ChannelType = $MSite.TeamsChannelType }
       $ChannelData.Add($ReportLine)
      } # End ForEach
   } # End if
} # End Foreach
Write-Host ("Total of {0} channel-connected sites found for {1} sites" -f $ChannelData.Count, $SiteCount)

Here’s an example of a record for a channel-connected site:

Parent      : https://office365itpros.sharepoint.com/sites/CorporateAcquisitionPlanning2020
Title       : Corporate Acquisition Planning 2020
URL         : https://office365itpros.sharepoint.com/sites/CorporateAcquisitionPlanning2020-LegalDiscussions
ChannelType : PrivateChannel

Figuring Out Inconsistencies

Interestingly, I found instances where the Microsoft 365 group which originally owned a team-connected site was no longer available in the tenant, but team-connected and channel-connected sites still existed. This is likely due to retention policies where sites come within the scope of a retention policy and the group did not. I used the following code to find these channel-connected sites:

# See if we can find parent groups
ForEach ($CSite in $ChannelSites) {
   $MatchURL = $CSite.URL.Split("-")[0]
   $Match = $Sites | ? {$_.Url -Match $MatchURL}
   If (!($Match)) {Write-Host "Can't find parent team-connected site for channel-connected"  $CSite.URL }
}

I also found some inconsistencies between the number of channel-connected sites reported using the new properties and the older method of using the site template to identity these sites:

$TTSites = Get-SPOSite -Limit All -Template "TEAMCHANNEL#0"

Some testing revealed that this is due to some provisioning delays in updating site properties. Essentially, if you update the membership of a channel, you force synchronization to update site properties.

Exposing Channel-Connected Sites

There’s no doubt that these updates add value. When Microsoft introduced private channels in November 2019, many complained that the sites used for sharing documents in private channels were invisible (they weren’t, but you had to use PowerShell to see them). Exposing details of private channels (and soon, shared channels) in the SharePoint Online admin center is a good thing: adding the properties to allow better filtering and reporting of channel-connected sites in PowerShell is even better.

Learn more about how Office 365 really works on an ongoing basis by subscribing to the Office 365 for IT Pros eBook. Our monthly updates keep subscribers informed about what’s important across the Office 365 ecosystem.

Change to Remove Inconsistency

Retention policies and retention labels both make sure that workloads like SharePoint Online retain information needed by organizations. Retention policies are broader in scope and apply default retention to any location coming within their scope. For instance, if you apply a retention policy to a set of SharePoint sites, any file within those sites come within the scope of the policy. Retention labels are more granular and apply to individual items, whether assigned by users or through auto-label policies (like the example of using an auto-label policy to retain Teams meeting recordings). Because they are more specific, retention labels take precedence over retention policies.

It’s up to an application how to implement the application of retention labels to items. It’s also up to applications how to respect the fact that a retention label exists on an item. Different behaviors have existed in the SharePoint Online and OneDrive for Business browser interfaces since the introduction of retention labels in 2017. According to MC264360 (June 24) – Microsoft 365 roadmap item 82063, Microsoft is closing the inconsistency and SharePoint Online will adopt the OneDrive for Business approach.

Deleting SharePoint Online Items

Today, if you try and delete an item in a SharePoint Online document library, the UI prompts for the deletion to proceed and if confirmed, attempts to delete the item. If the item is labeled, the deletion fails (Figure 1) and the user sees that removal isn’t possible because of the label.

There’s nothing to stop the user removing the label and then deleting the file, unless it’s a record label (only a site administrator can change a record label).

By comparison, you can delete an item in a SharePoint Online document library which comes within the scope of a retention policy. Although seemingly inconsistent (because the organization wishes to retain the items by policy), SharePoint Online allows the deletion to proceed and moves the item into the site recycle bin. Eventually, when the item expires in the recycle bin, SharePoint Online moves it into the site’s Preservation Hold library where it stays until its retention period lapses.

OneDrive’s Streamlined Approach

OneDrive for Business takes a streamlined approach to item deletion and allows users to remove items as they wish (Figure 2).

Deleted items go into the OneDrive for Business recycle bin (Figure 3). Users can recover deleted files from there using the Restore your OneDrive feature.

After 90 days, deleted files leave the recycle bin for either permanent removal or retention. If a retention policy or label applies to an item, it moves to the Preservation Hold library (Figure 4) and stays there until its retention period lapses. Of course, retention can be a complex business and an item might come under the scope of a retention policy after retention due to a label lapses. In any case, once no further retention applies to an item, a background job removes the item. Removed items are irrecoverable unless a backup exists.

The Goodness of Consistency

You can argue that either approach makes sense. Some like it that SharePoint Online stops people deleting labeled items. It’s a form of affirmation that the file is important. On the other hand, allowing deletion to happen but preserving files needed for retention is a lower-friction method which prevents potential user confusion (why can I delete that file but not this one?). Overall, achieving consistency across OneDrive for Business and SharePoint Online is a good thing and lowering friction is also a good thing, especially if it stops some support calls. We’ll see how users react (or even notice) after Microsoft rolls out the change in August.

Learn more about how Office 365 really works on an ongoing basis by subscribing to the Office 365 for IT Pros eBook. Our monthly updates keep subscribers informed about what’s important across the Office 365 ecosystem.

Opening the Black Box

My article about how to create an auto-label policy to apply retention labels to Teams meeting recordings resulted in several questions. As I noted in the article, tracking the progress of auto-labeling can be challenging due to the black-box nature of the background processes which search for recording files to label. One suggestion was to use the technique explained in this blog post to use the SharePoint Online PnP PowerShell module to connect to sites and retrieve information about retention job activity. For example:

$SiteURL = "https://office365itpros.sharepoint.com/sites/Office365Adoption/"
Connect-PnPOnline -Url $SiteURL -Interactive
get-pnppropertybag -key "dlc_policyupdatelastrun"
get-pnppropertybag -key "dlc_expirationlastrunv2"
2/23/2021 11:18:42 PM
2/2/2021 8:02:41 PM

The first value (dlc_PolicyUpdateLastRun) is the date when the background job to evaluate retention dates for items last ran. The second (dlc_ExpirationLastRunv2) tells you the last time the background job ran to execute the retention action defined in labels when retention periods expire.

The background jobs which evaluate retention dates and execute actions are not directly connected to auto-label processing, but they give an insight into how SharePoint Online processes sites. In a nutshell, if a site is active, the background jobs process its content. If not, the site is ignored. This makes a lot of sense because it avoids SharePoint doing a bunch of work to check items in sites where no work is necessary. I don’t know if another value stores a date when action must be taken to process expired items, but it would make sense if it did.

These values date back to legacy management processing in SharePoint on-premises and while they still work, Microsoft introduced a new retention processing engine last year which might eventually nullify their use.

Off to the Audit Log

Interesting as these values are, they don’t tell us anything about the application of labels. In the last article, I mentioned that the Office 365 audit log captures the TagApplied event when a person or policy applies a retention label to an item. The audit events are available roughly 15 minutes after they occur, so this source seemed like a good place to investigate.

I ended up writing a script to do the following:

• Find audit records for the TagApplied event in the last 14 days.
• Filter the records to find those which apply the retention label used by the auto-label policy. The same filter makes sure to only select records for policy rather than user application.
• Find the date of the recording from the file name generated by Teams. For instance, a recording named Call with James Ryan-20210217_141123-Meeting Recording.mp4 started at 14:11 on 17 February 2021.
• Calculate how long it took to auto-label the recording file (the difference between the date the call started and the audit record).
• Write the details to a SharePoint list to make the data available for additional analysis.

Here’s the main loop of the code to process the audit records. You can download the complete script from the Office 365 for IT Pros GitHub repository.

[array]$Records = (Search-UnifiedAuditLog -Operations TagApplied -StartDate $StartDate -EndDate $EndDate -Formatted -ResultSize 2000)
$TaggedRecordings = [System.Collections.Generic.List[Object]]::new() 
ForEach ($Rec in $Records) {
   $AuditData = $Rec.AuditData | ConvertFrom-Json
   If (($AuditData.DestinationLabel -eq $RetentionLabel) -and ($AuditData.UserType -eq "CustomPolicy")) { 
      $RecordingFileName = $AuditData.DestinationFileName
      $DateLoc = ($RecordingFileName.IndexOf("-202")+1)
      $RDate = $RecordingFileName.SubString($DateLoc,8)
      $TimeLoc = $DateLoc + 9
      $RTime = $RecordingFileName.SubString($TimeLoc,4)
      $RecordingDateTime = $RDate + $RTime
      [datetime]$RecordingDate = [datetime]::ParseExact($RecordingDateTime,"yyyyMMddHHmm",$null)
      [datetime]$TaggingDate = Get-Date($AuditData.CreationTime)
      $TimeToTag = ($TaggingDate - $RecordingDate)
      $TotalSeconds = $TotalSeconds + $TimeToTag.TotalSeconds
      $TimeToTagFormatted = "{0:dd}d:{0:hh}h:{0:mm}m" -f $TimeToTag
# Add the data about our record          
      $DataLine = [PSCustomObject] @{
         Workload            = $AuditData.Workload
         Recording           = $AuditData.DestinationFileName
         "Retention Label"   = $AuditData.DestinationLabel
         "Tagging Date"      = Get-Date($AuditData.CreationTime) -format g
         "Recording date"    = Get-Date($RecordingDate) -format g
         "Days to label"     = $TimeToTagFormatted
         Site                = $AuditData.SiteURL
         FullURL             = $AuditData.ObjectId }
    $TaggedRecordings.Add($DataLine) 
   } # End if
} # End ForEach

The Final Answer

After processing all the audit records, I know what Teams meeting recordings the auto-label policy has labelled and how long it took on average for an item to receive a label.

25 audit records found for auto-applying the Teams recordings retention label between 09/06/2021 19:36:43 and 23/06/2021 19:36:43
Average elapsed time to auto-label recordings: 02d:13h:28m
The report file is available in C:\temp\TaggedTeamsRecordings.csv.

The average time between creation and labeling depends on the gap between the meeting and when the labeling job runs. This seems to be on a weekly workcycle and usually runs over the weekend, so labeling a recording can take anything up to a week. An average of between two and four days is normal given that Teams captures new meeting recordings over the work week.

The same technique can be applied to track the progress of any auto-label policy.

Keep up with the changing world of the Microsoft 365 ecosystem by subscribing to the Office 365 for IT Pros eBook. Monthly updates mean that our subscribers learn about new development as they happen.

Initial Confusion from Garbled Message Center Notification

When Microsoft published message center MC261534 on June 11, they confused many people. The text was imprecise and inaccurate, starting off with “With this new feature, when you create a team in Microsoft Teams, a SharePoint team site will automatically get created in tandem.” It required several readings to understand what Microsoft wanted to communicate. Microsoft 365 roadmap 81945 didn’t help, saying “You will see specific updates, including updates to site features, site permissions alignment, site classifications and sensitivity labels, and improvements to the user interface.”

On June 15, clarifying text appeared. It’s still not as clear as I would like. Here’s what I believe will happen when the changes appear in late June to complete rolling out by early August.

Normal Channels

During the provisioning of a new team, the process creates a new SharePoint team site to store documents belonging to the team. Every team has a General channel, so the site also receives a General folder in the site’s document library. When a new channel is added to the team, the action creates a corresponding folder in the document library. Among other data, the channel folders store sub-folders for email sent to the channel. There can be up to 200 channels in a team.

Figure 1 shows a folder in a SharePoint Online document library for a normal Teams channel. In the user interface, we can see important visual elements such as:

• The access type is shown as Private group. In other words, users must be added to the team to gain access. If the team allows anyone to join, it is a Public group.
• The sensitivity label/classification is Guest Access. This value will depend on the labels/classifications defined in the tenant.
• The Settings menu contains Site permissions.
• There is a Go to channel link in the command bar.

SharePoint administrators and team owners (site administrators) can edit settings like the classification/sensitivity label through the Site information link in Settings. They can also amend site permissions through the Site permissions link. It’s rare to edit permissions (membership) of teams-connected sites through SharePoint; the usual approach is to make membership changes through Teams because this updates the team membership roster immediately.

Private Channels

Microsoft introduced private channels in September 2019. A private channel is a collaboration space within a team with access limited to a subset of the team’s membership. The subset of team members become the channel members There can be up to 30 private channels in a team.

Like normal channels, when a private channel is created, Teams creates a folder within a SharePoint document library to store documents shared in the channel. Unlike normal channels, all of which have their channels in the default SharePoint site owned by the team, private channels have their own SharePoint site. This arrangement ensures that Teams can restrict access to the channel members.

The SharePoint sites for private channels inherit their settings from the parent team. You can try to make changes to the site, but a synchronization process will overwrite the changes to restore the team settings.

What’s changing in MC261534? The user interface for the SharePoint document library already has no Site permissions link in the Settings menu and a Private channel indicator is visible. First, sensitivity labels/classifications become read-only so site administrators (channel owners) won’t be able to update this value in Site information. Second, the big Go to channel conversation button visible in Figure 2 is moving to the command bar (as it is in Figure 1).

Microsoft says that the update “improves the template utilized for the channel sites by having the classic SharePoint features off-by-default and will employ the full capabilities of the modern team site.” This refers to the fact that private channel sites use a special SharePoint site template called TeamChannel#0. The special template is adjusted by Microsoft to remove features like Site permissions.

The SharePoint Online admin center does not expose sites used by private channels. This is deliberately done because these sites inherit their settings from the parent team. However, you can find a list of the sites and find out which team a site belongs to with PowerShell:

$Sites = Get-SPOSite -Template "TeamChannel#0" -Limit All
ForEach ($Site in $Sites) {
   $SPOSite = Get-SPOSite -Identity $Site.url -detail
   $Group = Get-UnifiedGroup -Identity $SPOSite.RelatedGroupID.Guid
   Write-Host "Team" $Group.DisplayName "owns private channel site" $Site.URL}

Preparing for Shared Channels

The changes are appropriate and will improve matters. I don’t know if many people try to update sensitivity labels from SharePoint Online, but if they did, those changes would be reversed by synchronization. Even so, it’s better to stop channel owners trying to make changes.

What’s also clear is that Microsoft is preparing for shared channels (aka Teams Connect), due to be available later in 2021. Shared channels also use a separate SharePoint team site to share information among the channel members. There’s a fair chance that shared channels will be more popular than private channels, and if this is true, we’ll see more “special” channels created. Making sure that everything is ready for shared channels makes perfect sense.

Keep up with the changing world of the Microsoft 365 ecosystem by subscribing to the Office 365 for IT Pros eBook. Monthly updates mean that our subscribers learn about new development as they happen.

Make Web Sites into Apps

I confess to have been a little underwhelmed by Microsoft’s June 11 announcements (MC261535 and MC261537) that it will soon be possible to install OneDrive for Business and Microsoft Lists as Progressive Web Apps (PWAs). The relevant Microsoft 365 roadmap items are 80240 (OneDrive) and 80241 (Lists).

I should explain that I’m not so impressed as others might be because I’ve been installing various Office 365 web pages as apps in Edge for months (any Chromium-based browser works, Safari doesn’t). OWA works well as an app. The basic idea is that you use the Apps option in the browser to install a site as an app. Figure 1 shows what happens when you install OneDrive for Business as an app.

The only other thing to do is to name the app (Figure 2).

The web pages installed as apps show up in the Windows start menu (Figure 3) and can be pinned to the taskbar.

All About Access

The big advantage gained by installing web pages as apps is access. For instance, given the number of SharePoint sites in use today (many created by Teams), it’s often convenient to have an app pointing to a document library you use frequently. When an app starts, it has its own window. However, the functionality of the web page works in an app exactly like it does when it runs in a browser tab. As I said, it’s all about access, or rather, making information you use frequently more accessible.

If you can make Office 365 web pages into apps today, what’s Microsoft doing in MC261535 and MC261537? I think a couple of reasons exist:

• Make people aware that they can access OneDrive for Business and Lists as apps.
• Tune the pages so that they work well as PWAs.

Project Nucleus Arrives

Nice as it is to make OneDrive and Lists into apps, I’m much more impressed by the news in MC261538, which covers the introduction of a new general-purpose synchronization engine to the OneDrive sync client (Microsoft 365 roadmap item 68809).

Microsoft discussed Project Nucleus at the Ignite 2020 conference and said that they would use it to make Lists available offline (Figure 4). That’s what is being delivered with roll-out beginning in early July and due for completion in early August. Initially, Nucleus is only available for Windows 10 workstations.

A separate Microsoft Nucleus.exe runs to synchronize Lists. According to Microsoft, “the sync process begins when a user first navigates to any list or to the Lists web app. All eligible lists that are visible from the Lists app will be synced. Common operations on lists, such as changing list views, sorting, filtering, and grouping happen locally and finish quickly even on very large lists. All of these operations continue to work offline. Edits sync between your device and the cloud and you can resolve merge conflicts if there are any.”

Microsoft has done a lot of work over the years to improve the OneDrive sync client by adding features like differential synchronization to make it capable of dealing with large files. Nucleus takes on the job of dealing with the synchronization of large and complex datasets, apparently using SQL Lite as a metadata store to allow users to continue working during network outages or when the network connection is flaky. Microsoft says that “requests are handled through a secure localhost HTTP server” and that complete documentation covering the management of Nucleus is on the way.

Two Sides of the Same Coin

PWAs and Nucleus are linked in the grand plan to make ODSP information more accessible. Web sites installed as apps need offline capability and Nucleus provides this ability for OneDrive for Business and SharePoint (ODSP) apps in the same way as other local stores deliver for apps like OWA and Teams.

Keep up to date with developments like Project Nucleus by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers understand the most important changes happening across Office 365.

Protect Most Confidential SharePoint Online Sites

Adding to their container management capabilities, a feature for sensitivity labels demonstrate how to use authentication contexts with Entra ID conditional access policies to ensure secure access to information in SharePoint Online sites.

An authentication context is a way to tag information which needs special attention. First introduced in March 2021, authentication contexts are additional information required by a service provider before it grants access to a resource. Microsoft positions authentication contexts as a method to “target policies for data and actions within an app.” Put another way, authentication contexts allow organizations to apply more granular control to targeted data.

As implemented for sensitivity labels, authentication contexts link conditional access policies to let applications like SharePoint Online know that users must provide some extra information before they can access information in labelled sites. Let’s work through an example.

Define an Authentication Context

The first thing to do is to define an authentication context. In the Security section of the Entra ID admin center, open the Protection section, select Conditional Access and choose Authentication context. I created a context called Require MFA (Figure 1). The name is what you’ll see in apps like sensitivity labels. The ID for the context shown towards the bottom of the screen (c1 in this case) is an internal value written into the settings of sensitivity labels which use authentication contexts.

The Publish to apps checkbox is set, meaning that this authentication context is visible to apps like sensitivity labels and can be used by conditional access policies.

Create a Conditional Access Policy

The next step is to create a conditional access policy to use the authentication context to force multi-factor authentication. Conditional access policies can range from very simple to very complex. To prove that everything works as expected, I created a simple policy which blocks access to resources tagged with the authentication context unless the user authenticates with MFA. The new thing here is that you select the authentication context instead of apps (Figure 2). The requirement to use MFA is selected in the Grant section (not shown here).

Another easy thing to include in a conditional access policy to protect sensitive data is to include a terms of use (TOU) check to force users to read and acknowledge conditions governing access before they gain access to a site.

Updating the Sensitivity Label

To protect SharePoint Online sites with the conditional access policy, we configure a sensitivity label with the authentication context. Figure 3 shows the UI for sensitivity labels where we’ve selected the authentication context.

Any attempt to access a site with the label causes SharePoint Online to look at the label settings to see if an authentication context is specified. If one is found, SharePoint Online queries Entra ID to find which conditional access policy is associated with the authentication context and invokes the policy. In our example, attempts to access the site succeed if the user authenticates with MFA and fail otherwise.

Using PowerShell to Assign Authentication Contexts to Sites

The value of Sensitivity labels is that they make it easy to assign a set of different controls to containers, including conditional access policy protection to SharePoint Online sites. However, if your tenant doesn’t use sensitivity labels (yet), you can update sites with PowerShell. After you assign a label with an authentication context to a site, SharePoint Online updates two site settings:

• ConditionAccessPolicy: Holds the name of the conditional access policy protecting the site. When using an authentication context, this value is “AuthenticationContext.”
• AuthenticationContextName: Holds the name of the selected authentication context. As per Figure 1, it is “Require MFA.”

Check the values by running the Get-SPOSite cmdlet:

Get-SPOSite -Identity https://office365itpros.sharepoint.com/sites/SuperConfidential | ft con*, aut*

ConditionalAccessPolicy AuthenticationContextName
----------------------- -------------------------
  AuthenticationContext Require MFA

Now we know the properties to update, we can update other sites using the Set-SPOSite cmdlet:

Set-SPOSite -Identity https://office365itpros.sharepoint.com/sites/ProjectAjax -AuthenticationContextName "Require MFA" -ConditionalAccessPolicy "AuthenticationContext"

Testing the Block

To test that everything worked as expected, I used Teams. Many SharePoint sites created today are linked to Teams, so it seemed like a good test scenario. As expected, when I tried to access a site assigned the sensitivity label without using MFA, the Teams Files channel tab failed to connect and displayed an error (Figure 4).

Microsoft says that they are upgrading the Teams Files channel tab to prompt for MFA authentication when necessary. This will make access smoother and avoid users seeing the instant fail in the current experience.

When I tried to open the site using the SharePoint Online browser interface, it opened successfully. This puzzled me for a moment until I realized it was because the account had gone through a self-service password reset (SSPR) challenge and that the credentials gathered through that process satisfied the MFA requirement. This is because Microsoft now uses combined security information for MFA and SSPR (if your organization doesn’t use combined registration, you can enable it using these instructions). When I encountered the SSPR “your organization needs more information” challenge, it didn’t immediately make me think that MFA was involved, but it was!

To confirm that Entra ID enforces the conditional access policy for the site, you can check the Entra ID sign-in logs. Figure 5 shows that Entra ID invoked the policy because a user covered by the policy signed into SharePoint Online. The sign-in record doesn’t capture details of the site, but if only one sensitivity label uses the conditional access policy and you know who signed in, you can put two and two together to know what happened.

Increasing Control

Linking sensitivity labels with conditional access policies increases the granularity of control a label can exercise over SharePoint Online sites and increases the usefulness of sensitivity labels for container management. Multiple conditional access policies can use a context, which opens a bunch of different possibilities about how to control access in different circumstances.

Given the amount of confidential information stored in SharePoint Online, it’s nice to be able control conditional access so easily and a good example of how Microsoft is steadily building up the container management capabilities for sensitivity labels. Make sure that you have the necessary licenses before you use the feature!

Stay updated with developments across the Microsoft 365 ecosystem by subscribing to the Office 365 for IT Pros eBook. We do the research to make sure that our readers understand the technology.

Build Organization-Wide Consistency in Sharing Behavior

Updated: February 24, 2022

Sensitivity label container management settings can control the sharing capability of SharePoint Online sites. Separately, the advanced settings of sensitivity labels can control the default sharing link settings for sites and documents. Enforcing consistent sharing capabilities is a good example of how container management through sensitivity labels make it easier to apply organizational standards across sites in a Microsoft 365 tenant.

Controlling Site Sharing

If you create a sensitivity label and configure it to apply a sharing capability of “Only people in your organization,” any site which receives the label automatically enforces that sharing capability. Site owners cannot change the sharing capability of a site without changing the label assigned to the site. Although tenant administrators can’t stop site owners changing a label, this is an auditable action which organizations can track to revert if necessary.

Controlling Default Sharing Link Settings

SharePoint Online creates sharing links when users share content from a site (Figure 1). The sharing link identifies what the person receiving the link can do with the content (read or edit). It also identifies who can use the link (anyone, specific people, tenant accounts).

SharePoint administrators can configure settings for the default sharing link for a site through PowerShell by running the Set-SPOSite cmdlet from the SharePoint Online management module. The relevant parameters are:

• DefaultSharingLinkType: Defines the default sharing link type for the site. For example, if this is “Internal,” the default sharing link type is set to anyone in the organization. The default is None, meaning respect the organization setting (defined with Set-SPOTenant).
• DefaultLinkPermission: Set to View or Edit to define what the link recipient can do. The default is None, meaning respect the organization setting.
• DefaultLinkToExistingAccess: The default is False. If set to True, the default sharing link type is set to People with existing access.

Defining a default sharing link type does not mean that site users are limited to the settings used to create sharing link. Users can update their sharing links to use other settings (for example, change the permission from edit to view), providing they remain within the constraints defined for the site’s external sharing capability.

Updating Sensitivity Labels with Default Sharing Link Settings

Now generally available (February 2022), you can configure the advanced settings of sensitivity labels to control the default sharing links generated for sites. The advantage of this method over configuring settings using Set-SPOSite is that any site assigned a label inherits the settings automatically. You don’t have to configure each site individually.

For now, configuration is by updating the advanced settings for a label with PowerShell. Given past practice, it’s possible that we will see a GUI for advanced label settings sometime in the future.

To update label settings, you connect to the compliance endpoint with PowerShell. Do this by running the Connect-IPPSession cmdlet from the Exchange Online management module. You can then use the Set-Label cmdlet to update the sensitivity labels. The setting names for Set-Label do not correspond exactly with the values used by Set-SPOSite. Here are the values:

• DefaultSharingScope (DefaultSharingLinkType) can be SpecificPeople, Organization, or Anyone.
• DefaultShareLinkPermission (DefaultLinkPermission) can be Edit or View.
• DefaultLinkToExistingAccess is True or False (default False).

You can update link settings separately or together. For example, these commands set the default sharing scope and permission in two steps:

Set-Label -Identity 'Guest Access' -AdvancedSettings @{DefaultSharingScope = "SpecificPeople"}
Set-Label -Identity 'Guest Access' -AdvancedSettings @{DefaultShareLinkPermission = "Edit"}

Or set the two values in one command:

Set-Label -Identity 'Non-Business Use' -AdvancedSettings @{DefaultShareLinkPermission = "Edit"; DefaultSharingScope = "Anyone"}

To check the settings for the label and confirm the configuration, run the Get-Label cmdlet:

Get-Label "Non-Business Use" | Select -ExpandProperty Settings
[contenttype, Site, UnifiedGroup]
[tooltip, Apply this label to a team, group, or site intended to support a non-business use such as a sports club or approved employee society.]
[displayname, Non-business use]
[defaultsharingscope, Anyone]
[defaultsharelinkpermission, Edit]

To set the default sharing link for the site so that it overrides any existing setting and uses people with existing access instead, run:

Set-Label -Identity 'Confidential Access' -AdvancedSettings @{DefaultLinkToExistingAccess  = "True"}

Like any other changes made to sensitivity labels, it can take up to 24 hours before SharePoint Online respects updates to the default sharing link settings.

Update Default Sharing Link Settings for Documents

Being able to control the default sharing link settings for sites by applying sensitivity labels is good. Being able to control default sharing link settings at a document level is even better. Microsoft added this capability between the preview and general availability. The same mechanism is used.

• Update a sensitivity label with default sharing link settings.
• Apply the sensitivity label to documents.
• If users share the labeled documents, SharePoint Online or OneDrive for Business use the settings from the label to generate the sharing link unless the site settings are more restrictive, in which case they take precedence.

The idea here is that you might have some specific documents in a site that you want people to pay attention to if they share the documents. The hope is that users will notice the differences in the sharing link generated by SharePoint Online or OneDrive for Business and recognize that they should be extra careful. The good thing is that people often accept default sharing link settings without question. The bad thing is that people mightn’t notice that a document is more confidential than the rest…

Keep up with the changing world of the Microsoft 365 ecosystem by subscribing to the Office 365 for IT Pros eBook. Monthly updates mean that our subscribers learn about new development as they happen.

PSTs Should Never Be in Cloud Storage

Updated: July 14, 2021

On May 17, Microsoft published message center notification MC256835 to advise tenants about the introduction for what they call a “PST version retention policy.” This has nothing to do with retention labels or retention policies. Instead, it’s about controlling PST storage in SharePoint Online by limiting the number of versions kept for PST files stored in SharePoint Online and OneDrive for Business document libraries.

Versioning in SharePoint Online

Versioning is a SharePoint feature. In a nutshell, as users make changes to files in document libraries, they create versions of the files. In some cases, such as when editing Office documents using Autosave, a single edit session might generate twenty or thirty versions, depending on the number of changes made. The number of versions kept in a document library is defined in library settings (Figure 1) in a range of 300 to 50,000.

SharePoint keeps multiple versions of files to ensure that the user can go back to a previous version. To do this, select a document and then Version history. You can then select a version to restore (Figure 2).

Both SharePoint Online and OneDrive for Business also support options to restore a library to a point in time over the previous 30 days. Without versions, it would not be possible to do this.

Why PSTs End up in SharePoint and OneDrive

Versioning is good, so what’s the problem with PSTs? Before addressing that question, we should ask about why PST storage in SharePoint Online or OneDrive for Business comes about. A PST (Personal Storage Table) is for email storage. It is a container to allow users to store messages they wish to keep. People might have moved PSTs from network file shares into SharePoint, but it’s a bad idea to use PSTs in SharePoint.

• The PST file format is not intended for concurrent shared access. These are personal files. If a problem happens with a PST file stored in SharePoint, it might lead to data loss.
• Even though they are in SharePoint, the messages stored in PSTs are inaccessible for eDiscovery.
• Over the years, Microsoft consistently advised against the use of shared PSTs on network file shares because of the potential for corruption.

You might think the problem of concurrent access to a shared file is addressed by using the OneDrive sync client to have a local copy of PSTs synchronized with the master copy in SharePoint. But as pointed out in this post by a Microsoft support engineer, the way Outlook locks PST files for exclusive access creates many problems for the sync client (Figure 3). Basically, the sync client is frustrated by the lock taken out by Outlook and can’t process the PST.

People who replace local workstation storage with OneDrive for Business for well-known folders like Documents might end up with PSTs in OneDrive. To avoid problems, they should move these files out of a synchronized location.

The Impact of PST Storage in SharePoint Online

The problem now being addressed by Microsoft is that holding multiple PST versions can consume a huge amount of SharePoint storage quota. Remember, a PST is a container rather than an individual file, and if it’s in active use, Microsoft says this generates “multiple versions which leads to storage being quickly consumed.”

Because of the generous quotas available to OneDrive for Business users, consuming storage is less of an issue for OneDrive for Business than it is for SharePoint Online. Microsoft makes 1 TB plus 10 GB per licensed user available for the organization and charges extra if more storage is needed. Using retention labels and retention policies to ensure files cannot be removed from SharePoint can already consume large amounts of storage, so adding PSTs to the mix is like pouring fuel on a raging fire.

Microsoft’s solution is to retain no more than 30 days’ worth of PST versions. This is enough to ensure that the Restore library feature works, even when PSTs are in a library. While the best answer is not to allow users to store PSTs in SharePoint Online or OneDrive for Business, restricting versions for PSTs is an acceptable method to restrain storage demand. Organizations can block users from synchronizing PSTs by including the file type in the blocked files list defined in the Settings section of the SharePoint Online admin center (Figure 4). Given the impact this could have on users, it’s a good idea to communicate about the block before its implementation.

Microsoft Implements the New Policy

Starting June 28, organizations can use the Set-SPOTenant cmdlet from the SharePoint Online PowerShell module to control the new policy: By default, the policy will be on, meaning the permanent deletion of PST versions once they reach 30 days old. If you don’t want to restrict PST versions, you can opt-out from the policy by running:

Set-SPOTenant -DisableOutlookPSTVersionTrimming $True

The new switch for the Set-SPOTenant cmdlet is available in the 16.0.21411.12000 release of the SharePoint Online management shell (released on July 12). You can download the module from the PowerShell Gallery.

The opt-out command must be run by August 13, so organizations have roughly six weeks to decide to opt-out. The policy becomes effective on August 16 and running the command to opt-out afterwards will have no effect. The big caveat is that the opt-out applies only to existing libraries. Any new library created after August 13 will apply the 30-day retention for PST versions.

The Badness of PSTs

I’ve been trying to persuade organizations to stop using PSTs for years. They’re a 25-year-old answer to the problem of small server mailboxes which existed then and doesn’t now. PSTs are insecure, compromise the ability of organizations to search for information and apply compliance policies, and prone to failure. There is nothing to recommend their continued use and even less to think that it’s a good idea to store PSTs in SharePoint Online and OneDrive for Business. If you’re still unconvinced, listen to this on-demand webinar Why PSTs are Such a Bad Idea in the Cloud, where I try my very best to explain why.

Get straightforward and honest opinion about how to manage Office 365 tenants by subscribing to the Office 365 for IT Pros eBook. We think about managing tenants so you can learn from our experience and insight.

Teamify: An Offer Hard to Refuse

If you’re the owner of a SharePoint Online team site which is connected to a Microsoft 365 group, you might have seen the prompts (subtle harassment) to “teamify” the site. The prompt (Figure 1) promises that you can “collaborate in real-time and share resources across Microsoft 365 with your team.” Sounds good, unless you created the site to use with email-based Microsoft 365 groups (aka Outlook groups) and plan to use email as the collaboration mechanism.

Email-Based Microsoft 365 Groups

I use Outlook-based groups quite often to work with companies who don’t use Teams or might not use Microsoft 365. In 2016, Microsoft added support for Azure B2B collaboration to Outlook groups to enable collaboration with external users. Tenant users can post to Outlook groups through Outlook (desktop, mobile, or OWA) but guests rely on email to receive copies of posted messages. In many places, Outlook groups are more than sufficient to work with third parties. But email will always get the message through (within reason) and access to the group’s SharePoint sites makes it easy to work on shared documents.

Driving Teams Adoption

But Teams is where the action is, and Microsoft certainly knows how to create many different touch points to drive user awareness and usage. Teams also uses Microsoft 365 Groups, and it’s more than possible to team-enable Outlook Groups. In technical terms, it’s a matter of creating a team and setting some values for the group, like the resource provisioning option. After creating the team, the team picks up the existing group membership and resources.

Creating Tabs in the General Channel for Site Resources

It’s long been possible to link a team to a group-connected site (the so-called teamify process). Recently, Microsoft added the ability to create channel tabs for site resources when setting up the new team. Figure 2 shows the dialog used to collect information about resources like Microsoft Lists and individual pages.

In Figure 2, two resources are selected to become channel tabs. Figure 3 shows the result with two tabs created for the home page for the site and a Microsoft list hosted in the site.

After creating the team, the process creates tabs in the General channel for the selected resources. You can’t create new channels when you teamify an existing SharePoint site, so the resources must go into tabs in the General channel. After creating the team and building out its channel structure, you can remove resources from the General channel into other channels. Unfortunately, you need to recreate the tabs as there is no way to move a tab from one channel to another.

Hiding the New Team

A small, but potentially important point, is that team-enabling a SharePoint site does not update the properties which hide the group from Exchange clients (Outlook, OWA, and Outlook mobile). If you look at the properties with PowerShell, you’ll see that the group remains visible to Exchange clients and the GAL.

Get-UnifiedGroup -identity "Project Haycock"| fl hiddenfrom*

HiddenFromExchangeClientsEnabled : False
HiddenFromAddressListsEnabled    : False

Is this important? In April 2018, Microsoft decided to hide team-enabled groups from Exchange clients with the logic being that once a group has a team, collaboration flows through Teams channels and chats rather than email. It doesn’t make sense to expose teams to email clients, hence the hiding. You can make up your own mind if you need to run a PowerShell script to find and update team-enabled groups which are still visible.

Need to know more about how Office 365 and its apps really work? Subscribe to the Office 365 for IT Pros eBook and learn on an ongoing basis.

Two Notifications Mark a Special Update

A feature so good that it requires two identical message center notifications must be worthwhile. Such is the case for the ability of sensitivity labels container management to control the external sharing capability of SharePoint Online team sites, as announced in MC244217 and MC244216 on March 12. Both point to Roadmap item 70735.

Information Protection and Container Management

Sensitivity labels can include settings for information protection and container management. Information protection usually means that the assignment of a label to an Office document, Azure Purview data (preview), Power BI objects, or other files will encrypt the target content using Microsoft Information Protection (rights management). Container management means that labels impose settings on a Microsoft 365 group, including the team or SharePoint team site belonging to the group. A single label can include both information protection and container management settings and is therefore applicable to both files and containers, or the scope of the label can be one or the other use. I favor a restricted label scope because I think it makes labels easier to manage.

Container Management Settings

When Microsoft first introduced the ability of sensitivity labels to control container settings, a limited number of controls were available. You can configure a label to:

• Control access to the container to Azure B2B Collaboration guest accounts. Previously, this control over containers could only be set by updating the properties of the group with PowerShell. The options are to allow or block guest access.
• Set the access to be public or private. If a label is not present, the group owner can decide whether the group is public (available to any tenant user) or private (restricted to the group membership).
• Limit access to documents in a SharePoint when using unmanaged devices.

The set of available controls is useful and sensitivity labels are much better than the alternative (like text-based classifications), but Microsoft’s intention always was to expand the number of controls to make sensitivity labels a much more powerful policy-driven management method for containers. Adding control over the sharing capability for SharePoint sites is further evidence of their intent.

Controlling External Access to SharePoint Online Sites

Organizations often store confidential or sensitive documents in SharePoint sites. SharePoint Online supports four values for site sharing capability to control the degree of external sharing permitted for documents in a site:

• Disabled – allow no external sharing outside the organization.
• ExistingExternalUserSharingOnly – allow sharing only with the guest users already in your organization’s directory.
• ExternalUserSharingOnly – allow users to share documents with new external users, who must accept the sharing invitations and go through an authentication process to create a guest account.
• ExternalUserAndGuestSharing – allow sharing with all external users, and by using anonymous access links (Anyone links).

SharePoint Online administrators and site owners can set the sharing capability through:

• The SharePoint Online admin center.
• PowerShell, using the Set-SPOSite cmdlet to update the SharingCapability setting.
• And now, by assigning a sensitivity label which has the external sharing control configured.

Remember that SharePoint Online won’t allow you to assign a less restrictive access to a site than allowed by the tenant sharing setting. In other words, if the tenant explicitly blocks anyone access for all sites, assigning anyone access through a label will have no effect.

Setting External Sharing Capability in a Sensitivity Label

When editing a sensitivity label, administrators can define what sharing capability is set when an owner or administrator assigns the label to a site (Figure 1).

The Site Owner View

Not every site owner knows about admin tools, and a major benefit of controlling sharing capability with sensitivity labels is that it makes it easier for site owners to assign the appropriate level of sharing based on their knowledge of the content within the site. At least, that’s the theory, and a lot depends on the clarity of the names chosen for sensitivity labels. Ideally, the names should convey how sensitive the information stored in the site is (Figure 2).

Applying a sensitivity label to a group or team also applies it to the site and selecting a new sensitivity label for a site also applies it to the associated group and team.

PowerShell Support for Container Management

The PowerShell cmdlets to interact with sensitivity labels are available after connecting a session to the compliance endpoint. The easiest way to do this is to run the Connect-IPPSSession cmdlet from the Exchange Online management module.

Once connected, we can use the Get-Label cmdlet to find details of sensitivity labels and the Set-Label cmdlet to update their settings. For example, not all sensitivity labels are configured for container management, so to find the set of labels scoped for container management, run this code:

Connect-IPPSSession
$Labels = Get-Label
ForEach ($Label in $Labels) {
   If ($Label.ContentType -match "Site, UnifiedGroup") {
   Write-Host "Label" $Label.DisplayName "has container actions" }
}

Label Non-business use has container actions
Label General Access has container actions
Label Guest Access has container actions
Label Limited Access has container actions
Label Confidential Access has container actions

As an example of how to use Set-Label, here are two examples of updating labels to set different sharing capabilities.

Set-Label -Identity Confidential -AdvancedSettings @{sharingcapability="ExistingExternalUserSharingOnly"}
Set-Label -Identity Secret -AdvancedSettings @{sharingcapability="Disabled"}

After applying a label with a sharing capability setting configured to a site, SharePoint updates its sharing capability. You can check that the settings have changed with the Get-SPOSite cmdlet:

Get-SPOSite -Identity "https://office365itpros.sharepoint.com/sites/BlogsAndProjects/" | Select SharingCapability, SensitivityLabel

SharingCapability SensitivityLabel
----------------- ----------------
         Disabled 27451a5b-5823-4853-bcd4-2204d03ab477

Checking that Everything Works

Of course, it’s a good idea to check that the sharing capability set in a sensitivity label works after assigning the label to a site. Let’s assume that you assign a label which disables external sharing. The easy test is to see if sharing works. As Figure 3 shows, it is not allowed and you see one of SharePoint’s famous OSE errors.

Being able to control external sharing for SharePoint sites is just the latest control for sensitivity labels. Microsoft plans more in the future. With this in mind, if you haven’t already started using sensitivity labels, perhaps now is a good time to make a start?

Unlocking Protected SharePoint Documents

In my article about how to decrypt SharePoint Online documents with PowerShell, I explained how to use the Unlock-SPOSensitivityLabelEncryptedFile cmdlet to decrypt protected SharePoint files by removing the sensitivity labels protecting the files. The example script uses cmdlets from the SharePoint PnP module to return a set of files from a folder in a document library for processing, and the unlock cmdlet then removes protection from any file with a sensitivity label.

The script works, but it’s not as flexible as I would like. For instance, because PnP can’t distinguish files with labels, every document in the folder is processed whether it is labelled or not. This does no harm, but it’s not something that you might want to do in the case of something like a tenant-to-tenant migration where thousands of protected documents might need to be processed.

Update May 10, 2021: The latest version of the SharePoint Online PowerShell module contains the Get-FileSensitivityLabelInfo cmdlet. This can be run to return the label status of a file, including if the label assigned to the file encrypts the file. The existence of this cmdlet removes some of the need to use the Graph to find and remove labels from protected files, but the Graph is still the fastest way to get the job done.

Using the Sites Microsoft Graph API

Which brings me to an updated version of the script (available from GitHub), which uses the Sites API from the Microsoft Graph to navigate through SharePoint Online and find labelled documents to process. Apart from being able to search for documents with sensitivity labels, a Graph API is usually the fastest way to deal with large numbers of objects.

Because we’re making Graph calls from PowerShell, we need to create a registered app in Azure AD to use as the entry point to the Graph (the same steps as outlined in this post are used). The app needs to be able to read site data, so I assigned it Sites.Read.All and Sites.ReadWrite.All permissions (Figure 1).

Finding Protected Documents

The script accepts two parameters: the name of the site to search (not the URL) and an optional folder. If multiple matching sites are found, the user is asked to choose which one to search (Figure 2).

Once a target site is confirmed, the script figures out if a folder is specified and if that folder exists in the chosen site. In Graph terms, we’re now dealing with drive objects. The default drive is the root folder of a document library and each folder is a different drive. To find folders, we need to find the child objects in the root, identify the right folder, find its drive identifier, and use that to find the files in the folder. All good, clean Graph fun.

The Drive API returns a maximum of 200 items at a time, so some Nextlink processing is needed to fetch the complete set of files in a folder. Each file is examined to figure out if it has a sensitivity label with protection, and if so, the display name of the label. After processing all the files, we tell the user what we’ve found and ask permission to go ahead and decrypt the files (Figure 3). If the user chooses not to proceed, the script writes details of the protected files out to a CSV file.

Decrypting Files

Files are decrypted by calling the Unlock-SPOSensitivityLabelEncryptedFile cmdlet. There’s no native Graph API call to decrypt SharePoint documents. In any case, we’re running a PowerShell script so it’s easy to call the cmdlet.

An Example to Build On

The script is an example of what’s possible with a combination of PowerShell and Graph API calls. I’m sure that the code and the functionality can be improved (feel free to suggest changes and improvements via GitHub). I’m just happy to demonstrate how things work and how including the Graph enables some extra flexibility.

Read the Office 365 for IT Pros eBook to find much more information about how sensitivity labels work – and many PowerShell examples too!

Making Sure the Right Permissions are Set

Message Center notification MC244885 of March 17 covers the topic of “Adding Quick Permissions to the OneDrive/SharePoint Share Control” (Microsoft 365 roadmap item 70806). It all sounds very impressive, but really, it’s just a tweak to the control used when sharing documents from SharePoint Online and OneDrive for Business.

Microsoft has made many changes to the sharing control over the last few years. The most important initiative has been to impose consistency across Microsoft 365, with Teams the final workload to pick up the new control. Lately, Microsoft has been tweaking the sharing control to improve how it works, with the latest attempt being an ill-advised proposal to remove the share by Outlook option. Thankfully, Microsoft dropped that idea.

Permissions Front and Center

The latest change is more positive. It improves the usability of the option to control if a recipient of a sharing link can edit the shared file by making permissions front and center (Figure 1) instead of a behind-the-scenes option. That’s the beginning and end of the change. Whether this kind of change justifies a separate roadmap item and message center notification could be debated, but as it does affect what end users see in front of them, it’s probably warranted.

As it was before, the default in the new control is that a recipient cannot edit a shared file.

Available Now

The new control is available now for both SharePoint Online and OneDrive for Business if your tenant is configured for targeted release. General availability will follow in early April. Teams has not yet picked up the new control, but that’s OK because in most cases, people share documents in personal chats when everyone in the chat gets the same permission.

Avoiding the Need to Remove and Recreate Guest Accounts

Microsoft 365 applications like Microsoft 365 Groups, Teams, SharePoint Online, and Planner use Entra ID B2B Collaboration to enable guest user access to their resources. The result is that many tenants have a proliferation of guest accounts to manage. I’ve written quite a few tools to help, including a report of guest accounts and their membership of Microsoft 365 Groups and a comprehensive report of tenant and guest members in Groups and Teams. Management can even be a challenge for guests who want to renounce their membership of a tenant.

In any case, the details of some guest accounts change over their lifetime. On March 2, Microsoft issued documentation for Reset redemption status for a guest user. This doesn’t sound very exciting, but it’s really very interesting because the feature allows tenant administrators to adjust how a guest account is signed into without using the previous technique of removing and recreating an account. The downside of that approach is that access is lost to all the resources available to the guest account like Teams, SharePoint sites, shares to individual documents, and so on. After recreating the account, access must then be regranted for each resource. This process is tedious, especially when the guest features in multiple groups.

Microsoft anticipates that the reset feature will be used in scenarios such as:

• The user wants to sign in using a different email and identity provider. In other words, they now have a different account. For instance, the user might have moved companies and wishes to continue working with your company (a common scenario for professionals like IT consultants and lawyers).
• The account for the user in their home tenant has been deleted and recreated. Entra ID won’t recognize the link between the guest account and the user’s new account.
• The user’s responsibilities have been passed along to another user and they want to assign access to the resources which supported those responsibilities to that user.

Part of the change is performed using the Entra ID admin center. The rest is done with PowerShell cmdlets from the AzureAD Preview module, which you can download from the PowerShell Gallery.

Change the Email (Sign-in) Address for a Guest Account

Unlike tenant accounts, guest users don’t use their user principal name to sign in. Instead, they use their email address. To work, the reset feature changes the sign-in name for the guest account and nothing else. The mail user object created in Exchange Online to allow guest users to receive email is also updated.

In this example, I have a guest account for Jacko Winters. The original email address for this account is Flayosc@outlook.com. The guest is a member of multiple teams and shares some SharePoint documents. I want to reassign access to all these resources to another account called Flayosc@yandex.com. It’s an example of the first scenario described above.

The first step is to update the Mail attribute (Email address) for the guest account with the email address you want to use. Do this through the Entra ID admin center (Figure 1). The new email address cannot belong to any other mail-enabled object in the tenant, such as another guest account. If it does, Entra ID won’t allow you to update the account.

Moving to PowerShell, connect to AzureAD and get the Entra ID account identifier for the guest account you want to replace.

Connect-AzureAD
$ObjectId = (Get-AzureADUser -SearchString “Jacko Winters”).ObjectId
$ObjectId
558d8cbb-a5a2-4ea1-b950-0d0748ca5634

Now create a new User object and populate it with the object identifier for the account.

$OldUser = New-Object Microsoft.Open.MSGraph.Model.User -ArgumentList $ObjectId
$OldUser

Id                                   OdataType
--                                   ---------
558d8cbb-a5a2-4ea1-b950-0d0748ca5634

Issuing a New Invitation

The next thing to do is check that the values returned from the two commands match. If they do, use the New-AzureADMSInvitation cmdlet to reissue an invitation to the new email address. The identifier for the guest user account is passed in the InvitedUser parameter. The myapps.microsoft.com landing page is a default site showing apps available to a user. Here’s the command I ran:

New-AzureADMSInvitation -InvitedUserEmailAddress Flayosc@yandex.com -SendInvitationMessage $True -InviteRedirectUrl "http://myapps.microsoft.com" -InvitedUser $OldUser -ResetRedemption $True

Update: Given the deprecation of the AzureAD module in March 2024 (and the disappearance of the ResetRedemption parameter from the New-AzureADMSInvitation cmdlet), you should switch to the Microsoft Graph PowerShell SDK. This code is the equivalent using the Get-MgInvitation cmdlet:

$User = Get-MgUser -Filter "startsWith(mail, 'Flayosc@yandex.com')"
New-MgInvitation `
    -InvitedUserEmailAddress 'Flayosc@yandex.com' `
    -InviteRedirectUrl "http://myapps.microsoft.com" `
    -ResetRedemption `
    -SendInvitationMessage `
    -InvitedUser $User

See this documentation for more information.

Entra ID creates a new invitation to access the resources currently available to the guest account and sends it to the new email address. You’ll see a response like this:

Id                      : 129c1c12-da99-4879-b258-d14b34601d46
InvitedUserDisplayName  :
InvitedUserEmailAddress : Flayosc@yandex.com
SendInvitationMessage   : True
InviteRedeemUrl         : https://login.microsoftonline.com/redeem?rd=https%3a%2f%2finvitations.microsoft.com%2fredeem%
2f%3ftenant%3db662313f-14fc-43a2-9a7a-d2e27f4f3478%26user%3d129c1c12-da99-4879-b258-d14b34601
d46%26ticket%3dLStZd8uAONAIbLNIZyfaUZ91VsRczLbzqbFOeHsonSE%253d%26ver%3d2.0
InviteRedirectUrl       : http://myapps.microsoft.com/
InvitedUser             : class User {Id: 558d8cbb-a5a2-4ea1-b950-0d0748ca5634
OdataType: }

InvitedUserMessageInfo  : class InvitedUserMessageInfo {
                            CcRecipients: System.Collections.Generic.List`1[Microsoft.Open.MSGraph.Model.Recipient]
                            CustomizedMessageBody:
                            MessageLanguage:
                          }

InvitedUserType         : Guest
Status                  : PendingAcceptance
ResetRedemption         : True

Accepting the Reissued Invitation

The invitation arrives at the email address (Figure 2) and the user can accept the invitation to confirm their credentials (set a password) and create an OAuth consent to allow the tenant to read details of the user’s account (Figure 3).

Once the user consents to the permissions, the user account is updated to set the UserState property to Accepted and write the date of the redemption in UserStateChangedOn. We now have a fully functional guest account again. The important point is that the object identifier and user principal name for the account do not change. The only thing which changes is the mail address associated with the account.

The Entra ID audit log contains details of the issue (Figure 4) and redemption of the invitation. While the activity tab confirms the target address for the invitation, the target tab confirms the guest account.

Accessing Resources

In this instance, the guest account has access to several teams and some SharePoint documents. SharePoint access is immediate, including the sites used by Teams. Guest access to Planner also works properly.

After testing that access worked for SharePoint and Planner, I turned to Teams. I expected access to the Teams app to take longer because of the need to complete the process which synchronizes Entra ID with the membership roster used to control access to individual teams. Until this happens, the user is refused access to Teams (Figure 5) and the old email address assigned to the guest account remains visible in Teams (Figure 6). [Note that the display name of the guest account has reverted to Flayosc instead of Jacko Winters]

Unsurprisingly, because the account information in Teams is now outdated, any attempt to add the guest account as a new member of a team also generates an error (Figure 7).

To try to force synchronization, I updated the display name and several other attributes of the account. This had no effect, so I added a couple of new users to the group using Teams to force Teams to refresh its membership roster. The updates flowed through to Entra ID, but nothing happened in Teams.

Get-AzureADGroupMember -ObjectId b647d5ff-3bda-4333-b768-7990084569b6

ObjectId                             DisplayName                   UserPrincipalName
--------                             -----------                   -----------------
cff4cd58-1bb8-4899-94de-795f656b4a18 Tony Redmond                  Tony.Redmond@office365itpros.com
b3eeaea5-409f-4b89-b039-1bb68276e97d Ben Owens (Business Director) Ben.Owens@office365itpros.com
a6bfb216-e88c-4f1f-86d7-04747e5fc686 Ben James                     Ben.James@Office365itpros.com
9ba20686-f869-46e8-85a2-00ec8a035e48 James Joyce                   James.Joyce@office365itpros.com
acb778e8-f587-45de-ae3a-e76007e043b2 Paul Howett                   Paul.Howett@office365itpros.com
98dda855-5dc3-4fdc-8458-cbc494a5a774 Sean Landy                    Sean.Landy@office365itpros.com
6b52fba5-349e-4624-88cd-d790883fe4c4 Ken Bowers                    Ken.Bowers@office365itpros.com
558d8cbb-a5a2-4ea1-b950-0d0748ca5634 Jacko Winters                 flayosc_outlook.com#EXT#@office365itpro

Get-AzureADuser -ObjectId 558d8cbb-a5a2-4ea1-b950-0d0748ca5634 | ft mail, displayname, objectid

Mail               DisplayName   ObjectId
----               -----------   --------
flayosc@yandex.com Jacko Winters 558d8cbb-a5a2-4ea1-b950-0d0748ca5634

The Original email address can’t be used to sign into Teams either. Eventually, after a couple of days, Teams synchronized with Entra ID and the updated account details became visible in Teams. However, the updated account could not sign into Teams.

Come Home to Teams

Working with the Entra ID development group, the problem was diagnosed to due to the way Teams tries its best to bring a user to their home tenant. In the case of guest users, Teams uses the sign in address to locate the tenant and headed off to the wrong place. When using an explicit redirect to the tenant identifier, like https://teams.microsoft.com/?tenantId=c662313f-14fc-43a2-9a7a-d2e27f4f3478, the user can connect.

Obviously, there’s some work for Teams to do to cope when administrators assign new email addresses to guest accounts, but at least the problem is known, and Microsoft will no doubt fix the issue soon.

All this work for a few lines in Chapter 13 of the Office 365 for IT Pros eBook. It just goes to prove how much work and effort the writing team puts in to keeping content accurate, refreshed, and updated. Subscribe now to receive monthly updates of goodness.

Azure AD Authentication Failure Stops Users Working

By now, you’ve probably heard about the second large Azure AD authentication outage since September. The March 15 incident calmed down after a few hours, but while it was ongoing users were unable to connect to Microsoft 365 applications when authentication was necessary. It wasn’t a happy experience. Microsoft plans to set a new SLA of 99.99% availability for Azure AD authentication on April 1, 2021. Perhaps they were making a few tweaks to the Azure AD infrastructure to prepare the ground for the upgraded SLA when things went wrong.

The current 99.9% SLA applies to the Azure AD tier for Office 365, but a Microsoft comment posted to the announcement for the new SLA said that the 99.99% level will only apply to those with Azure AD Premium licenses. I guess we shall have to wait and see the details of the SLA when Microsoft publishes the text of the agreement on April 1.

Microsoft 365 applications continued working during the outage unless authentication was necessary. Because they’re built on the Microsoft Graph APIs, the Teams clients authenticate hourly, so they were heavily affected. Outlook desktop stayed online throughout, and users reported varying degrees of useability for other apps.

Working in Word

While the outage progressed, I worked on a Word document for my blog post. All my Word documents are either in SharePoint Online document libraries or OneDrive for Business, so the OneDrive sync client is kept busy. The sync client is responsible for the differential synchronization of files up to the new 250 GB limit. Office apps autosave to capture changes. Not only does autosave ensure that you should never lose much if an app or workstation crashes, it’s also way changes get to other copies of Office documents open for co-authoring. And it’s why SharePoint Online keeps a minimum of 100 versions of documents. If you use the Office desktop apps heavily and store files online, the OneDrive sync client is busy.

OneDrive Sync Client Goes Nuts

Until that is, the OneDrive sync client decides that it should remove all the local copies of files from a SharePoint folder. This was a rather bizarre side effect of the Azure AD outage. At least, although I can’t prove that the outage caused the OneDrive sync client to do something very strange, the problem happened at the same time.

I noticed the issue when File Explorer reported nothing in the local folder which holds the synchronized copies of SharePoint files. The folder usually holds hundreds of files (423 as I write), so something had clearly happened. I opened the OneDrive sync client (build 21.041.0228.0001) and discovered that the client had removed the local files an hour ago (Figure 1), meaning that the client decided to remove the files at around 21:45 UTC, during the period when Microsoft was rolling out remediation for the Azure AD outage.

The problem was easily fixed by going to SharePoint Online and choosing to synchronize the folder again (Figure 2).

The OneDrive sync client started to download local copies immediately (Figure 3) and a full set of documents was soon on my local drive.

Curious and Problematic Synchronization

You can argue that all’s well that ends well, but no good reason exists for the OneDrive sync client to do what it did. Perhaps the Azure AD authentication problem caused the client to believe that it was no longer allowed to download files from the SharePoint site. If so, it would be better if the client issued a warning to say what’s about to happen and offered the user a chance to authenticate with their credentials rather than concluding that everything should be removed now.

Failure to authenticate is the logical root cause which lead to the mass deletion of local files. Every document in the folder has a retention label to stop SharePoint removing documents (set as a default label for the library). The normal course of events is that you can remove a local copy of a file from File Explorer only for the OneDrive sync client to restore the file once it discovers the deletion block imposed by the retention label. Despite the presence of the retention labels, the OneDrive sync client removed all the local files. If my theory holds, the OneDrive sync client concluded that the user had no access to SharePoint Online, so it should remove the local copies as this wouldn’t impact the retained file in SharePoint.

What’s also curious is that just one folder was affected. The OneDrive sync client left everything else alone. My conclusion is that the folder was in active use because I had a Word document stored in that folder open at the time, and autosaved changes were flowing back to SharePoint Online. No need existed for the OneDrive sync client to go near my other folders (like those holding files for the Office 365 for IT Pros eBook), so it left them alone.

It’s not just me who has encountered odd synchronization issues leading to mass removal of files. Fellow MVPs Vasil Michev and Paul Robichaux have also had difficulties. It seems like Microsoft has some work to do to smoothen how the OneDrive sync client handles what could be transient authentication issues.

Maybe I shouldn’t have disabled the new OneDrive sync client file delete warning!

Update March 18: Microsoft has two advisories linked to the problem. SP244708 (SharePoint) and OD244709 (OneDrive). The symptoms experienced by people are different, but the root cause is the same.

Sharing is Caring, Unless You Can’t

Software has a nasty habit of making people feel incompetent when they don’t understand why functionality doesn’t work as expected. Maybe it’s age, but I become increasingly frustrated when software doesn’t work as I want it to. Which brings me to the OSE204 error signaled by SharePoint Online when I tried to share a document (Figure 1).

The first question is why SharePoint insists on acting as if it’s mainframe software of the mid-1980s by issuing cryptic error codes. Fortunately, SharePoint includes some intelligible text to explain why it complained about an action, which makes it even stranger why OSE204 appears. It must be a SharePoint thing, just like Exchange people can explain the finer points of why DMARC is important.

Microsoft publishes a page to explain SharePoint and OneDrive sharing errors and helpfully asserts that the reader must be a SharePoint administrator to resolve the underlying problems. Reading through the text, we discover that users can encounter eleven different sharing errors from OSE202 to OSE404, which seems like a lot. Then again, sharing is a complicated business when you consider the permutations of sharing with people inside your tenant, guest users, external users, and so on.

Why OSE204 Happened

The page says: ”Error OSE204 indicates that sharing is turned off for the site that you’re trying to share from” and explains that an administrator can fix the problem by updating the sharing capability of the site. Unfortunately, it doesn’t cover what I did to provoke the problem, which was to change the sensitivity label assigned to the site. Sensitivity labels used for container management can control the sharing capability for a site. When I updated the assigned label, SharePoint applied the label settings and blocked sharing to external users. What happened is very logical and an excellent example of how powerful policy-based management through sensitivity labels is. But figuring out what had happened didn’t make me feel any less incompetent.

A SharePoint administrator isn’t needed to fix the problem. Because the sensitivity label assigned to the site controls the sharing capability, a site owner can restore sharing with external users by selecting a sensitivity label which permits this action.

Removing the Outlook Sharing Option

Message center notification MC237377 (updated February 19) says that the Outlook sharing option (send the sharing link via email) shown in Figure 1 is being withdrawn in March 2021. The logic is that the option adds no value. It works by creating a sharing link that’s inserted into an OWA compose window with a message addressed to the recipient entered in the Send link dialog. Microsoft says that much the same happens if you go through the normal process of creating and sharing a link. However, some differences do exist:

• The sender can change the message title. Instead of receiving a message with a subject like “Tony Redmond has shared SharePoint Sharing Errors with you,” the sender can change the subject to make it more impactful.
• The sender can enter more text to set context for the sharing link. SharePoint limits the text that a sharer can insert to 500 characters. When a link is inserted into an OWA message, the sharer can use the full editor available in OWA to insert whatever text, graphics, objects, and formatting they choose. Or even make sure that their carefully-crafted corporate autosignature is in the message (Figure 2).
• The sender can set message properties. OWA offers many possibilities for a sender to set properties of a message. For instance, they could use the Encrypt-only feature or add a sensitivity label to protect its contents.

The workaround is to copy the sharing link to the clipboard and paste the link into a message. The advantage of this is that no dependency exists on OWA. You can use the email client of your choice to format the message to sharing recipients in whatever way you choose.

Debatable Choice to Remove the Outlook Option

It’s understandable that Microsoft would want to rationalize functionality. In this case it’s debatable if removing a convenient option is justifiable. Perhaps Microsoft’s telemetry informs them that people make little use of the Outlook option. If that’s so, the removal is appropriate, but Microsoft isn’t saying.

I think it’s a pity that Microsoft plans to remove the Outlook option for sending sharing links. Although I normally use the default method to share links with people, the option to compose a message of more than 500 characters with a full editor is useful when sharing specific documents. I guess the folks who made the decision never feel the same need.

Update March 11: In a triumph for good sense, Microsoft said “Based on learning’s from our early rings and your feedback we have made the decision to not proceed as outlined below. The Send link to Outlook sharing option will remain until further notice. Thank you for your feedback.”

The Office 365 for IT Pros eBook includes extensive coverage of using sensitivity labels for container management. We don’t cover error OSE204 or the other ten errors you can see when sharing SharePoint or OneDrive content. Something must be left out to keep the book to a reasonable size.

Switching Storage from Stream to ODSP

Following on from the change in timing for the general switchover of Teams meeting recordings from Stream Classic to OneDrive for Business and SharePoint Online (ODSP for short) to July 2021, Microsoft is leveraging SharePoint permissions to have better control access to recordings. This wasn’t possible in Stream Classic, but it is now that Teams is adopting SharePoint-based sharing.

After you switch Teams meeting recordings to ODSP, new meeting recordings are not stored in Stream. Instead:

• Recordings for personal and group chats and personal (private) meetings are stored in the OneDrive for Business account of the user who starts the recording. This user is the owner of the recording.
• Recordings for channel meetings are stored in the channel folder of the document library in the SharePoint team site owned by the team.

In both cases, the MP4 files for the recordings are in the Recordings folder.

No Downloads Please

In message center MC230505 (updated February 18), Microsoft makes the important clarification that the only person allowed to download a recording for a personal chat or meeting is the owner. Everyone else is assigned view-only permission to the file.

A change due to roll out in early April and finish in June will block users with view-only permission from downloading the file. Only those with edit access to recordings can change the permissions to allow others to download the files. The change is described in Microsoft 365 roadmap 70543. Organizations cannot override the assignment of permissions to meeting recordings or the way the permissions work.

Channel Meetings are Different

Channel meetings are treated differently. Once someone uses the Teams calendar app or the channel calendar app to create a channel meeting, the team which owns the channel becomes the owner of the event. The person who schedules the meeting can still update meeting settings, but they are not the owner.

This is important because the Microsoft 365 Groups access model which underpins Teams dictates that team members have equal access to group resources. The simplicity of the Groups membership model makes it easy to understand, but sometimes its lack of granularity is regrettable and forces change, such as the introduction of private channels in Teams to support confidential access to resources for a subset of team members. Because team members enjoy the same level of access to group resources, they have edit permission for meeting recordings stored in the document library of the SharePoint site owned by the team.

Don’t Discuss Sensitive Information in Channel Meetings

The devil is always in the detail. In this case, Microsoft recommends that organizations do not use channel meetings to discuss confidential or sensitive information. The reason why is simple: if you do, any team member can access files shared in the meeting or download the meeting recording, which is probably not what you want to happen with sensitive material.

Instead, use private meetings when you need control over who can join the meeting and who will be able to access information shared in the meeting. Recent changes to meeting settings allow precise control over who can join a meeting automatically, meaning that you can be sure that someone can’t sneak in using a meeting link shared by another participant.

So much change, so much detail. Stay abreast of developments by subscribing to the Office 365 for IT Pros eBook and receive monthly updates (a completely new book). It’s the best value in IT!

Understand How People Use Sensitivity Labels to Protect Office Documents

If you enable support for sensitivity labels in SharePoint Online and OneDrive for Business (and you should), most of the previous frustrations that organizations have experienced in dealing with protected go away. Protected (encrypted) content can be indexed and found by eDiscovery, co-authoring is supported (with Office Online), and so on. And very importantly, Office 365 captures audit events when people apply, remove, or change sensitivity labels with Office documents.

Originally, only sensitivity label actions performed by the Office Online apps were captured. This is fine, but most user interactions with Office documents occur through the desktop apps. The gap in coverage is closing and the latest versions of the Microsoft 365 apps for enterprise (aka Office click to run) now create audit records when they apply or remove labels from documents. I’m using version 2012 – current channel preview (build 13350.20316) as the basis for this article, but I can see that audit records have been generated since mid-December.

Although the latter part of December is a period of low work activity, the number of events captured since compared against previous months confirms the view that desktop apps are used more heavily to generate documents, spreadsheets, and presentations. At least, in my tenant.

Separate Audit Events

Nice as it is to have the additional insight into the use of sensitivity labels, it’s regrettable that Microsoft did not use the same operation names when generating audit records for the desktop apps as they do for the online apps. The operation is the name of an auditable action.

It’s possible that the logic here is that the actions originate in two different sources and the different operations mean that administrators can conduct precise audit searches to find records for either the desktop or online apps – or both.

The new operations are:

• SensitivityLabelApplied: A sensitivity label is applied to an Office document. This operation is also used when capturing a record for the application of a label to a SharePoint site. The two can be distinguished by the record type, which will be either SensitivityLabeledFileAction (for Office) or SharePoint. Events are recorded when users apply sensitivity labels to Outlook messages, but not for messages protected by OME. OWA and Outlook mobile clients don’t currently generate audit events when users label messages.
• SensitivityLabeledFileOpened: An Office document with a sensitivity label is opened by a desktop app.
• SensitivityLabelRemoved: A sensitivity label is removed from an Office document.
• SensitivityLabeledFileRenamed: An Office document with a sensitivity label is renamed to become a new file. This event is also logged when a labelled file stored on a local device (not a copy synchronized by OneDrive) is edited.

As in many cases with Office 365 audit log records, the new events need to be parsed out before they’re useful. This is reasonably easy to do with PowerShell, albeit at the need to examine and interpret the payload content of each type of event.

Reporting Audit Events

Seeing is believing and it’s always easier to understand how things work when you have a practical example. I’ve written a script to grab all the events for sensitivity labels for the last three months and create a report. Each of the event types is unpacked and interpreted to make it clear what the event means. The output is a CSV file which can be analyzed in whatever way you wish. Or you can examine the output on-screen through the Out-GridView cmdlet (Figure 1).

The script is available in GitHub. You’ll need to connect to the Exchange Online management module and the security and compliance endpoint to run the cmdlets in the script. The compliance endpoint is used to fetch the list of sensitivity labels defined in the organization and create a hash table of GUIDs/identifiers (the keys) and label names (values). Some audit events contain label names but it’s more typical to only find a label identifier recorded, so lookups against the hash table translate identifiers into label names.

As you can see from the output, in my tenant most audit records are recorded when an Office desktop app opens a protected file:

Job complete. 370 Sensitivity Label audit records found for the last 90 days

Labels applied to SharePoint sites:  51
Labels applied to new documents:     45
Labels updated on documents:         5
Labeled files renamed:               29
Labeled files opened (desktop):      200
Labels removed from documents:       40
Mismatches detected:                 0
----------------------

Report file written to C:\temp\SensitivityLabelsAuditRecords.csv

In this case, no mismatches are noted between the label applied to a site (container management) and those assigned to documents stored in the site. My users might just be learning how to label documents properly!

We write tons of PowerShell scripts to check out how Office 365 really works and understand where any fault lines might be. Our GitHub repository is available to all. Even better, we explain how to use our scripts and other PowerShell commands to manage Office 365 in the Office 365 for IT Pros eBook.