Copilot Catch Up Fails to Impress

On August 28. 2024, Microsoft posted a revision to message center notification MC799636 (Microsoft 365 roadmap item 388746) to announce their decision to completely roll back their deployment of the new “catch up” feature for Copilot for Microsoft 365 and Teams.  The roll back operation will start on September 2. Previously, the feature was supposed to reach general availability in early August 2024.

According to Microsoft, the deployment had reached 50% of users. These are eligible accounts with Copilot for Microsoft 365 licenses. Given that many large enterprises have committed to Copilot for Microsoft 365, the number of affected users might be in the low several million range.

The Black Box of Card Determination

The blurb for the Copilot catch up feature says that it “helps users [to] take action on important updates.” In other words, Copilot had found something it deemed of interest to the signed-in user and brought the item to their attention in a series of cards that rotated through a carousel. In my case, the feature only turned up a couple of days ago. Figure 1 shows what I see. The three cards cover a Word document (chapter 2 for the Office 365 for IT Pros eBook), a calendar invitation, and a Loop workspace.

Like many elements of artificial intelligence, the process to determine what cards to display is a black box. Looking at the cards chosen by Copilot, I can see the logic of selecting the Word document because it’s a reminder that its content has changed, and I need to review the updates. The meeting is probably there because it happens later today. To help me prepare, Copilot found an email sent about a Teams service outage which frankly is of zero relevance to the meeting. The sole connection is that the word Teams appears in the subject for both the meeting invitation and email. The selection reminds me of “I must find something” instead of “I must find something useful.” I can’t account for why Copilot chose the Loop workspace because nothing has happened in it for months. Perhaps Copilot Catch Up wanted to be diverse in its choice of application sources.

I don’t think I shall miss the carousel. It seems like a modern take on the many ways that Office apps suggest documents to users or the way that the now-deprecated Delve highlights documents to users. The Viva Insights for email feature available in Outlook clients is another example of how Microsoft seeks to extract value from user data to highlight “things to do.”

Automatic Document Summaries by Copilot for Microsoft 365

The list key points link on the card for the Word document (Figure 2) appears to do what I expect to see when Microsoft deploys message center notification MC871010 (Microsoft 365 roadmap item 399921), scheduled for late August. This update promises that Copilot “will generate a summary in the window at the top of the Word document.”

MC866152 (23 August 2024) also covers the same ground for Microsoft Copilot (the version that doesn’t use the Graph).

I have some problems with the idea of generating automatic summaries because I fear it might slow down the opening of large documents based on the current performance in the Copilot chat app. It seems like a terrific demo feature that works well for 2-page documents. I can’t imagine how it will cope with the 1,300 pages of the Office 365 for IT Pros eBook source file. MC866452 says that the summary window can be collapsed to hide it. There’s no detail about what I really want to do, which is to disable automatic summaries for all Word documents.

Copilot Can’t do Everything

Figuring out what features will really be interesting and useful is an aspect of software engineering that is often very difficult. Sometimes I think Microsoft tries too hard, tries to be too clever, or utterly fails to understand how people outside Microsoft work. Applying artificial intelligence to as many aspects of Microsoft 365 as possible is madness, even if it keeps senior management happy.

The truth is that not every idea discussed around a conference table in Redmond or virtually in a Teams call is valuable. Sometimes it takes exposure in the harsh light of reality to figure out what works and what doesn’t. The demise of Copilot catch up is a reminder to us all that just because a new feature appears, its value needs to be assessed in terms of how it contributes to the success of the business.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across the Microsoft 365 ecosystem. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.