New Purview eDiscovery Due “by end of 2024”

eDiscovery is probably not where most Microsoft 365 tenant administrators spend a lot of time. Running eDiscovery cases is quite a specialized task. Often, large enterprises have dedicated compliance teams to handle finding, refining, analyzing, and understanding the material unearthed during eDiscovery together with liaison with outside legal and other expertise.

Starting with Exchange 2010, Microsoft recognized that eDiscovery was a necessity. SharePoint Server had its own eDiscovery center, and these elements moved into Office 365. In concert with their own work, Microsoft bought Equivio, a specialized eDiscovery company, in January 2015 to acquire the technology that became the eDiscovery premium solution.

Over the last few years, Microsoft has steadily added to the feature set of the eDiscovery premium solution while leaving the eDiscovery standard and content search solutions relatively unchanged. The last makeover that content search received was in 2021, and it wasn’t very successful. I thought it was slow and unwieldy. Things have improved since, but content searches have never been a great example of snappy performance and functionality, even if some good changes arrived, like the KQL query editor in 2022. (Microsoft has now renamed the keyword-based query lanuage to be KeyQL to differentiate it from the Kusto Query Language used with products like Sentinel).

Time marches on, and Microsoft has decided to revamp eDiscovery. In an August 12, 2024,announcement, Microsoft laid out its plans for the next generation of eDiscovery. The software is available in preview, but only in the new Microsoft Purview portal.

The new portal handles both Purview compliance and data governance solutions. Microsoft plans to retire the current Purview compliance portal by the end of 2024 (Figure 1). Whether that date is achieved is quite another matter. As reported below, there’s work to be done to perfect the new portal before retirement is possible.

Big Changes in the New Purview eDiscovery

Apart from a refreshed UI, the big changes include:

Rationalization of eDiscovery into a single UI. Today, Purview includes content searches, eDiscovery standard, and eDiscovery premium, each with their own UI and quirks. In the new portal, a single eDiscovery solution covers everything, with licensing dictating the functionality revealed to users. If you have an E5 license, you get premium eDiscovery with all its bells and whistles. If you have E3, you’ll get standard eDiscovery.

Better data source management: Microsoft 365 data sources span many different types of information. In the past, eDiscovery managers picked individual mailboxes, sites, and OneDrive accounts to search. A new data source picker integrates all sources

Support for sensitivity labels and sensitive information types within queries: The query builder supports looking for documents and messages that contain sensitive information types (SITs, as used by DLP and other Purview solutions) or protected by sensitivity labels. Overall, the query builder is much better than before (Figure 2).

The output of queries is handled differently too. Statistics are presented after a query runs (Figure 3), and the ability to test a sample set to determine if the query finds the kind of items that you’re looking for still exists.

Exporting query results doesn’t require downloading an app. Everything is taken care of by a component called the Process manager that coordinates the retrieval of information from the various sources where the query found hits. Everything is included in a compressed file that includes individual SharePoint files, PSTs for messages found in Exchange mailboxes, and a folder called “LooseFile” that appears to include Copilot for Microsoft 365 chats and meeting recaps.

Not Everything Works in the New Purview eDiscovery

Like any preview, not everything is available in the software available online. For instance, I could not create a query based on sensitivity labels. More frustratingly, I could find no trace of content searches in the new interface, despite Microsoft’s assertion that “users still have access to all existing Content Searches and both Standard and Premium eDiscovery cases on the unified eDiscovery case list page in the Microsoft Purview portal.” Eventually and after originally posting this article, a case called Content Searches appeared at the bottom of the case list. Navigating to the bottom of a case list (which could be very long) isn’t a great way to find content searches and it seems unnecessarily complicated. Perhaps a dedicated button to open content searches would work better?

Many administrators have created content searches in the past to look for data. For instance, you might want to export selective data from an inactive mailbox. In the new eDiscovery, content searches are created as standard eDiscovery cases, a change that Microsoft says improves security control by allowing the addition or removal of users from the case. Given that I have 100+ content searches in one case, I think that the new arrangement overcomplicates matters (how can I impose granular security on any one of the content searches if they’re all lumped together into one case?). It’s an example of how the folks developing the eDiscovery solution have never considered how tenant administrators use content searches in practice.

Interestingly, Microsoft says that the purge action for compliance searches can now remove 100 items at a time from an Exchange mailbox. They mention Teams in the same sentence, but what this really means is that the purge can remove compliance records for Teams from the mailbox that later synchronize with Teams clients to remove the actual messages.

Much More to Discover

Leaving aside the obvious pun, there is lots more to investigate in the new eDiscovery. If you are an eDiscovery professional, you’ll be interested in understanding how investigations work and whether Copilot (Security and Microsoft 365) can help, especially with large review sets. If you’re a tenant administrator, you should make sure that you understand how content searches and exports work. Microsoft has an interactive guide to help, but more importantly, we will update the eDiscovery chapter in the Office 365 for IT Pros eBook once the new software is generally available.

Learn how to exploit eDiscovery and the data available to Microsoft 365 tenant administrators through the Office 365 for IT Pros eBook. We love figuring out how things work.

Loop Component eDiscovery is Possible Without Being Easy

Following yesterday’s article about using Loop components in Teams channel conversations, I was asked how I felt about how well Loop supports Microsoft 365 compliance solutions. The point is that Microsoft emphasizes the collaboration capabilities of Loop within an organization (but not outside because of the lack of external access) without delivering full support for basic compliance functionality like eDiscovery.

My view is simple. Loop components have been around for two years. In that time, there hasn’t been much change in how these components support compliance. In November 2021, when I wrote about Loop component support for Teams chat, I noted that the compliance records generated for chat messages contained pointers to the Loop files stored in OneDrive for Business. This is enough to find Loop components, but not in the context of the chat.

Loop Component eDiscovery with Content Search

In May 2022, I followed up by examining the topic of eDiscovery for Loop components in more detail and noted that it’s possible to run a content search for a keyword included in a Loop component but can’t open the file from preview. You can download the file and open it in OneDrive for Business, but only after giving the file a .fluid extension. The same is true for the components used in Teams channel conversations. I don’t remember searches ever finding retained copies of previous versions of chat components (stored in the site preservation hold library). This happens for components used in channel conversations (Figure 1).

In all cases, I could open the downloaded copy of a component. OneDrive for Business calls the web version of the Loop app to open the files (Figure 2).

In yesterday’s article, I used a compound message to illustrate Loop components in channel conversations. A compound message includes text and embedded elements, like a Loop component or a Stream video. You’ll notice that the results shown in Figure 1 only list the Loop components. To find the complete message, you must use keywords that are in the message and Loop components (the same or different keywords). You can then see the message posted to the channel (Figure 3).

Downloading Messages With Pointers to the Loop Component

There’s no trace of the Loop component because content search preview only displays text (including links and emojis). But when you download the compliance record and view the resulting message item, you can see the attachments. The loop component is represented as ‘card.html.’ The channel post was an announcement, so the other attachment is the graphic used in the announcement header.

For those wondering why the downloaded compliance record is displayed as an Outlook message, that’s because Teams compliance records are simplified copies of the actual Teams data stored as message items in Exchange Online mailboxes (a group mailbox in this case). Microsoft Search indexes the message items to make them available for eDiscovery. However, Loop components used in channel conversations are indexed separately in SharePoint Online and that’s why the search has two hits: one for the message, and one for the component.

Loop Component eDiscovery Premium

The situation is a little better in Purview eDiscovery Premium. Instead of a simple content search, eDiscovery Premium uses review sets retrieved from a collection of sources. The presentation of information from a review set is more insightful (Figure 5). In this instance, we can see that the content of the card.html attachment reveals that the Loop component is presented in an adaptive card called FluidEmbedCard.

Work to Do to Improve Loop Component eDiscovery

Microsoft is obviously enthused with Loop components. The technology is interesting and does a good job of making collaboration more accessible for users within a single tenant. However, it’s disappointing that eDiscovery of Loop components is still challenging two years after the first introduction of the technology in a Microsoft 365 application. You can certainly find the components, but investigators have too much work to do to knit everything together to create a seamless picture of how people use Loop components in Teams channel conversations.

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across Office 365. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.