Table of Contents
A Community-Driven Security Configuration Analyzer for Entra ID Tenants
The irrepressible Merill Fernando, a product manager in the Microsoft Entra ID organization, came together with Security MVPs Faben Bader and Thomas Naunheim to create the Maester tool. Announced on April 10, Maester is labeled as a “Microsoft Security test automation framework” and installation instructions are available here. It is a great example of a community-driven project.
Maester is built using Pester and Microsoft Graph APIs. Basically, it runs a bunch of tests against an Entra ID tenant (usually a Microsoft 365 tenant) and measures tenant security configuration settings against the MITRE ATT&CK framework using the Entra ID Security Configuration Analyzer. The output is a report telling the administrator what tests passed and what failed. In my case, the first run of Maester said that my tenant failed 42 tests (Figure 1).
On the surface, failing 42 tests seems like a dreadful outcome and it did generate some concern. However, like anything else that measures something against benchmarks, you need to understand what’s being measured, why a configuration is in a certain state, and if the current settings are valid or should be adjusted.
Conditional Access Policies and Break Glass Accounts
If you use conditional access policies to check inbound connections, at least one break glass account should exist to prevent the possibility of policy misconfiguration locking everyone out (this happens – all the time). I’ve written a PowerShell script to check conditional access policies to make sure that they include exclusions for break glass accounts, adding the accounts to policies when necessary.
Unhappily, my script (which runs regularly as an Azure Automation scheduled job) only processes enabled (active) conditional access policies and ignores those that are in the report-only state. The lack of break glass accounts on some policies in report only mode caused Maester to be unhappy (Figure 2).
To make Maester happy, I adjusted the script to update all conditional access policies.
Another fail reported by Maester said that no conditional access policy existed to require multi-factor authentication for guest accounts. Obviously, something odd happened behind the scenes because that exact policy is in place since January 2022.
Use Your Knowledge to Put Tool Recommendations into Context
The point is that you should never accept a recommendation made by software unconditionally. Always be suspicious until the recommendation is proven, just like you should be suspicious of any text created by generative AI. Context is invaluable and tenant administrators know far more about their business and operations than any tool can aspire to learn.
An example is the use of Entra ID License Utilization Insights where Maester reported the same figures calculated by the Entra admin center to say that I have 5 Entra P1 licenses but 42 active B2B users that need these licenses because they use conditional access policies (to mandate MFA, see above). But my tenant is configured to use the monthly active user billing model for premium features and I pay for this usage every month through an Azure subscription. Microsoft has some work to do to get its insights sorted out, and anything built on top of their data will be flawed until the data is corrected.
Good Links to the Graph Explorer and Graph APIs
We’re discussing the V0.1 release of a community project here and some bugs are expected. To be more positive, I like the way that Maester includes links to the Graph Explorer when it’s possible to use the Explorer to patch configurations with a Graph request. An example is where the access granted to directory information for guest account is unrestricted. The recommendation is to restrict access to prevent guest accounts being able to enumerate directory information, which means that guest accounts should have a restricted access role (GUID 2af84b1e-32c8-42b7-82bc-daa82404023b instead of the default (10dae51f-b6af-4016-8d66-8c2a99b929b3).
It’s easy to fix this problem in the Entra admin center, but who can resist the chance to run a Graph request instead of clicking a button? The link provided opens the Graph Explorer with the request to list the authorization policy (Figure 3). This is a GET transaction so it only fetches the data to check, but for extra marks you can add a request body and PATCH the policy. A future version of Maester might do that work for you if the developers don’t think it too dangerous.
Support the Maester Tool!
It would be easy to keep nitpicking but that’s not the right thing to do. Community projects need to be cherished and supported. Things will improve in time as people find glitches to fix and knowledge grows. The important thing is that Maester is a new tool for Microsoft 365 tenant administrators to use to improve their knowledge of Entra ID security features that can make their tenant more secure and harder to compromise. That’s always a good thing, which is why I like Maester.
Make sure that you’re not surprised about changes that appear inside Microsoft 365 applications by subscribing to the Office 365 for IT Pros eBook. Our monthly updates make sure that our subscribers stay informed.