Lessons About AI to Learn from Bing Chat Enterprise

Bing Chat Enterprise and its Place in the Copilot Spectrum

Microsoft published message center notification MC649341 in late August to inform eligible customers (with Microsoft 365 E3 and E5; A3 and A5 (faculty only); and Business Standard and Business Premium licenses) that they had enabled Bing Chat Enterprise in preview for their tenants. On September 21, Bing Chat Enterprise then featured in the announcement of General Availability for Microsoft 365 Copilot, when Microsoft listed Bing Chat Enterprise as one of the commercial SKU (product) line-up for Microsoft Copilot (Figure 1).

Bing Chat Enterprise is available to the same Microsoft 365 product SKUs as Microsoft 365 Copilot is (Microsoft 365 E3 and E5, Microsoft 365 Business Standard and Premium). When formally available, other customers can buy a Bing Chat Enterprise license for $5/user/month.

I didn’t pay too much attention to Bing Chat Enterprise when Microsoft made their big announcement because the details about Microsoft 365 Copilot are more interesting. Since then we’ve learned that Microsoft will require eligible customers to buy a minimum of 300 Copilot licenses and that all transactions must be approved by Microsoft sales. In other words, a Microsoft partner can’t go ahead and order 300 licenses for one of their customers without approval. Although unpopular with partners, this restriction and the minimum purchase requirement are likely to be short-term measures to allow Microsoft to ramp-up support and other capabilities for Copilot but they might frustrate smaller organizations.

For instance, Microsoft 365 Business Premium is an eligible SKU for Copilot but it tops out at 300 users. The current rule means that a customer running Microsoft 365 Business Premium must buy Copilot for everyone in their organization (costing $108,000 annually). I guess many organizations will wait for the initial rush to work through Microsoft systems before considering a Copilot deployment.

Managing Bing Chat Enterprise

Which brings me back to Bing Chat Enterprise (BCE). While you’re waiting for the mists to clear around Microsoft 365 Copilot, BCE is a good tool to educate users about how to interact with generative AI. BCE is like the Microsoft 365 Chat app that comes with Copilot. The big difference is that Microsoft 365 Chat has access to user data stored in Microsoft 365 repositories like SharePoint Online, Exchange Online, and Teams. BCE must make do with whatever Bing Search can find. However, the same kind of interactive prompting to find and refine information happens.

Microsoft has deployed a Bing Chat Enterprise service plan to user accounts with eligible licenses. This action is described in message center notification MC665935 (updated September 11) and replaces the original tenant on/off switch previously deployed (MC677230) through an online page. Microsoft plans to remove the tenant on/off switch soon and base user access to BCE exclusively on the service plan from November 2023.

The advantage of using a service plan is that administrators can selectively enable or disable BCE for accounts by either editing accounts through the Microsoft 365 admin center or with PowerShell by removing service plan identifier 0d0c0d31-fae7-41f2-b909-eaf4d7f26dba from accounts using the Set-MgUserLicense cmdlet. For example, this command removes the BCE service plan from an account with a Microsoft 365 E5 license:

$DisabledServicePlan = @("0d0c0d31-fae7-41f2-b909-eaf4d7f26dba") 
Set-MgUserLicense -UserId Sean.Landy@Office365itpros.com -AddLicenses @{SkuId = "06ebc4ee-1bb5-47dd-8120-11324bc54e06"; DisabledPlans = $DisabledServicePlan} -RemoveLicenses @()

Learning from Bing Chat Enterprise

Learning how to prompt AI tools for answers is a key skill for users to acquire. Microsoft has a nice write-up on the subject where executives give examples of how they use Copilot and include the Copilot Lab to help users acquire knowledge about prompting. However, as we know from queries given to search engines, many never move past the simplest query, and if that happens with Copilot, there’s little chance that people will be satisfied with the results.

Interacting with BCE to find and refine answers to questions is good practice for Copilot. Sure, Copilot prompts will be different because they can reference documents and other items stored in Microsoft 365 and direct that the output should be in a specific form, like an email, but the principle behind conversational interrogation remains the same.

For example, I asked BCE to generate a PowerShell script to check that an account already had a specific license before attempting to assign the license. The first response used cmdlets from the now-deprecated and non-functioning Microsoft Online Services module. I asked BCE to try again, this time using cmdlets from the Microsoft Graph PowerShell SDK. Figure 2 shows the response.

The script code looks like it should work except that it won’t. The command to pipe a variable to the Update-MgUser cmdlet will fail horribly because the SDK does not currently support piping. It’s one of the SDK foibles that Microsoft is working on to fix.

AI can make things up (“hallucinations”), but in this instance BCE based its answer on Microsoft documentation and contributions to the well-respected and chock-full-of-knowledge StackOverflow site.

The learning for users is to never accept what AI produces without checking the generated answer first to be sure that it is correct and answers the original question, even if the cited sources seem impeccable. Maintaining a healthy level of scepticism about AI generated text is essential because it’s possible that someone would prompt Copilot for some information, see what looks like good information coming back, and email that information without thinking that it could be wrong, contain sensitive content, or is otherwise inappropriate to share.

Learning with AI

We’re at the start of what could be a transformational phase in how we deal with Office information. Good as the technology might be at the start, it’s going to take time for people to master driving AI to do the right things. Rubbish in equals rubbish out. AI just makes rubbish generation faster, if you allow it to happen.

---

So much change, all the time. It’s a challenge to stay abreast of all the updates Microsoft makes across Office 365. Subscribe to the Office 365 for IT Pros eBook to receive monthly insights into what happens, why it happens, and what new features and capabilities mean for your tenant.