Featured

Azure OpenAI Service helps customers accelerate innovation with large AI models; Microsoft expands availability

Vanessa HoMay 24, 2022

Customers shopping for a used car can sometimes feel overwhelmed digging through countless specs and reviews, but CarMax, the largest used car retailer in the U.S., is making it easier for customers to find the most useful information.

Thanks to powerful AI language models, potential buyers can now see summaries of customer reviews for every make, model and year of vehicle that CarMax sells, about 5,000 combinations in a vast inventory of approximately 45,000 cars. The summaries provide easy-to-read takeaways from real customer reviews: whether it’s a great family car, how comfortable the ride is or if there’s enough space to pack for weekend adventures.

CarMax has also used the models to create new website content that allows customers to easily see what’s new for each version of a car, helping them decide whether new features are worth splurging on.

CarMax generated the massive amount of original content in just a few months — a rate previously impossible — with powerful GPT-3 natural language models built by the company OpenAI. To expand the work, CarMax is now using Azure OpenAI Service, which combines access to OpenAI’s models with Azure’s enterprise-grade capabilities such as security, compliance and regional availability.

“With Azure OpenAI Service, we are able to create content that empowers our customers so they can be informed before they make a decision,” said Shamim Mohammad, CarMax executive vice president and chief information and technology officer. “Being able to innovate at scale, being able to make our customers’ experience easier, being able to experiment — all those things are possible because of this service and the partnership we have with Microsoft.”

At its Microsoft Build conference today, Microsoft announced that Azure OpenAI Service is now available in a limited access preview. Customers who want to use the service can apply for access. Introduced at Microsoft Ignite 2021 as a new product within the Azure Cognitive Services family, a part of Azure AI, Azure OpenAI Service was previously available by invitation only.

With Azure OpenAI Service, we are able to create content that empowers our customers so they can be informed before they make a decision.

The service has a new responsible AI system that filters out harmful content and helps detect abuse. Additionally, Azure OpenAI Service now offers access to more models, including GPT-3, Codex and embeddings models. Codex can generate code and translate plain language to code, while embeddings make semantic search and other tasks easier. The service also offers new capabilities for customers to fine tune models for more tailored results.

Giving customers the guarantees and promises of Azure

Azure OpenAI Service is enabling customers across industries from health care to financial services to manufacturing to quickly perform an array of tasks. Innovations include generating unique content for customers, summarizing and classifying customer feedback, and extracting text from medical records to streamline billing. The most common uses have been writing assistance, translating natural language to code and gaining data insights through search, entity extraction, sentiment and classification.

Chart outlining how Azure OpenAI Service has helped Farmlands summarize customer interactions. — Azure OpenAI Service has helped Farmlands, a rural supplies cooperative in New Zealand, summarize customer interactions, classify them as “neutral,” “negative” or “positive” and extract keywords to gain insights into customer trends more quickly. Image courtesy of Farmlands.

“One of the most interesting things is the variety of use cases that can be supported off of a single model,” said Eric Boyd, corporate vice president for Azure AI at Microsoft. “Azure OpenAI Service is really leading the way with these new large language models and giving customers the guarantees and promises in Azure that this is going to be reliable and secure, and their privacy will be protected, as they explore this incredible frontier of what’s possible with these new technologies.”

Through OpenAI’s API and Azure OpenAI Service, CarMax used GPT-3 to abstractly summarize and fine tune 100,000 customer reviews into 5,000 well-written summaries. The job would have taken CarMax’s editorial team 11 years to complete, said Kevin Hopwood, a principal software engineer at the company.

The summaries and other model-generated content have improved customer engagement and search engine optimization, while the time saved has enabled CarMax’s content creators to focus on deeper research, long-form articles and more creative tasks, he said.

“The best thing we can do is free up their time so they can explore new content ideas and new ways to engage customers,” said Hopwood.

Used car retailer CarMax has used Azure OpenAI Service to help summarize 100,000 customer reviews into short descriptions that surface key takeaways for each make, model and year of vehicle in its inventory.

Azure’s security, compliance, reliability and other enterprise-grade capabilities will enable CarMax to scale its use of GPT-3 in cases requiring the extraction of millions of keywords. The model’s ability to learn with just a few examples of intended outputs, a process called few-shot learning, will help CarMax’s 60 product teams use the models without needing any additional teams of data scientists.

“Being an Azure service, this tool puts a lot of power into our traditional Microsoft C# engineers that they didn’t have before,” said Sean Goetz, director, application systems at CarMax. “We can expand it to other teams just like any other Microsoft tool set.”

Building systems to support responsible AI

The power of GPT-3, which is pre-trained on a vast amount of internet text, comes with a risk of generating harmful or unintended results. Microsoft has made significant investments to help guard against abuse and unintended harm, which includes requiring applicants to show well-defined use cases and incorporate Microsoft’s principles for responsible AI use. One important way CarMax and other customers meet the criteria is by having humans in the loop to make sure model outputs are accurate and up to content standards before they’re published.

The new responsible AI system integrated in Azure OpenAI Service can help filter out content that is sexual, violent, hateful or related to self-harm. The team plans to add additional filters and customization features as they work with customers during the preview period and learn what’s needed in practice.

We’ve built the system not to say we’re filtering everything and we’ve got the right answer, but to be able to get that feedback and rapidly adapt.

The filtering system also identifies possible patterns of abuse or unintended harm, in which case Microsoft would work with customers to investigate, respond and block an abusive user if needed. An incident response team is available to quickly update content filters as language evolves. For example, if a new racial slur emerges, the team will rapidly respond and block the term as hate speech.   

“We’ve built the system not to say we’re filtering everything and we’ve got the right answer, but to be able to get that feedback and rapidly adapt,” said Sarah Bird, Microsoft’s responsible AI lead for Azure AI. “We’re also not making a judgment about whether or not it’s OK for users to post certain content online. We’re saying we don’t think our AI system should generate this type of content.”

To help customers build robust systems, Microsoft is also providing UX design guidelines and patterns, and a transparency note describing the limits, intended uses and characteristics of the service.

“We think very carefully about the tools that we create and how they’re going to be used, and we’re really trying to make sure they’re going to be used responsibly so that the benefits of this technology can accrue to everybody,” Microsoft’s Boyd said.

Responsible AI principles have been a helpful guide for Farmlands, New Zealand’s largest rural supplies cooperative, where human reviewers are also part of its work with Azure OpenAI Service to improve customer service and generate new content for its website.

Customer service agent for Farmlands looking at two screens and working on a laptop. — Farmlands, New Zealand’s largest rural supplies cooperative, has used Azure OpenAI Service to summarize customer interactions so associates don’t have to put customers on hold and spend valuable minutes scrolling through notes to get up to speed. Photo courtesy of Farmlands.

Farmlands used OpenAI’s API and Azure OpenAI Service to condense 350,000 customer interactions into brief write-ups that help call center associates quickly understand a customer’s situation and respond effectively. In the past, associates had to scroll through emails and case notes while keeping a customer on hold. The summaries can pick up subtleties like sarcasm and complexities like a case involving 11 interactions with a customer who received the wrong fuel.

“It saves our call center staff time and improves our customers’ experience,” said Gareth Pullar, an insights and analytics manager at Farmlands.

The retailer is also using the service to extract keywords from interactions and to classify interactions as “positive,” “neutral” or “negative.” It’s exploring an internal technical chatbot and the generation of product descriptions for its e-commerce site. Using Azure OpenAI Service means engineers, who use a full Azure stack, will be able to gain data insights into customer trends more easily, while scaling their use of model-generated content throughout the company, Pullar said.

“In terms of generating content, it’s a game-changer,” said Jean van Schalkwyk, Farmlands business intelligence lead. “It’s just really exciting technology.”

Related links:

Top photo: Azure OpenAI Service helped CarMax turn 100,000 customer reviews into readable summaries that surface key takeaways and help people shopping for a used car make informed purchasing decisions. Photo courtesy of CarMax.