Over the past year, we’ve built Microsoft Copilot and released more than 150 updates to it, and we’ve developed the Copilot stack, which takes everything we’ve learned so far and lets developers build their own copilots.

In addition, over the past two years, GitHub Copilot has become the most widely adopted AI developer tool, with 1.8 million paid subscribers.

And just a day ago, we introduced a new category of Copilot+ PCs, the fastest, most AI-ready PCs ever built.

We are the industry leader in AI, and that’s why, as we begin our flagship event for developers, Microsoft Build, there’s one question on everyone’s mind: What’s next?

On Monday, we introduced the world to a new class of Windows PCs, Copilot+ PCs. These devices are designed to allow developers to deliver differentiated AI experiences on the edge, and, together with the great features that we are announcing at Build, make Windows the most open platform for AI and the best place for developers.

At Build, we also are announcing further branches of this technology revolution, including:

• How Microsoft Fabric is helping developers and customers leverage data in motion, or digital information being transported within or between computer systems, to build intelligent apps.
• New frontier models that allow developers to explore multimodal capabilities supporting text, images, video and other types of data in their AI applications, evolving beyond text prompts and completions.
• Partnerships that show how AI can impact the future of myriad industries, including education.
• And how an open and flexible cloud-based platform, built on a foundation of safe and responsible AI, can help developers innovate.

Microsoft Build is always an exciting time for us. About 200,000 people have registered to join us for three days of technical learning and community connection, with 4,000 attending in person in Seattle. Attendees can choose from more than 300 sessions, demos and expert and instructor-led labs from Microsoft and our partners. For those not able to attend the live event, most content will be available on demand. In total, we are announcing about 60 new products and solutions at the event.

With that introduction to Build in mind, let’s explore some of the news and announcements.

In-the-moment decision making with Real-Time Intelligence

For the most efficient AI apps, businesses need to be able to qualify, analyze and organize data at ingestion. This has proven to be a difficult step. The new Real-Time Intelligence within Microsoft Fabric provides an end-to-end Software as a Service (SaaS) solution that empowers customers to act on high-volume, time-sensitive and highly granular data to make faster and more informed business decisions.

Real-Time Intelligence, now in preview, can be helpful for analysts with simple low/no-code experiences, and it also can benefit pro developers with code-rich user interfaces. For example, racing team Dener Motorsport has been using Microsoft Fabric to support real-time analytics, storage and reporting, enabling them to sustain optimal performance and maintain the cars in good repair, which can keep drivers safer. Dener plans to use Real-Time Intelligence as part of their winning strategy to gain insights at ingestion during races.

Building apps requires a level of flexibility, customization and efficiency to make it feasible for developers. The new Microsoft Fabric Workload Development Kit makes this possible by enabling independent software vendors (ISVs) and developers to extend applications within Fabric, creating a unified user experience.

GitHub Copilot experience leans into extensibility

GitHub is introducing the first set of GitHub Copilot extensions, developed by Microsoft and third-party partners, in private preview. These additions allow developers and organizations to customize their GitHub Copilot experience with their preferred services like Azure, Docker, Sentry and more directly within GitHub Copilot Chat.

GitHub Copilot for Azure, one of the extensions from Microsoft, showcases how building in natural language with a broader range of capabilities can propel development velocity. Using the extension through Copilot Chat, developers can explore and manage Azure resources, while also troubleshooting issues and locating relevant logs and code.

New frontier models and multimodal capabilities in Azure AI

GPT-4o, OpenAI’s newest flagship model, is now available in Azure AI Studio and as an API. This groundbreaking multimodal model integrates text, image and audio processing to set a new standard for generative and conversational AI experiences.

We also announced that Phi-3-vision, a new multimodal model in the Phi-3 family of AI small language models (SLMs) developed by Microsoft, is now available in Azure. Phi-3 models are powerful, cost-effective and optimized for personal devices. Phi-3-vision offers the ability to input images and text and receive text responses. For example, users can ask questions about a chart or ask an open-ended question about specific images.

Developers can experiment with these state-of-the-art frontier models in the Azure AI Playground, and they can start building with and customizing with the models in Azure AI Studio.

Microsoft, Khan Academy using AI to empower educators

Microsoft and Khan Academy are announcing a multi-faceted partnership to turn the transformative potential of AI into reality.

To start, Microsoft is enabling Khan Academy to offer all K-12 U.S. educators free access to Khanmigo for Teachers, an AI-powered teaching assistant that frees up teachers’ time so they can focus on what matters most – engaging with and supporting their students. Microsoft is donating access to Azure AI-optimized infrastructure to increase the availability of Khanmigo for Teachers, which will now be powered by Azure OpenAI Service.

Khan Academy is collaborating with Microsoft to explore opportunities to improve math tutoring in an affordable, scalable and adaptable manner with a new version of Phi-3, a family of SLMs developed by Microsoft. They also plan to bring more Khan Academy content into Copilot and Teams for Education, expanding resources for learners.

New partnership with Cognition AI

Microsoft and Cognition will bring Cognition’s autonomous AI software agent, Devin, to customers to help them with complex tasks such as code migration and modernization projects. As part of the agreement, Devin will be powered by Azure. Cognition AI is an applied AI lab building end-to-end software agents to help developers achieve more.

Powerful new virtual machines help fuel AI adoption

Microsoft has a unique systems approach to AI infrastructure, which includes hardware and software from Microsoft and our partners, all optimized to run AI workloads at scale and fine-tuned for customer needs. We are the first cloud provider to bring AMD’s leading MI300X AI accelerator chip to power customers’ AI training and inferencing needs, with the general availability of the Azure ND MI300X v5 virtual machine series optimized for demanding AI and high-performance computing (HPC) workloads like Azure OpenAI Service.

Following the launch of Azure Cobalt 100, Microsoft’s first custom-designed compute processor, the company is announcing a preview of new Cobalt 100 Arm-based virtual machines (VMs), based on the company’s custom silicon series announced in November 2023. Cobalt 100 Arm-based VMs are the first generation of VMs to feature Microsoft’s new Cobalt processor, custom-built on an Arm architecture, and optimized for efficiency and performance when running general-purpose and cloud-native workloads. Customers can expect up to 40% improved performance compared to comparable Azure VMs.

The evolution of Copilot

Copilot has been a game-changer for many people since it was first released. Using modern AI and large language models (LLMs) like Open AI’s GPT-4, copilots across Microsoft products have assisted people with complex tasks, serving as a personal, behind-the-scenes AI assistant.

Now, we are introducing Team Copilot, the expansion of Copilot for Microsoft 365 from a behind-the-scenes, personal AI assistant to a new, valuable member of your team. You will be able to invoke Copilot where you collaborate – in Teams, Loop, Planner and more. Team Copilot can be a meeting facilitator in meetings, managing the agenda, tracking time and taking notes. It can act as a collaborator in chats by surfacing important information, tracking action items and addressing unresolved issues. It can serve as a project manager to help ensure every project runs smoothly and notify the team when their input is needed. These initial experiences, coming in preview later this year, will enable us to learn, iterate and refine as we enter a new phase of innovation where Copilot begins to take more action on behalf of individuals and teams.

Microsoft Copilot Studio is introducing new agent capabilities, empowering developers to build copilots that can proactively respond to data and events, tailored to specific tasks and functions. Copilots built with this new category of capabilities can now independently manage complex, long-running business processes by leveraging memory and knowledge for context, reason over actions and inputs, learn based on user feedback and ask for help when they encounter situations that they don’t know how to handle. Users can now put Copilot to work for them – from IT device procurement to customer concierge for sales and service.

Copilot extensions, including plugins and connectors, allow customers to enhance Microsoft Copilot by connecting it to new data sources and applications, expanding its functionality.

We’re excited about these announcements and all the other updates and features being unveiled at Build. For more information, today you can watch keynotes from Microsoft Chairman and CEO Satya Nadella, Executive Vice President of Experiences and Devices Rajesh Jha and Chief Technology Officer Kevin Scott.

At Microsoft, we have commitments to ensure Trustworthy AI and are building industry-leading supporting technology. Our commitments and capabilities go hand in hand to make sure our customers and developers are protected at every layer.

Building on our commitments, today we are announcing new product capabilities to strengthen the security, safety and privacy of AI systems.

Security. Security is our top priority at Microsoft, and our expanded Secure Future Initiative (SFI) underscores the company-wide commitments and the responsibility we feel to make our customers more secure. This week we announced our first SFI Progress Report, highlighting updates spanning culture, governance, technology and operations. This delivers on our pledge to prioritize security above all else and is guided by three principles: secure by design, secure by default and secure operations. In addition to our first party offerings, Microsoft Defender and Purview, our AI services come with foundational security controls, such as built-in functions to help prevent prompt injections and copyright violations. Building on those, today we’re announcing two new capabilities:

• Evaluations in Azure AI Studio to support proactive risk assessments.
• Microsoft 365 Copilot will provide transparency into web queries to help admins and users better understand how web search enhances the Copilot response. Coming soon.

Our security capabilities are already being used by customers. Cummins, a 105-year-old company known for its engine manufacturing and development of clean energy technologies, turned to Microsoft Purview to strengthen their data security and governance by automating the classification, tagging and labeling of data. EPAM Systems, a software engineering and business consulting company, deployed Microsoft 365 Copilot for 300 users because of the data protection they get from Microsoft. J.T. Sodano, Senior Director of IT, shared that “we were a lot more confident with Copilot for Microsoft 365, compared to other large language models (LLMs), because we know that the same information and data protection policies that we’ve configured in Microsoft Purview apply to Copilot.”

Safety. Inclusive of both security and privacy, Microsoft’s broader Responsible AI principles, established in 2018, continue to guide how we build and deploy AI safely across the company. In practice this means properly building, testing and monitoring systems to avoid undesirable behaviors, such as harmful content, bias, misuse and other unintended risks. Over the years, we have made significant investments in building out the necessary governance structure, policies, tools and processes to uphold these principles and build and deploy AI safely. At Microsoft, we are committed to sharing our learnings on this journey of upholding our Responsible AI principles with our customers. We use our own best practices and learnings to provide people and organizations with capabilities and tools to build AI applications that share the same high standards we strive for.

Today, we are sharing new capabilities to help customers pursue the benefits of AI while mitigating the risks:

• A Correction capability in Microsoft Azure AI Content Safety’s Groundedness detection feature that helps fix hallucination issues in real time before users see them.
• Embedded Content Safety, which allows customers to embed Azure AI Content Safety on devices. This is important for on-device scenarios where cloud connectivity might be intermittent or unavailable.
• New evaluations in Azure AI Studio to help customers assess the quality and relevancy of outputs and how often their AI application outputs protected material.
• Protected Material Detection for Code is now in preview in Azure AI Content Safety to help detect pre-existing content and code. This feature helps developers explore public source code in GitHub repositories, fostering collaboration and transparency, while enabling more informed coding decisions.

It’s amazing to see how customers across industries are already using Microsoft solutions to build more secure and trustworthy AI applications. For example, Unity, a platform for 3D games, used Microsoft Azure OpenAI Service to build Muse Chat, an AI assistant that makes game development easier. Muse Chat uses content-filtering models in Azure AI Content Safety to ensure responsible use of the software. Additionally, ASOS, a UK-based fashion retailer with nearly 900 brand partners, used the same built-in content filters in Azure AI Content Safety to support top-quality interactions through an AI app that helps customers find new looks.

We’re seeing the impact in the education space too. New York City Public Schools partnered with Microsoft to develop a chat system that is safe and appropriate for the education context, which they are now piloting in schools. The South Australia Department for Education similarly brought generative AI into the classroom with EdChat, relying on the same infrastructure to ensure safe use for students and teachers.

Privacy. Data is at the foundation of AI, and Microsoft’s priority is to help ensure customer data is protected and compliant through our long-standing privacy principles, which include user control, transparency and legal and regulatory protections. To build on this, today we’re announcing:

• Confidential inferencing in preview in our Azure OpenAI Service Whisper model, so customers can develop generative AI applications that support verifiable end-to-end privacy. Confidential inferencing ensures that sensitive customer data remains secure and private during the inferencing process, which is when a trained AI model makes predictions or decisions based on new data. This is especially important for highly regulated industries, such as healthcare, financial services, retail, manufacturing and energy.
• The general availability of Azure Confidential VMs with NVIDIA H100 Tensor Core GPUs, which allow customers to secure data directly on the GPU. This builds on our confidential computing solutions, which ensure customer data stays encrypted and protected in a secure environment so that no one gains access to the information or system without permission.
• Azure OpenAI Data Zones for the EU and U.S. are coming soon and build on the existing data residency provided by Azure OpenAI Service by making it easier to manage the data processing and storage of generative AI applications. This new functionality offers customers the flexibility of scaling generative AI applications across all Azure regions within a geography, while giving them the control of data processing and storage within the EU or U.S.

We’ve seen increasing customer interest in confidential computing and excitement for confidential GPUs, including from application security provider F5, which is using Azure Confidential VMs with NVIDIA H100 Tensor Core GPUs to build advanced AI-powered security solutions, while ensuring confidentiality of the data its models are analyzing. And multinational banking corporation Royal Bank of Canada (RBC) has integrated Azure confidential computing into their own platform to analyze encrypted data while preserving customer privacy. With the general availability of Azure Confidential VMs with NVIDIA H100 Tensor Core GPUs, RBC can now use these advanced AI tools to work more efficiently and develop more powerful AI models.

Achieve more with Trustworthy AI 

We all need and expect AI we can trust. We’ve seen what’s possible when people are empowered to use AI in a trusted way, from enriching employee experiences and reshaping business processes to reinventing customer engagement and reimagining our everyday lives. With new capabilities that improve security, safety and privacy, we continue to enable customers to use and build trustworthy AI solutions that help every person and organization on the planet achieve more. Ultimately, Trustworthy AI encompasses all that we do at Microsoft and it’s essential to our mission as we work to expand opportunity, earn trust, protect fundamental rights and advance sustainability across everything we do.