The PC Reimagined By Microsoft, OpenAI, And Qualcomm

There have already been some huge announcements made over the past two weeks around AI and PCs, and the world’s largest PC conference, Computex in Taiwan, is still a week away. The first major announcement was OpenAI’s introduction of GPT-4o, arguably the first real AI agent that is cable of multimodal interaction. The second is the new AI PCs or what Microsoft is calling the Copilot+PC based on Qualcomm’s Snapdragon X Elite and X Plus processors. Individually, the two announcements are significant, but when combined they usher in a new era of intelligent computing; an era that is right out of Star Trek or 2001: A Space Odyssey.

The Platforms

While what we can do with technology is by far the most critical point, all technology requires underlying hardware to enable it and this new era of computing is no different. Microsoft outlined the requirements for AI PCs in late 2023, which included a minimum of 40 TOPS of AI performance, 16GB of memory, 256GB of SSD storage, and a dedicated Copilot key on the keyboard. It should be noted that TOPS isn’t necessarily the best system-wide measure of AI performance as it doesn’t take into account elements, such as model optimization, software, memory and I/O, that are critical to user’s AI experience. However, TOPS is the measure Microsoft is using to delineate minimum performance requirements.

When comparing this to the average gaming, creator, and STEM (Science, Technology, Engineering, and Mathematics) PC, these specifications seem rather modest. The only things that stand out are the new keyboard key and the 40 TOPS of AI performance, which is lower performance than current discrete GPUs but new to PC processors/SoCs through the relatively recent integration of dedicated neural processing units (NPUs). I say relatively because over the past few generations all PC processors have added integrated NPUs, but only the Qualcomm Snapdragon X Elite current exceeds the 40 TOPS Microsoft requirement with just the NPU.

With 45 TOPS of AI NPU performance and 75 TOPS of total SoC performance, the Qualcomm Snapdragon X Elite currently leads in AI performance of all the announced PC SoCs. The Snapdragon X Elite also leverages Qualcomm’s expertise in building low-power SoCs for all mobile devices, especially smartphones. According to Microsoft the new Surface Pro devices will provide up to 15 hours of web browsing and 22 hours of video playback.

While Microsoft highlighted the Surface products, similar platforms using the Snapdragon X Elite SoCs are launching from Acer, ASUS, Dell, HP, Lenovo, and Samsung starting at around $999. ASUS held an online event just after Microsoft to announce the new Vivobook S15 using the Qualcomm Snapdragon X Elite and other platforms using Snapdragon X Plus SoCs for premium AI experiences at lower price tiers.

The Agent

The new PC platforms are impressive, but the new capabilities with Microsoft Copilot and GPT-4o running both on-device and in the cloud are game-changing. It is important to note the difference between an AI assistant and an AI agent. We have had assistants like Cortana, Siri, Alexa, and Hey Google for over a decade. These AI assistants help us perform basic tasks through our electronic devices. AI agents, however, are capable of acting creatively based on the request of the user, as well as all the information it has access to, including the user’s web search history on the device, current location, type of task being performed, and external information available on the web or through environmental sensors. In addition, with the latest large language models like OpenAI’s GPT-4o (that’s the letter “o”, not a zero), AI agents can interact with a user through natural conversation.

Microsoft has spent the past year rearchitecting the Windows experience around Copilot, which uses LLMs from OpenAI and others. It added a Copilot feature into the Edge browser, Microsoft Office applications, Microsoft Teams, and third-party applications. As announced at the Microsoft Build Conference last week, Microsoft has developed several applications that will run completely on the new AI PCs or Copilot+PC according to Microsoft. These include a Copilot live translation of 44 different languages into English, an information search of everything a user has done or accessed on the device called Recall, and a new cocreator function in Microsoft Paint than leverages Stable Diffusion models. Microsoft is also offering ways to create custom Copilots through Sharepoint, the new Copilot Studio, and Visual Studio.

Microsoft is also working with third-party independent software developers (ISVs) like Adobe, Esri, GitHub, Thompson Reuters, and many more to integrate Copilots into their applications. Today, these all look like the typical text-based chatbot. However, Microsoft is integrating GPT-4o for multimodal support. Pretty soon you can text, reference images, or just talk to these applications and your computer like HAL 9000 (from 2001: Space Odyssey).

With the introduction of generative AI, the way we think about searching for and developing content has changed. But Microsoft views the value of Copilot in a much broader view. It views these AI agents or “Copilots” as a new interface to working with digital platforms that can be quickly and easily customized to the particular application or task. Microsoft also believes that these Copilots will be used by everyone and on every device. I am paraphrasing a bit here from Microsoft CEO Satya Nadella.

Key Takeaways

It is easy to see the value of each of the recent AI and PC announcements on their own, but when you look at the bigger picture, there are a few key things to note. The first and most important is that the speed at which AI is coming is accelerating. AI models like GPT-4o combined with new PC and server hardware and applications are rapidly changing what we can do with electronic devices and how we interact with them. While this is not Artificial General Intelligence (AGI), it is artificial intelligence that is becoming practical and easily deployed.

The second is that on-device AI is real. While it still has its limitations, the improvements in small language models built on the ever-increasing intelligence of large language models is making AI practical and secure.

The third takeaway is the changing architectures for PCs and other electronic devices. Having NPUs capable of high-performance matrix multiplication is becoming critical. For this generation of Surface products and this new Copilot+PC class of PCs, Microsoft chose to launch with a Qualcomm processor because of its performance and power efficiency. In Microsoft’s view, the real competition is not from x86 PCs, but from Apple Macs. According to Microsoft, the new Surface products will be “59% percent faster than the MacBook Air with the M3 (SoC) and have 20% longer battery life.”

In addition, this competition pits two Arm-compatible SoCs against one another for PC SoC supremacy. Please note that these are both completely custom CPU and NPU designs from Apple and Qualcomm, not standard Arm processors, so do not count out the x86 camp of AMD and Intel just yet. AMD and Intel also have PC SoCs with NPUs and will be introducing new SoCs this year as well. Intel has already provided hints of its new Lunar Lake SoC. However, this does put Qualcomm in a head-to-head competition with AMD and Intel for the first time. According to Microsoft, it completely rearchitected Windows to run on the Snapdragon SoCs. Going forward, designing SoCs for AI is going to be critical.

Final Thoughts

To some, this leap in computing is frightening, while to technophiles like me, it is extremely exciting. In either case, it is real and 2024 will mark a major shift in the history of computing from PCs to AI PCs.

There will also likely be divisions within the AI PC segment. I mentioned earlier that while the focus seems to be on architecting SoCs with higher AI performance, there is still the potential for discrete AI acceleration which is being performed with existing GPUs. I will discuss this more in my next article on AI PCs.

And finally, there is still one feature missing – cellular connectivity. It is great to have AI on PCs, but it would be nice to have connectivity anywhere and anytime for mobile PCs. This is something the PC ecosystem still struggles to provide in a simple and cost-effective manner. Maybe 2024 will see a change here as well especially with Qualcomm, the leader in mobile cellular wireless chipsets, taking such a prominent slot in this first wave of AI PCs.