Qualcomm on-device AI powers future products from phones to PCs

Qualcomm recently made some major AI announcements at the Mobile World Congress Barcelona event. Each announcement supports Qualcomm’s goal of running AI at scale everywhere, including at the edge. The announcement included:

• Large-scale AI running on a wide range of next-generation edge devices such as PCs, cars, phones, industrial IoT devices, Wi-Fi access points, and cellular infrastructure.
• Qualcomm AI Hub provides developers with resources including over 75 AI models optimized for Snapdragon and other Qualcomm platforms.
• Qualcomm researchers are developing large-scale multimodal models and customized large-scale vision models that can run on Android smartphones and Windows PCs.

These advances rely on new Qualcomm hardware. The company’s Snapdragon 8 Gen 3 SoC for mobile devices was announced at MWC, and the Snapdragon X Elite for PCs was announced a few months ago. These chipsets enable on-device AI in many devices, including: Samsung Galaxy S24 smartphoneLatest iPhone And many mobile phones are manufactured in China. In the coming months, Qualcomm is expected to launch an AI PC SoC with a CPU, GPU, and NPU capable of processing speeds of 45 TOPS.

Qualcomm also announced the AI-powered Snapdragon X80 5G modem and next-generation Wi-Fi 7 chip FastConnect 7900. These products are explained in detail by my colleague Anshel Sag.

Let’s take a closer look at AI-related announcements.

Qualcomm AI Hub

AI-enabled phones, PCs, and other devices require new applications to take full advantage of their AI capabilities. Qualcomm created the AI ​​Hub to provide developers with the resources they need to create and deploy AI applications on Snapdragon or other Qualcomm platforms. One of the key enablers of AI Hub is a library of AI models that are quantized and optimized for high performance on these platforms.

Part of the appeal of hubs is their ease of use. Developers simply select their desired model from over 75 popular AI and generative AI models and choose a framework reference (TensorFlow, PyTorch, or Onyx). The model is also available on Hugging Face and GitHub.

Selecting a specific model allows developers to select target platforms and devices. Then, for deployment, developers insert a few lines of code to integrate the optimized model into their workflow and leverage on-device AI capabilities running on one of Qualcomm’s platforms.

In addition to numerous features such as image and text generation, the hub provides up to 4x faster inference on devices powered by Snapdragon or other Qualcomm platforms.

Large-scale AI models can now run on Android

At this event, Qualcomm successfully demonstrated the world’s first large-scale language model and large-scale multimodal model running on Android smartphones. With 7 billion parameters, his LLM can accept text, images, and audio, allowing users to have multiple conversations on their phones using multiple interactions.

Users ask questions or input information, and LMM AI responds accordingly. Multi-turn conversations require contextual memory so that the AI ​​can remember the context and complete history of the information exchanged between the user and the AI. Multiturn LLM has the ability to handle the complex questions required for highly interactive applications such as customer service.

LoRA for Android smartphones

Qualcomm also achieved another technological milestone with the first low-rank adaptation to run on Android smartphones. LoRA was originally developed by Microsoft to reduce the complexity of model training in terms of latency, cost, and hardware requirements. LoRA offers a new approach to image generation and is different from generative AI tools such as DALL-E. LoRA reduces memory requirements and improves efficiency by reducing model complexity, allowing you to customize and adapt lightweight versions of your models without downloading or fine-tuning the full model. can.

Snapdragon X Elite Platform

Qualcomm built Snapdragon X Elite to run AI. Its design includes a 4nm SoC architecture, a 12-core Qualcomm Oryon CPU with dual-core boost capability, and an integrated Qualcomm Adreno GPU for graphics. This device can last for several days on a single battery charge, eliminating the hassle of frequent charging. One of the most important advantages is that Snapdragon X Elite can run his 13 billion parameter generative AI LLM model.

To demonstrate the SoC’s capabilities, Qualcomm ran a performance comparison between the Snapdragon X Elite PC and a competing x86 with the most efficient GPU, CPU, and NPU configuration. For the comparison, we used GIMP, a powerful open source raster graphics editor with the Stable Diffusion plugin for GAI image generation. As the figure shows, the Snapdragon X Elite with 45 TOPS NPU produced images three times faster than its x86 competitors.

Qualcomm’s role in localizing AI

All of Qualcomm’s AI announcements at MWC Barcelona represented significant advances. The company has successfully pushed its AI and computing beyond the boundaries of traditional devices, from smartphones to PCs. The Snapdragon 8 Gen 3 SoC and Snapdragon X Elite PC platform for mobile devices can perform a variety of AI functions across generative AI, computer vision, and natural language processing. Qualcomm also provided developers with enhanced AI capabilities.

The ability to run large multimodal models and lightweight versions of models on Android smartphones enables new capabilities across a wide range of domains. Android users will be able to interact with virtual assistants in applications such as financial services, healthcare, and e-commerce, just to name a few. AI also gives games a whole new dimension. In addition to challenging challenging opponents, AI game agents can generate new game content and dynamically adjust rules and game play depending on a player’s expertise and desire for greater challenge. You can also Applying AI makes almost any game more challenging and realistic. Android smartphones can run complex AI models efficiently, providing better user experiences and improved functionality through natural language processing and computer vision.

The full democratization of AI will be heavily influenced by Qualcomm’s commercialization of on-device AI at scale at the edge and elsewhere. This is a concept that supports the industry’s goal of enabling intelligent computing everywhere. Taken together, the MWC announcements position Qualcomm as a major player in the AI ​​space. These capabilities should help drive AI adoption at scale.