Just in time for the next generation of AI PC processors, Microsoft has announced that Windows ML is finally ready for production. It’s a key development for local AI and the chips that power its use.
On Tuesday, Microsoft quietly said that Windows ML is now generally available, seven years after first introducing the technology at Build.
Windows ML will hopefully solve a persistent problem: consumers care about AI, they simply don’t care where it lives. For many people, “AI” is synonymous with ChatGPT, which resides in the cloud. Qualcomm, Intel, and AMD all have invested heavily into local AI, and powerful NPUs capable of dozens of TOPS (trillions of operations per second).
Analysts say that applications need to be specifically coded for NPUs. Windows ML changes that. “Windows ML is the built-in AI inferencing runtime optimized for on-device model inference and streamlined model dependency management across CPUs, GPUs and NPUs,” Microsoft says.
“This ability to run models locally enables developers to build AI experiences that are more responsive, private and cost-effective, reaching users across the broadest range of Windows hardware,” Microsoft adds.
Essentially, Windows ML is designed to tally all the resources on a PC, and allocate the task or app to the most suitable hardware: GPU, CPU, or NPU. It’s unclear whether Windows ML or the developer will prioritize NPU efficiency or GPU power, but that’s beside the point. If the app can take full advantage of the available hardware, that’s a win-win for everyone.