Games are increasingly being rendered using AI, so Microsoft is bringing AI into the way future graphics chips will render games.
Microsoft introduced DirectX Linear Algebra as well as the DirectX Compute Graph Compiler into its DirectX programming interface on Thursday, with previews of each technology due later this year.
If nothing else, Microsoft’s positioning statement regarding both technologies helps explain where each will fit in.
“[Machine learning] is no longer a niche optimization or a postprocess trick,” Adele Parsons, a graphics manager for Microsoft, wrote in a blog post. “It’s increasingly embedded throughout the graphics pipeline, influencing how frames are generated, how content is authored, and how game developers realize their artistic vision. DirectX is evolving to support this future— one where ML is a first-class citizen alongside traditional rendering workloads.”
Most enthusiasts understand how artificial intelligence or machine learning is used by graphics chips. Upscaling asks the GPU to render the scene using a less-complex lower resolution, then uses AI techniques to upscale or increase the resolution to the desired quality. Frame generation asks the GPU to render a particular frame, then another; it then uses AI to interpolate what the player should see in the intervening frames. While that might introduce a bit of latency (lag) it can push frame rates to far higher levels, greatly improving visual smoothness.
Mark Hachman / Foundry
Both techniques are combined to augment the actual rendered output of the GPU, allowing budget or integrated GPUs, like Intel’s new Panther Lake, to compete with older, discrete GPUs in how they play games.
Microsoft’s Max McMullen, a software engineering manager at Microsoft, invited representatives from AMD, Intel, and Nvidia to appear on stage with him at the Game Developer Conference in San Francisco, as evidence of their support.
It’s just math
What DirectX Linear Algebra does is simply support the math used by AI. Traditional GPUs used vector-matrix operations for calculating 3D shapes and lighting. Processors designed for AI, such as a new breed of workstation GPUs, use matrix-matrix math. But logic like Nvidia’s Tensor cores, which have become more and more important over time, perform matrix-matrix calculations.
DirectX Linear Algebra isn’t so much about giving game developers control over AI, however. What Microsoft has discovered, according to its blog post, is that certain features, like temporal upscaling, depend on matrix math — and that those work very well when applied to shaders. Shaders are like rendering instructions for your GPU, and they’re typically downloaded before you begin playing a game — which Microsoft hates.
The DirectX Compute Graph Compiler, however, could have much more potential. Older tools like AMD’s first-gen FSR draw scenes by looking at changes on a per-pixel level, describing changes from one frame to the next. But modern versions of FidelityFX Super Resolution (as well as Nvidia DLSS) have migrated to full-model integration, where the entire scene or model is examined. Instead of instructing pixels to “move,” the AI essentially calculates where the pixels should be and assigns them accordingly.
Put another way, per-pixel interpolation might not know if a “ball” moved behind a “tree.” The idea is that a full-model interpolation would, and provide a more accurate representation of the scene. What Microsoft is trying to do is migrate this into the DirectX pipeline itself.
Among other things, a game designed around both DirectX technologies could essentially talk to the GPU and construct its own shaders — and could do this far into the future for GPUs that weren’t available at the time of the game’s publication, noted Don Brittain, a distinguished engineer at Nvidia.
Some gamers, however, reject the notion of “fake frames,” where AI tries to correctly guess what the GPU would otherwise render. Both of these DirectX technologies would push this concept further. Executives spoke of “neural texture compression,” where AI would essentially guess at what a compressed texture should look like when uncompressed; and “neural lighting,” where AI would calculate where it thought light rays should go.
Mark Hachman / Foundry
The tradeoff is to make more advanced features available to a broader swath of gamers. Neural texture compression could decrease the need for the gobs of memory and storage game textures consume — up to 30 percent, McMullen said. Neural radiance could reduce the need for dedicated ray-tracing units, and make photorealistic “path tracing” more accessible for more gamers.
Neither technology, however, is near. The DirectX Compute Graph Compiler will be available for private preview this summer, Microsoft said. DirectX Linear Algebra will enter public preview in April. It will be sometime after that before they become part of DirectX proper, and then become adopted by the industry at large.