Processing unit that enables asynchronous task dispatch

What is this patent about?

’381 is related to the field of high-performance computing and graphics processing, specifically addressing how a processing unit manages multiple concurrent workloads. In traditional architectures, a GPU typically processes a single command stream at a time, requiring a costly and time-consuming context switch to swap out state data when a higher-priority task needs to interrupt a running process. This overhead creates significant latency and necessitates extra hardware for backing up data, which limits the efficiency of real-time or high-priority compute tasks.

The underlying idea behind ’381 is the implementation of virtual engines that allow a single shader core to maintain and execute multiple contexts simultaneously. Instead of treating the GPU as a serial processor that must drain its pipeline to switch tasks, the invention enables the hardware to receive multiple task streams in parallel. By partitioning the physical resources of the shader core, the system can handle different priority levels—such as background tasks and low-latency 3D rendering—without the performance penalty of traditional context switching.

The claims of ’381 focus on a processing architecture where a plurality of engines receives tasks from an external scheduler and loads their respective state data into a shared shader core. The independent claims specifically protect the mechanism by which the shader core executes a first task and a second task concurrently based on their individual state data. This includes the coordination between a primary processor, like a CPU, and the specialized processing unit to ensure that tasks are dispatched and executed based on their specific priority types.

In practice, the invention works by utilizing spatial and temporal partitioning of the shader core’s SIMD units. Spatial partitioning assigns specific subsets of processing elements to different tasks, while temporal partitioning allows tasks to share the same elements over interleaved time slices. A scheduling module within the GPU manages these resources dynamically, ensuring that high-priority, low-latency tasks get immediate access to the necessary compute cycles while standard graphics tasks continue to progress in the background.

This approach differs from prior solutions by eliminating the need to flush the pipeline and swap memory contents during a task change. By allowing multiple active contexts to reside within the local data share of the shader core at once, the invention enables a GPU to behave more like a multi-tasking CPU. This is particularly effective for large chips where resources can be divided among concurrent tasks based on demand, ensuring that no single low-latency task is delayed by the overhead of stopping a massive graphics workload.