Priority-based command execution

What is this patent about?

’943 is related to the field of heterogeneous computing and command scheduling between different processing units. Specifically, it addresses the latency challenges encountered when a central processing unit (CPU) offloads non-graphical tasks, such as physics simulations or artificial intelligence calculations, to a graphics processing unit (GPU). In traditional systems, these time-sensitive computational tasks often get stuck behind large batches of rendering commands, leading to performance bottlenecks because the CPU cannot receive the necessary results in time for the next frame.

The underlying idea behind ’943 is the implementation of a multi-tiered, priority-aware pipeline that spans from the software abstraction layer in the CPU down to the hardware execution level in the GPU. By categorizing commands into distinct priority levels—such as high-priority computational tasks versus mid-priority rendering tasks—the system can bypass the standard first-in, first-out bottlenecks. The core insight is to maintain this priority separation throughout the entire transfer process, ensuring that urgent data does not lose its precedence while moving from the application driver to the execution cores.

The claims of ’943 focus on a method and architecture that utilizes a plurality of virtual devices within the CPU to abstract the GPU. Each virtual device is assigned a specific priority level and maintains its own queue. The independent claims describe a hardware-software handshake where commands are routed through these priority-specific virtual devices and then mapped to a corresponding set of hardware buffers on the GPU. A specialized command processor then manages these buffers, specifically ensuring that high-priority commands are retrieved and dispatched to the processing core before those in lower-priority queues.

In practice, the invention works by using a multithreaded command processor and a run-list controller to monitor the status of multiple ring buffers simultaneously. When a high-priority command arrives in its dedicated buffer, the command processor can trigger a preemption mechanism. This allows the GPU to finish its current atomic operation and immediately pivot to the urgent task, rather than waiting for an entire rendering command buffer to complete. The system tracks the state of these multiple streams using dedicated sets of head and tail pointer registers for each priority level.

This approach differs from prior solutions by moving away from a single, monolithic command stream that treats all GPU tasks as asynchronous background work. By integrating priority-based scheduling directly into the hardware ring buffers and the CPU's virtual device drivers, the invention enables the GPU to function as a low-latency co-processor for real-time logic. This allows for a more efficient distribution of labor, where the CPU is relieved of heavy mathematical burdens without the penalty of missing frame-timing windows.