Contributed By Daniel Wong, University of California, Riverside
The previous research update post can be found here.
Project Overview
As data centers continue to grow, energy consumption has become a first-order design constraint. Yet Linux power management has not kept pace with modern workloads, which demand faster, coordinated decisions that preserve QoS under increasingly volatile load conditions. In our previous post, we discussed our initial progress in optimizing Linux power management by controlling CPU frequency states. In this update, we show how both P-states and C-states can be dynamically adjusted based on tail-latency requirements, creating additional opportunities for power savings.
Our work uses eBPF to dynamically manage both the frequency states (P-states) and the CPU idle states (C-states) to reduce server power consumption without degrading tail latency. We also highlight the need for coordination between frequency-state and idle-state management, and their challenges. This update explores how both can be managed and optimized together through eBPF-based policies, without requiring intrusive changes to the Linux power management subsystem or scheduler.
Finally, we discuss some directions that can help move toward more effective power management with greater opportunities for energy saving: (1) handling interrupts more carefully, (2) applying traffic shaping, and (3) involving custom scheduler. These techniques can improve the system’s ability to enter deeper idle states without violating the residency time and still meet latency targets.
QoS-aware Power Management with eBPF Governors
Towards our broader goal of enabling dynamic power management without compromising QoS, we developed eBPF Governors. A major challenge in QoS-aware dynamic power management is that the system needs timely feedback from latency-critical workloads while also avoiding the overhead of frequent DVFS reconfiguration. Traditional userspace governors can use application-level QoS metrics, but they often rely on costly feedback paths and sysfs-based frequency updates. These limit the responsiveness of the governors under rapidly changing workloads.
The work addresses these challenges by moving the P-state decision algorithm into low-overhead eBPF programs running in the kernel. Instead of directly selecting fixed-frequency states through userspace, it guides hardware-managed P-states using performance preference hints. This allows the hardware to retain fast, low-level control over frequency selection while making those decisions aware of application-level QoS requirements. The framework also provides efficient QoS reporting paths into the kernel. It uses asynchronous eBPF map mechanisms to pass performance feedback with low overhead, and it can further reduce feedback latency by using XDP to intercept network-based QoS signals directly in the kernel.
Figure 1: Overview of eBPF Governors
This initial work, “Agile QoS-aware Dynamic Power Management with eBPF Governors”, has been accepted at the ACM International Conference on Supercomputing (ICS) 2026. We show that, under highly volatile latency-critical workloads, eBPF Governors achieve 2-6x more power savings than state-of-the-art power management policies. At the same time, it maintains QoS and reduces workload latency by avoiding the overheads associated with userspace-based power-state updates. The paper is available here: https://danielwong.org/files/eBPF-Governor-ICS26.pdf
CPU Idle States: Power-Saving Opportunities and Costs
While CPU frequency scaling controls how fast a core executes work, CPU idle-state management controls how much power the core consumes when there is no work to run. Like CPUFreq, the CPUIdle subsystem can create significant opportunities for power savings, but using idle states too aggressively can also introduce overhead. The main challenge is that entering and exiting an idle state is not free. Deeper idle states usually save more power, but they also have higher entry and exit costs. A core may take longer to wake up from a deep idle state, and that wake-up delay can directly affect latency-sensitive workloads.
Additionally, waking up too early can also be detrimental. Residency time refers to the minimum amount of time a CPU core should remain in an idle state for that state to actually save power. In other words, the core must sleep long enough to offset the overhead of entering and exiting the idle state. If a core enters a deep idle state but wakes up before satisfying the residency time, the system may not save power and may still pay the latency cost of waking up.
For latency-critical workloads, CPU cores often become idle during the gaps between request arrivals. These inter-arrival gaps create opportunities to use idle states. However, the gaps are not always long enough for deeper idle states to be beneficial. If the next request arrives too soon, waking from a deep idle state can hurt tail latency without providing meaningful energy savings. Prior works have explored ways to create or better exploit these idle windows. For example, some techniques delay work while still meeting the service level objectives (SLO), shape traffic to create longer idle periods, or use network behavior to decide when the system should rely on polling versus interrupts. These ideas show that idle-state management is not only about choosing the deepest available sleep state but about choosing the right state based on workload timing, latency slack, and expected idle duration.
Setting up the Environment with Custom Linux Kernel
To experiment with coordinated CPU frequency and idle-state management, we used a custom Linux kernel with our own cpufreq ext and cpuidle ext support. The key idea behind this environment is to make Linux power-management policies programmable from eBPF while still keeping the actual hardware control inside the existing kernel power-management framework. For both cpufreq and cpuidle, add an ext governor to the available scaling governors and can be selected for eBPF-based policies.
Similar to our previous work with eBPF Governors, this setup exposes carefully controlled kernel hooks to eBPF rather than moving the entire power-management subsystem into a custom runtime. The difference is that, in this work, eBPF programs act as policy providers for the kernel governors. The cpufreq ext governor allows an eBPF program to provide CPU frequency decisions through BPF `struct_ops`, while the cpuidle ext governor allows an eBPF program to select CPU idle states through a similar `struct_ops` interface. The kernel still handles the low-level driver interactions, while the eBPF program supplies the decision logic.
This design gives us a safe and flexible way to prototype QoS-aware power-management policies. Instead of relying on userspace polling or sysfs-based control loops, the eBPF policy can run inside the kernel and respond quickly to workload feedback. At the same time, the interface remains restricted. eBPF programs are only allowed to access selected fields and callbacks exposed by the extensible governors, keeping the control path narrow and avoiding unnecessary interference with unrelated kernel components.
This custom-kernel setup provides the foundation for this work. With both extensible governors available, we can load eBPF programs that implement custom policies for frequency scaling and idle-state selection. This allows us to manage P-states and C-states together from inside the kernel, instead of treating frequency tuning and idle-state selection as independent mechanisms.
Coordinating Frequency and Idle States using eBPF Governors
To this end, this project explores a workload-aware CPU power management runtime that connects live application latency feedback with kernel-level power management mechanisms. The goal is to reduce CPU power consumption while still respecting latency requirements. The runtime (1) monitors application latency through either an XDP-based latency monitor or eBPF-based monitors, such as eBeeMetrics, (2) converts that feedback into a latency slack estimate relative to a target SLO, and (3) uses this signal to guide CPU power decisions. When there is sufficient latency slack, the system can save power more aggressively by lowering CPU frequency or allowing deeper idle states. When latency approaches the target, or when feedback becomes stale, it shifts to a conservative mode by selecting higher frequencies and shallower idle states to protect QoS.
Figure 2: Workload-Aware Control Flow for Coordinated CPU Power Management
This work treats CPU power management as a configurable policy layer rather than a single hard-coded controller. Each experiment is described by a structured configuration that separates three key concerns: the active control mode, the source of runtime feedback, and the actuator policies used to control CPU power behavior. For example, a configuration can specify that the runtime should operate in joint mode, use live QoS feedback from an XDP-based latency monitor, and apply separate eBPF policies for CPU frequency and idle-state selection:
{
"controller": {
"mode": "joint"
},
"metrics_input": {
"kind": "live_qos",
"implementation": "xdp_latency_stats"
},
"actuators": {
"cpufreq_policy": "server_slack_band_lowmidmax",
"cpuidle_policy": "server_slack_band_deep_else_shallow"
}
}
The controller.mode field defines which control path is active. In baseline mode, it observes only the workload. In cpuidle_only mode, it controls idle-state selection. In cpufreq_only mode, it controls frequency selection. In joint mode, it coordinates both CPU frequency and idle-state decisions using the same runtime feedback signal. The metrics_input section defines where feedback comes from. This feedback is converted into a compact control signal based on latency slack, which represents how much room the workload has before it reaches its latency target.
We map latency slack into three control bands, which guide how aggressively the runtime can save power while still protecting QoS.
| Band Name | Slack Amount | Priority |
| Critical | Little or no slack | Prioritize latency and protect QoS |
| Guarded | Some slack, but limited | Balance latency safety with moderate power savings |
| Relaxed | Enough slack available | Save power more aggressively |
Both actuator policies consume this shared slack state. For CPU frequency, the server_slack_band_lowmidmax policy maps the slack band to a frequency range. In the critical band, it selects a high frequency, typically close to 100%. In the guarded band, it selects a middle frequency, such as 70%. In the relaxed band, it selects a lower frequency, such as 40%. If feedback becomes stale or invalid, the policy falls back to maximum frequency as a conservative choice.
For CPU idle states, the server_slack_band_deep_else_shallow policy maps the same slack band to an idle-state decision. When the system is relaxed and there is enough slack to cover the cost of entering a deeper idle state, we allow deeper idle states. Otherwise, it selects a shallow idle state to avoid adding wake-up latency. If feedback is stale, the policy also falls back to shallow idle, again favoring latency safety.
This structure makes it extensible. New policies can be added by defining a new policy name, implementing the corresponding eBPF `struct_ops` program, and extending the daemon to validate and attach that policy. The configuration interface remains stable: users select policies by name, while the runtime handles the details of loading the correct BPF program and connecting it to the appropriate kernel governor. In this way, it separates the control framework from the control policy. The framework provides feedback collection, configuration, calibration, BPF attachment, telemetry, and experiment management. The policies define how live workload feedback should translate into CPU frequency and idle-state decisions.
Why Coordination Matters
Managing CPU frequency states and idle states independently can improve power efficiency in some cases, but the larger opportunity comes from coordinating them. Frequency scaling controls how fast a core executes work, while idle-state selection controls how deeply the core can sleep when there is no work to do. These two decisions are tightly connected.
Figure 3: Power-saving opportunities depend on coordinating CPU frequency, idle-state depth, and available latency slack.
For example, running at a higher frequency may consume more power during execution, but it can also finish requests sooner and create a longer idle window. If that idle window is long enough, the core may enter a deeper sleep state and save more power overall. However, deeper idle states also have higher exit latency and require the core to remain idle long enough to satisfy the state’s residency time. If the core wakes up too quickly, entering a deep idle state may not save power and may even hurt latency.
This creates a trade-off. After waking from a deep idle state, the core may need a higher frequency to catch up and protect the latency SLO quickly. But it does not always need to wake up at the maximum frequency. If the workload still has enough slack, a lower or middle frequency may be sufficient to meet the tail-latency target while saving additional power. The best decision therefore depends on both the available latency slack and the expected idle opportunity.
This project builds on that idea by treating frequency and idle-state decisions as part of the same control problem. When there is little slack, the system prioritizes latency by using higher frequencies and shallower idle states. When there is more slack, it can lower frequency and allow deeper idle states. Coordinating these decisions allows the system to save power more aggressively while still respecting QoS requirements.
Understanding the Coordination Space
We next compare a set of preliminary policies to understand how CPU frequency and idle-state control affect latency and power behavior under the same workload and load pattern. The default Linux configuration uses the schedutil governor for CPU frequency management and the menu governor for CPU idle-state management. We compare this baseline against three eBPF-based control modes:
- cpufreq_only, where only the frequency policy is enabled;
- cpuidle_only, where only the idle-state policy is enabled; and
- joint, where both frequency and idle-state policies are enabled together.
Figure 4: Preliminary comparison of baseline, cpufreq-only, cpuidle-only, and joint policies under the same workload and load pattern.
These early results are not intended to represent the final power-saving potential of the system. Instead, they help us understand how each control path behaves and where better policies are needed. Across the four configurations, we observe that latency and power behavior can vary noticeably even for the same workload and input load pattern. This reinforces the need for workload-aware power management rather than relying on a fixed policy for all conditions.
The baseline and cpufreq_only configurations are able to maintain power savings more effectively in this experiment. However, controlling only idle states does not yet create enough room for meaningful power savings. The joint configuration is therefore the more interesting direction: it allows frequency and idle-state decisions to be made from the same runtime signal, creating an opportunity to coordinate (1) when to finish work quickly, (2) when to lower frequency, and (3) when to allow deeper idle states.
Overall, these preliminary experiments show that simply enabling both control knobs is not sufficient. The next step is to improve the individual policies and, more importantly, improve the coordination between them. Better joint policies should account for latency slack, request rate, feedback freshness, expected idle duration, and idle-state residency time. We expect this direction to expose larger power-saving opportunities while continuing to protect latency SLOs.
Future Directions
Interrupt Handling: Frequent interrupts can wake cores too early, preventing them from staying in deep idle states long enough to save power. Future policies could become interrupt-aware by deciding when to steer, defer, or coalesce interrupts so that cores can remain idle longer while still meeting latency targets.
Traffic Shaping Tradeoffs: We plan to explore traffic shaping at the ingress path, for example, using XDP-based delay or batching. Carefully delaying requests without harming the SLO can create longer idle windows, making deeper C-states more useful. However, shaping traffic also adds queueing delay, so the policy must balance the power-saving benefit of longer idle periods against the risk of increasing tail latency.
Scheduler Integration: A scheduler-aware policy could consolidate work onto fewer cores, leaving other cores idle for longer periods and making deeper sleep states more effective. Combining scheduling decisions with cpufreq and cpuidle control in eBPF would move the system closer to a fully coordinated, workload-aware power-management framework.
In general, this project demonstrated how eBPF can unlock new pathways towards efficient power management in modern server systems. Building on our foundation, we will continue to push the boundaries of how eBPF mechanisms can benefit and provide greater flexibility and efficiency in modern data center settings.