Skip to content

feat: add --power-monitor flag for dynamic AC/Battery model device switching#146

Open
gnubesoft wants to merge 2 commits into
SearchSavior:mainfrom
gnubesoft:feature/power-monitor
Open

feat: add --power-monitor flag for dynamic AC/Battery model device switching#146
gnubesoft wants to merge 2 commits into
SearchSavior:mainfrom
gnubesoft:feature/power-monitor

Conversation

@gnubesoft

Copy link
Copy Markdown

Summary

This PR adds an optional --power-monitor CLI argument to openarc serve start that automatically switches loaded LLM models between the Intel GPU (on AC power) and the Intel NPU (on Battery power) based on the laptop's charging state. This allows developers to maximize performance when plugged in and conserve energy when running on battery.

Key Changes

  1. Power Monitor Background Task:
    • Implemented an asynchronous background loop in src/server/main.py that polls battery status every 5 seconds using psutil.
    • On power state changes, it unloads active LLM models and reloads them on the target device (GPU for AC power, NPU for Battery).
    • Integrated the background task lifecycle into the FastAPI lifespan manager (starts on boot, cancels cleanly on shutdown).
  2. Model Registry Enhancements:
    • Modified register_unload in src/server/model_registry.py to support a wait: bool parameter. When wait=True, it blocks until the model's background unload task completes, preventing race conditions during hot-swapping.
    • Added a public thread-safe helper get_loaded_models to access registered model records without reaching into private registry properties (_models, _lock).
  3. CLI Integration:
    • Added --power-monitor Click option in src/cli/groups/serve.py which passes the setting via environment variables.
    • Updated the serve start CLI help examples.
  4. Dependencies:
    • Added psutil>=6.0.0 to pyproject.toml project dependencies.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant