Express memory offset in reported-clock domain (2x at NVML boundary) #21

Merged
Pakobbix merged 1 commits from fix/mem-offset-reported-domain into main 2026-11-12 12:49:34 +00:00
Collaborator

Problem

On GDDR7/Blackwell the memory V/F offset NVML APIs operate in a domain 2x the reported memory clock (measured RTX 5090, driver 615.78.08): raw +3000 moved under-load clock only +1500 MHz; driver range -2000..+6000 = NvAPI -1000..+3000. UI/API/CLI exposed raw values, so the slider's "+3000" only reached half the driver headroom.

Changes

  • limits.py: _MEM_OFFSET_DOMAIN_FACTOR=2 — multiply on set, divide on readback/range, MEM only (GPC untouched). API/CLI/profiles/UI now speak the reported-clock domain (nvidia-smi); UI +3000 = full headroom (raw +6000).
  • limits.py: call nvmlDeviceGet/SetClockOffsets via raw function pointer for the real driver rc — nvidia-ml-py wrappers (13.615.71) discard ret and always report success, masking erroring drivers and skipping the deprecated-API fallbacks.
  • PerformancePanel.tsx: fallback range -1000..+3000, removed stale Math.min(max,3000) cap, domain tooltip.
  • Docs: domain documented in API.md, Usage-Guide.md.

Verified live

  • set(1500)→raw 3000→clock 15301; set(3000)→raw 6000→max 16801
  • Simulated erroring driver (rc=26, wrappers absent): get/set fell back to deprecated APIs correctly
  • /etc/nvcurve/profiles/*.json migrated 3000→1500 (identical hw behavior); auto-load verified after restart
  • Deployed: reinstalled, service restarted, /api/limits reports new domain

Not included: profile schema versioning (guards old/new mixups) — follow-up.

## Problem On GDDR7/Blackwell the memory V/F offset NVML APIs operate in a domain 2x the reported memory clock (measured RTX 5090, driver 615.78.08): raw +3000 moved under-load clock only +1500 MHz; driver range -2000..+6000 = NvAPI -1000..+3000. UI/API/CLI exposed raw values, so the slider's "+3000" only reached half the driver headroom. ## Changes - limits.py: _MEM_OFFSET_DOMAIN_FACTOR=2 — multiply on set, divide on readback/range, MEM only (GPC untouched). API/CLI/profiles/UI now speak the reported-clock domain (nvidia-smi); UI +3000 = full headroom (raw +6000). - limits.py: call nvmlDeviceGet/SetClockOffsets via raw function pointer for the real driver rc — nvidia-ml-py wrappers (13.615.71) discard ret and always report success, masking erroring drivers and skipping the deprecated-API fallbacks. - PerformancePanel.tsx: fallback range -1000..+3000, removed stale Math.min(max,3000) cap, domain tooltip. - Docs: domain documented in API.md, Usage-Guide.md. ## Verified live - set(1500)→raw 3000→clock 15301; set(3000)→raw 6000→max 16801 - Simulated erroring driver (rc=26, wrappers absent): get/set fell back to deprecated APIs correctly - /etc/nvcurve/profiles/*.json migrated 3000→1500 (identical hw behavior); auto-load verified after restart - Deployed: reinstalled, service restarted, /api/limits reports new domain Not included: profile schema versioning (guards old/new mixups) — follow-up.
ARIA added 1 commit 2026-11-12 12:43:37 +00:00
On GDDR7/Blackwell (measured: RTX 5090, driver 615.78.08) the memory
V/F offset NVML APIs operate in a domain 2x the memory clock NVML
reports. Applying +3000 moved the under-load clock by only +1500 MHz;
the driver range -2000..+6000 maps to NvAPI's -1000..+3000.

- limits.py: multiply by _MEM_OFFSET_DOMAIN_FACTOR (2) on set, divide
  on get/readback/range for the MEM domain only; GPC untouched.
  API/CLI/profiles/UI now all speak the reported-clock domain
  (what nvidia-smi shows), so a +3000 slider value reaches the
  driver's full +6000 headroom (old UI half-used it).
- limits.py: call nvmlDeviceGet/SetClockOffsets via the raw function
  pointer for the real driver return code - nvidia-ml-py wrappers
  discard it and always report success, which would have masked
  erroring drivers and skipped the deprecated-API fallbacks.
- PerformancePanel: fallback range -1000..+3000, remove stale
  Math.min(max, 3000) cap that clamped the new max, add domain tooltip.
- Docs: document the domain conversion (API.md, Usage-Guide.md).
Collaborator

This PR looks good — ready to merge. The 2x domain factor is applied consistently at every NVML touch point (set, readback, range, and both deprecated-API fallbacks), the pynvml raw-pointer replacement correctly restores real return codes with sane degradation to the wrapper/CDLL/deprecated paths, and the frontend fallbacks (-1000..+3000) plus docs match the new reported-clock domain. (Note: the repo test suite could not execute in this environment due to missing NVIDIA driver libraries and a sandbox mmap restriction, but the affected code paths are uncovered by the existing tests anyway.)

This PR looks good — ready to merge. The 2x domain factor is applied consistently at every NVML touch point (set, readback, range, and both deprecated-API fallbacks), the pynvml raw-pointer replacement correctly restores real return codes with sane degradation to the wrapper/CDLL/deprecated paths, and the frontend fallbacks (-1000..+3000) plus docs match the new reported-clock domain. (Note: the repo test suite could not execute in this environment due to missing NVIDIA driver libraries and a sandbox mmap restriction, but the affected code paths are uncovered by the existing tests anyway.)
Pakobbix merged commit 9d36bddb79 into main 2026-11-12 12:49:34 +00:00
Pakobbix deleted branch fix/mem-offset-reported-domain 2026-11-12 12:49:35 +00:00
Sign in to join this conversation.