Red Hat AI 3.5 adds fair-share GPU sharing across tenants, and lets teams check a model against safety benchmarks before it ships

Red Hat shipped Red Hat AI 3.5 on 9 September, and put multi-tenant GPU sharing, pre-deployment safety benchmarking, and per-user token metering in the same release. The company positions it as the version where a platform team can run more than one workload on a shared GPU pool without giving up isolation between them, and check a model against safety and capability tests before it reaches production.

The New Stack's coverage, published on 10 September, lays out the pieces and quotes Red Hat's senior director of product for Red Hat AI, Tushar Katarki, saying the release is meant to run AI "as a mission-critical service rather than an unpredictable experiment".

Fair-share GPU scheduling with an admission gate

The scheduler splits GPU capacity between tenants by fair share, and adds priority-aware serving with an admission control on top. Red Hat's own description says the admission control protects real-time inference by refusing or delaying lower-priority requests when the GPU is busy, and lets background workloads take unused capacity when it is free. Both pieces are in one shared pool rather than in separate provisioned clusters, so a team does not buy a second GPU for a batch job.

Applied mathematician Joshua Estrin told The New Stack the trade-off in one line: "every GPU request now becomes a priority decision", and the platform must "prove whose workload actually executed and ran, who had access, what it cost, and what happens when demand spikes".

EvalHub, run the benchmarks before you deploy

EvalHub is Red Hat's evaluation service, and it now covers custom models, retrieval-augmented systems, and agents, with integrated Garak safety and toxicity scores. Red Hat's press release says a team can generate the compliance certifications this produces before a model is deployed, and use the same benchmarks to compare versions of a model or the same model on different infrastructure. Red Hat says more than 20 new models were validated with these scores in this release, including Google Gemma 4, NVIDIA Nemotron 3, and Alibaba Qwen.

The point is that the safety check runs against the model in the environment it will actually serve from, rather than against a leaderboard number the model author published.

Observability, per-user token metering, and CPU offloading

New dashboards report inference health, GPU utilisation, model performance, per-user token consumption, and distributed inference workloads. Per-user metering is what Red Hat calls MaaS showback, and it is available to non-admin users, so a team can see its own consumption without opening a support ticket.

CPU offloading is generally available in this release, and storage offloading is in developer preview. Both let a running model handle longer conversations or larger documents by moving parts of the key-value cache off the GPU, without adding more GPU hardware.

What is not in the announcement

Red Hat's press release names no benchmark number for how much throughput the fair-share scheduler recovers on a busy GPU, no example latency under admission control, and no price for Red Hat AI 3.5. It calls the release generally available on the date of the announcement, and it is also available as part of Red Hat AI Factory with NVIDIA. The comparison you can measure yourself is the one that matters: a workload on the older version, then the same workload on 3.5, on the same hardware.

Source

Red Hat press release: Red Hat Puts Safety and Observability at the Core of Enterprise AI with Red Hat AI 3.5, and The New Stack: Red Hat AI 3.5 tackles the GPU queue that can stall AI pilots.