New updates across Red Hat’s AI portfolio drive major transformations in the enterprise sector. Through Red Hat AI, the company aims to further expand the capabilities needed to accelerate technology adoption, offering more freedom and confidence to customers in deploying generative AI (gen AI) in hybrid cloud environments. Starting with the launch of Red Hat AI Inference Server, third-party validated models in Red Hat AI, and integration with Llama Stack APIs and Model Context Protocol (MCP), the company is repositioning itself in the market for various artificial intelligence modalities.
According to Forrester, open-source software will be the engine to accelerate enterprise AI efforts. As the AI landscape becomes more complex and dynamic, the Red Hat AI Inference Server and third-party validated models offer efficient inference and a tested collection of AI models optimized for performance on the Red Hat AI platform. With the integration of new APIs for gen AI agent development, including Llama Stack and MCP, Red Hat works to simplify deployment complexity, empowering IT leaders, data scientists, and developers to advance their AI initiatives with more control and efficiency.
Efficient inference in hybrid cloud with Red Hat AI Inference Server
The Red Hat AI portfolio features the new Red Hat AI Inference Server, offering faster, more consistent, and cost-effective inference at scale in hybrid cloud environments. This addition is integrated with the latest versions of Red Hat OpenShift AI and Red Hat Enterprise Linux AI, and is also available as a standalone solution, enabling organizations to deploy intelligent applications more efficiently, flexibly, and with better performance.
Tested and optimized models with Red Hat AI and third-party validation
Red Hat AI’s third-party validated models, available on Hugging Face, simplify the selection process for businesses when finding the right models for their needs. Red Hat AI provides a collection of validated models, along with deployment guidance that increases customer confidence in model performance and result reproducibility. Selected models are also optimized by Red Hat, with model compression techniques that reduce their size and increase inference speed, helping to minimize resource consumption and operational costs. Additionally, the continuous model validation process helps Red Hat AI customers stay at the forefront of gen AI innovation.
Standardized APIs for AI application and agent development with Llama Stack and MCP
Red Hat AI is integrating the Llama Stack, initially developed by Meta, along with the MCP from Anthropic, to provide standardized APIs for building and deploying AI applications and agents. Currently available as a developer preview in Red Hat AI, Llama Stack offers a unified API for inference access with vLLM, retrieval-augmented generation (RAG), model evaluation, guardrails and agents, for any gen AI model. MCP enables models to integrate with external tools, providing a standardized interface for connecting with APIs, plugins, and data sources in agent workflows.
The latest version of Red Hat OpenShift AI (v2.20) offers additional improvements for building, training, deploying, and monitoring generative and predictive AI models at scale. Highlights include:
- Optimized model catalog (technical preview):easier access to validated models from Red Hat and third parties, with deployment via web console and full lifecycle management with integrated OpenShift registry.
- Distributed training with KubeFlow Training Operator: running model fine-tuning with InstructLab and distributed PyTorch workloads across multiple Red Hat OpenShift nodes and GPUs, with distributed RDMA networking for acceleration and better GPU utilization to reduce costs.
- Feature store (technical preview):based on the upstream Kubeflow Feast project, provides a centralized repository for managing and provisioning data for training and inference, optimizing data flow and improving model accuracy and reusability.
O Red Hat Enterprise Linux AI 1.5 brings new updates to Red Hat’s foundational model platform, focused on developing, testing, and running large-scale language models (LLMs). Key features of RHEL AI 1.5 include:
- Availability on Google Cloud Marketplace,expanding customer choice to run Red Hat Enterprise Linux AI on public clouds (in addition to AWS and Azure), simplifying deployment and management of AI workloads on Google Cloud.
- Enhanced multilingual capabilitiesfor Spanish, German, French, and Italian via InstructLab, enabling model customization with native scripts and expanding possibilities for multilingual AI applications. Users can also use their own “teacher” and “student” models for greater control in customization and testing, with future support planned for Japanese, Hindi, and Korean.
O Red Hat AI InstructLab on IBM Cloud is now generally available. This new cloud service further simplifies the model customization process, improving scalability and user experience. Businesses can use their data more efficiently and with greater control.
Red Hat’s vision: any model, any accelerator, any cloud
The future of AI should be defined by unlimited opportunities and not constrained by infrastructure silos. Red Hat envisions a horizon where organizations can deploy any model, on any accelerator, in any cloud, delivering an exceptional and more consistent user experience without exorbitant costs. To unlock the true potential of gen AI investments, businesses need a universal inference platform—a new standard for continuous, high-performance AI innovations, both now and in the years to come.
Red Hat Summit
Attend Red Hat Summit keynotes to hear the latest updates from Red Hat executives, customers, and partners:
- Modern infrastructure aligned with enterprise AI—Tuesday, May 20, 8 AM–10 AM EDT (YouTube)
- Hybrid cloud evolves to drive enterprise innovation—Wednesday, May 21, 8 AM–9:30 AM EDT (YouTube)
