On Wednesday (13), Red Hat completed the purchase process to acquire Neural Magic, a US company, a pioneer in Generative Artificial Intelligence (GenAI) software and algorithms. Neural Magic's expertise in performance engineering, along with its commitment to open source, aligns with Red Hat's vision for delivering high-performance AI that fits different customer scenarios and use cases, anywhere in the hybrid cloud.
Although the promise of GenAI dominates much of the current technological landscape, the large language models (LLMs) that underpin these systems continue to grow. As a result, building reliable and cost-effective LLM services requires significant computing power, energy resources, and specialized operational skills. Currently, these obstacles prevent most organizations from realizing the benefits of a more secure, deployment-ready, and personalized AI.
With the acquisition of Neural Magic, Red Hat aims to address these challenges by making GenAI more accessible to more organizations through the open innovation of vLLM. Developed by UC Berkeley, vLLM is a community-maintained open-source project for open model service (how GenAI models infer and solve problems), supporting all major model families, advanced inference acceleration research, and diverse hardware backends, including AMD GPUs, AWS Neuron, Google TPUs, Intel Gaudi, NVIDIA GPUs, and x86 CPUs. Neural Magic's leadership in the vLLM project combined with Red Hat's strong portfolio of hybrid cloud AI technologies will offer organizations an open path to building AI strategies that meet their unique needs, wherever their data resides.
For Matt Hicks, President and CEO of the company, the acquisition of Neural Magic, along with the development of the vLLM initiative, is the first step in positioning the company as a benchmark in artificial intelligence. "We are excited to complement our hybrid cloud-focused AI portfolio with Neural Magic's groundbreaking AI innovation, furthering our ambition to be not only the 'Red Hat of open source,' but also the 'Red Hat of AI,'" he said.
Red Hat + Neural Magic: Enabling a future with hybrid cloud-ready AI.
Neural Magic spun off from MIT in 2018 with the goal of building high-performance inference software for deep learning. With Neural Magic's technology and performance engineering expertise, Red Hat seeks to accelerate its vision for the future of AI, driven by Red Hat's AI technology portfolio. Built to overcome the challenges of large-scale enterprise AI, the company uses open-source innovation to further democratize access to the transformative power of AI through:
- Licensed open-source models ranging from 1 billion to 405 billion parameters that can operate anywhere in the hybrid cloud—in enterprise data centers, across multiple clouds, and at the edge.
- Customization features that allow organizations to more easily tailor LLMs to their private data and use cases with a more robust security framework.
- Experience in inference performance engineering, resulting in greater operational and infrastructure efficiencies.
- An open source ecosystem and network of partners and support structures that give customers more choice, from LLMs and tools to certified server hardware and chip architectures.
vLLM Leadership to Enhance Red Hat AI
Neural Magic will leverage its expertise and knowledge in vLLM to build an enterprise-level technology foundation that enables customers to optimize, deploy, and scale LLM workloads in hybrid cloud environments with complete control over infrastructure choice, security policies, and model lifecycle. Neural Magic also conducts model optimization research, builds the LLM Compressor (a unified library for optimizing LLMs with state-of-the-art sparsity and quantization algorithms), and maintains a repository of pre-optimized models ready for deployment with vLLM.
Red Hat AI aims to help customers reduce AI costs and skills barriers with powerful technologies such as:
- Red Hat Enterprise Linux AI (RHEL AI) , a platform for building models to seamlessly develop, test, and operate the IBM Granite family of open-source LLMs for enterprise applications in Linux server deployments;
- Red Hat OpenShift AI is an AI platform that provides tools to rapidly develop, train, serve, and monitor machine learning models in distributed Kubernetes environments on-site, in the public cloud, or at the edge.
- InstructLab is an open-source community project created by Red Hat and IBM that allows anyone to shape the future of GenAI through the collaborative enhancement of Granite LLMs, licensed as open source, using InstructLab's fine-tuning technology.
Neural Magic's technological leadership in vLLM will enhance Red Hat AI's ability to support LLM deployments in any environment and anywhere in the hybrid cloud with a ready, highly optimized, and open inference stack.
The transaction is still subject to U.S. regulatory approval and other customary closing conditions.
