Red Hat acquires Neural Magic, a software and algorithm pioneer in GenA

On Wednesday (13), Red Hat completed the purchase process to acquire Neural Magic, an American company that pioneered Generative Artificial Intelligence (GenAI) software and algorithms.Neural Magic's expertise in performance engineering, along with its commitment to open source, aligns with Red Hat's vision for delivering high-performance AI that fits different scenarios and customer use cases, anywhere in the hybrid cloud.  

While the promise of GenAI dominates much of the current technology landscape, the large language models (LLMs) that underpin these systems continue to grow. As a result, building reliable and cost-effective LLM services requires significant computational power, energy resources, and specialized operational skills.

With the acquisition of Neural Magic, Red Hat aims to address these challenges by making GenAI more accessible to more organizations through the open innovation of vLLM. Developed by UC Berkeley, vLLM is a community-maintained open source project for open model service (how gen AI models infer and solve problems), with support for all major model families, advanced inference acceleration research, and diverse hardware backends including AMD GPUs, AWS Neuron, Google Hat, Intel Gaudi, NVIDIA GPUs, and x86 CPUs.

For Matt Hicks, President and CEO of the company, the purchase of Neural Magic, along with the development of the vLLM initiative, is the first step to putting the company as a benchmark in artificial intelligence. “We are excited to complement our hybrid cloud-focused AI portfolio with Neural Magic's revolutionary AI innovation, increasing our desire to not only be the open source‘’s open source Hat, but also the ‘s HAT of AI’ said. 

Red Hat + Neural Magic: enabling a future with hybrid cloud-ready gen AI 

Neural Magic spun off from MIT in 2018 with the goal of building high-performance inference software for deep learning (Deep Learning). With Neural Magic's technology and performance engineering expertise, Red Hat seeks to accelerate its vision for the future of AI, powered by Red Hat's AI technology portfolio.Built to overcome the challenges of large-scale enterprise AI, the company uses open source innovation to further democratize access to AI's transformative power through:

• Open source licensed models at scale from 1 bi to 405 bi parameters that can operate anywhere from hybrid cloud to enterprise data centers, multi-cloud and at the edge. 
• Tweak features that allow organizations to customize LLMs more easily with their private data and use cases with a more robust security framework.
• Experience in inference performance engineering resulting in increased operational and infrastructure efficiencies 
• An open source and partner ecosystem and support frameworks that enable customers to have more choice, from LLMs and tools to certified server hardware and chip architectures.

VLLM leadership to enhance Red Hat AI

Neural Magic will use its vLLM expertise and knowledge to build an enterprise-level technology foundation that enables customers to optimize, deploy and scale LLM workloads in hybrid cloud environments with full control over infrastructure choice, security policies and model lifecycle. Neural Magic also develops model optimization research, builds the LLM Compressor (a unified library to optimize LLMs with state-of-the-art sparsity and quantization algorithms), and maintains a repository of pre-optimized models ready to deploy with vLLM.

Red Hat AI seeks to help customers reduce AI costs and skill barriers with powerful technologies such as: 

• Red Hat Enterprise Linux AI (RHEL AI), a platform for base models to seamlessly develop, test and operate the IBM Granite family of open source LLMs for enterprise applications in Linux server deployments; 
• Red Hat OpenShift AI, an AI platform that provides tools to quickly develop, train, serve, and monitor machine learning models in distributed Kubernetes environments on-site, in the public cloud, or at the edge; 
• InstructLab, an open source community project created by Red Hat and IBM that enables anyone to shape the future of GenAI through collaborative enhancement of Granite LLMs, licensed as open source using fine-tuning technology from InstructLab. 

Neural Magic's technology leadership in vLLM will enhance Red Hat AI's ability to support LLM deployments in any environment and anywhere in the hybrid cloud with a ready, highly optimized and open inference stack.

The transaction is still expected to undergo regulatory approval and other customary closing conditions.