This Wednesday (13), Red Hat has completed the acquisition process to acquire Neural Magic, American company, pioneering in software and algorithms for Generative Artificial Intelligence (GenAI). The expertise of Neural Magic in performance engineering, together with your commitment to open source, aligns with Red Hat's vision for delivering high-performance AI that fits various customer scenarios and use cases, in any place of the hybrid cloud.
Although the promise of GenAI dominates much of the current technological landscape, the large language models (LLMs) that support these systems continue to grow. As a result, the construction of reliable and cost-effective LLM services requires significant computational capacity, energy resources and specialized operational skills. Currently, these obstacles make the benefits of a personalized AI unfeasible, ready for deployment and safer for most organizations
With the acquisition of Neural Magic, Red Hat aims to address these challenges by making GenAI more accessible to more organizations through the open innovation of vLLM. Developed by UC Berkeley, vLLM is an open source project maintained by the community for open model service (such as how gen AI models infer and solve problems), with support for all major model families, advanced research on inference acceleration and various hardware backends, including AMD GPUs, AWS Neuron, Google TPUs, Intel Gaudi, NVIDIA GPUs and x86 CPUs. Neural Magic's leadership in the vLLM project combined with Red Hat's strong portfolio of hybrid cloud AI technologies will provide organizations with an open path to building AI strategies that meet their unique needs, wherever your data is
For Matt Hicks, President and CEO of the company, the purchase of Neural Magic, together with the development of the vLLM initiative, are the first step to position the company as a reference in artificial intelligence."We are excited to complement our hybrid cloud-focused AI portfolio with Neural Magic's revolutionary AI innovation", increasing our desire to not just be the 'Red Hat of open source', but also the 'Red Hat of AI', said.
Red Hat + Neural Magic: Enabling a Hybrid Cloud-Ready Gen AI Future
Neural Magic spun off from MIT in 2018 with the goal of building high-performance inference software for deep learning.. With the technology and performance engineering expertise of Neural Magic, Red Hat is looking to accelerate its vision for the future of AI., driven by Red Hat's AI technology portfolio. Built to overcome the challenges of large-scale enterprise AI, the company uses open source innovation to further democratise access to the transformative power of AI by:
- Licensed open source models ranging from 1 billion to 405 billion parameters that can operate anywhere in the hybrid cloud — in corporate data centers, in several clouds and on the edge.
- Adjustable features that allow organizations to customize LLMs more easily with their private data and use cases with a more robust security framework
- Experience in inference performance engineering, resulting in greater operational and infrastructure efficiencies
- An open source ecosystem and partners and support structures that enable customers to have more choice, from LLMs and tools to certified server hardware and chip architectures
vLLM Leadership to Advance Red Hat AI
Neural Magic will use its expertise and knowledge in vLLM to build a technology base, at enterprise level, which allows customers to optimise, deploy and scale LLM workloads in hybrid cloud environments with full control over infrastructure choice, security policies and model life cycle. Neural Magic also develops model optimization research, builds the LLM Compressor (a unified library to optimize LLMs with state-of-the-art sparsity and quantization algorithms) and maintains a repository of pre-optimized models ready to be deployed with vLLM
Red Hat AI seeks to help customers reduce AI costs and skills barriers with powerful technologies, how
- Red Hat Enterprise Linux AI (RHEL AI), a platform for basic models to develop, test and fully operate the IBM Granite family of open source LLMs for enterprise applications in Linux server deployments;
- Red Hat OpenShift AI, an AI platform that provides tools to develop, train, quickly serve and monitor machine learning models in on-site distributed Kubernetes environments, in the public cloud or on the edge;
- InstructLab, an open source community project created by Red Hat and IBM that allows anyone to shape the future of GenAI through the collaborative improvement of Granite LLMs, licensed as open source using InstructLab's fine-tuning technology.
Neural Magic's technology leadership in vLLM will increase Red Hat AI's ability to support LLM deployments in any environment and anywhere in the hybrid cloud with a ready-made inference stack, highly optimised and open
The transaction is still subject to U.S. regulatory approval and other customary closing conditions.
