Red Hat Acquires Neural Magic, the AI Optimization Startup

Red Hat, an IBM subsidiary focused on open-source software, announced the acquisition of Neural Magic, a startup that optimizes artificial intelligence (AI) models to improve their performance on mainstream processors and GPUs.

"Neural Magic's technology will enable Red Hat to offer its customers an enterprise-grade AI stack based on the open source vLLM project, thereby optimizing the deployment of AI models in cloud environments with a focus on infrastructure control and security," Matt Hicks, CEO of Red Hat told TechCrunch.

Founded in 2018 in Massachusetts by scientist Alex Matveev and MIT professor Nir Shavit, Neural Magic specializes in developing software that optimizes AI models to run on mainstream processors and GPUs.

Unlike specialized chips such as TPUs (Tensor Processing Units), which require dedicated hardware, Neural Magic's technology enables AI models to run on general-purpose hardware, offering higher memory capacity and lower costs compared to dedicated AI chips.

"AI workloads need to run wherever customer data lives across the hybrid cloud; this makes flexible, standardized, and open platforms and tools a necessity, as they enable organizations to select the environments, resources, and architectures that best align with their unique operational and data needs," said Hicks in Red Hat's press release.

Neural Magic has attracted the attention of several companies due to its focus on AI optimization. It has raised US$50 million in venture capital funding from investors such as Andreessen Horowitz, Comcast Ventures, and Ridgeline Ventures. In addition to its proprietary software, Neural Magic offers open-source tools, a strategy that facilitates the adoption of its technology across various corporate environments.

“Open source has proven time and again to drive innovation through the power of community collaboration,” said Brian Stevens, CEO of Neural Magic. “At Neural Magic, we [ha]ve assembled some of the industry’s top talent in AI performance engineering with a singular mission of building open, cross-platform, ultra-efficient LLM serving capabilities. Joining Red Hat is not only a cultural match, but will benefit companies large and small in their AI transformation journeys.”

According to Red Hat, the acquisition of Neural Magic aligns with its current strategies in artificial intelligence, as the company has been investing in AI-based technologies that complement its hybrid cloud infrastructure. In particular, the vLLM project, in which Red Hat is already participating, enables AI models to run efficiently in environments such as Red Hat Enterprise Linux AI and Red Hat OpenShift AI.

“As our clients look to scale AI across their hybrid environments – virtualized, cloud-native LLMs built on open foundations will become the industry standard,” said Dario Gil, IBM senior vice president and director of Research in Red Hat’s press release. “Red Hat’s leadership in open source combined with the choice of efficient, open source models like IBM Granite and Neural Magic’s offerings for scaling AI across platforms empower businesses with the control and flexibility that they need to deploy AI across the enterprise.”

Red Hat made the announcement during the KubeCon annual cloud computing conference, where it also presented Climatik, an energy optimization tool for data centers developed in collaboration with Intel, Bloomberg, and IBM. In addition, the company launched new versions of its OpenShift AI and Device Edge platforms, focused on artificial intelligence and edge computing solutions.

Red Hat's move is not isolated in the industry; companies such as Nvidia and Intel have pursued similar strategies. In April 2024, Nvidia acquired Run:ai, an AI infrastructure optimization company, and in 2020, Intel bought SigOpt, a company focused on optimizing AI models. These acquisitions reflect a trend in the industry towards acquiring innovative startups that offer AI optimization and scalability solutions.