The concept of artificial intelligence (AI) is not new, but recent advances in related technologies have turned it into a tool used by all of us daily. The growing importance and proliferation of AI is both exciting and potentially alarming, as the foundations of many AI platforms and features are essentially black boxes controlled by a small number of powerful corporations.
Major organizations, like Red Hat, believe that everyone should have the ability to contribute to AI. AI innovation should not be restricted to companies that can afford massive amounts of processing power and the data scientists needed to train these large language models (LLMs)
Instead, decades of open source experience in software development and community collaboration enable everyone to contribute to and benefit from AI, while helping shape a future that meets our needs. There’s no doubt that the open source approach is the only way to realize AI’s full potential, making it safer, more accessible, and democratized.
What is open source?
While the term “open source” originally referred to a software development methodology, it has expanded to encompass a more general way of working that is open, decentralized, and deeply collaborative. The open source movement now extends far beyond the software world, and the open source way has been embraced by collaborative efforts worldwide, including sectors like science, education, government, manufacturing, healthcare, and more.
Open source culture has some core principles and values that make it effective and meaningful, for example:
- Collaborative participation
- Shared responsibility
- Open exchange
- Meritocracy and inclusion
- Community-driven development
- Open collaboration
- Self-organization
- Respect and reciprocity
When open source principles form the basis of collaborative efforts, history shows that amazing things become possible. Notable examples range from the development and proliferation of Linux as the world’s most powerful and ubiquitous operating system to the emergence and growth of Kubernetes and containers, along with the development and expansion of the Internet itself.
Six advantages of open source in the AI era
There are numerous benefits to developing technologies through open source, but six advantages stand out above the rest.
1. Increased innovation speed
When technology is developed collaboratively and openly, innovation and discovery can happen much faster than in closed organizations with proprietary solutions.
When work is shared openly and others can build upon it, teams save enormous amounts of time and effort because they don’t need to start from scratch. New ideas can extend previous projects. This not only saves time and money but also strengthens outcomes as more people work together to solve problems, share insights and review each other’s work.
A broader collaborative community is simply capable of achieving more: bringing people together and connecting expertise to solve complex problems and innovate faster and more effectively than small, isolated groups.
2. Democratizing access
Open source also democratizes access to new AI technologies. When research, code, and tools are shared openly, it helps eliminate some of the barriers that typically limit access to cutting-edge innovations.
The InstructLab is a great example of this premise. This initiative is an open source, model-free AI project that simplifies the process of contributing skills and knowledge to LLMs. The goal is to enable anyone to help shape generative AI (gen AI), including those without the specialized data science skills and training typically required. This allows more individuals and organizations to reliably contribute to training and refining LLMs.
3. Enhanced security and privacy
Because open source projects lower entry barriers, a larger and more diverse group of contributors can help identify and address potential security challenges in AI models as they’re being developed.
Most data and methods used to train and fine-tune AI models are closed and held under proprietary logic. People outside these organizations rarely gain any insight into how these algorithms work or whether they contain potentially dangerous data or inherent biases.
However, if a model and its training data are open, anyone interested can examine them, reducing security risks and minimizing platform biases. Moreover, contributors following open philosophy can create tools and processes to track and audit future model and application development, enabling monitoring of different solution development paths.
This openness and transparency also builds trust, since users can directly examine how their data is being used and processed, allowing them to verify that their privacy and data sovereignty are respected. Additionally, companies can also protect their private, confidential, or proprietary information by using open source projects like InstructLab to create their own fine-tuned models while maintaining strict control over them.
4. Provides flexibility and freedom of choice
While monolithic, proprietary black-box LLMs are what most people see and think about regarding generative AI, we’re beginning to see growing momentum toward smaller, independent, purpose-built AI models.
These small language models (SLMs) are typically trained on much smaller datasets to give them basic functionality, then further adapted for specific use cases with domain-specific data and knowledge.
These SLMs are significantly more efficient than their larger counterparts, and have performed as well (if not better) when used for their intended purpose. They’re faster and more efficient to train and deploy, and can be customized and adapted as needed.
This is largely what the InstructLab project was created for. With it, you can take a smaller open source AI model and expand it with whatever additional data and training you desire.
For example, you could use InstructLab to create a highly tailored, purpose-built customer service chatbot incorporating your organization’s best practices. This approach lets you deliver the best of your customer service experience to everyone, everywhere, in real time.
Most importantly, this lets you avoid vendor lock-in and provides flexibility in where and how you implement your AI model and any applications built upon it.
5. Enables a vibrant ecosystem
In the open community, “nobody innovates alone“, and this belief has held true since the community’s earliest founding months.
This idea will remain valid in the AI era within Red Hat, the leader in open solutions, which will provide various open source tools and frameworks as part of Red Hat AI, a solution through which partners will generate more value for end customers.
A single vendor can’t provide everything an organization needs, or even keep up with technology’s current pace of evolution. Open source principles and practices accelerate innovation and enable a vibrant ecosystem by fostering partnerships and collaboration opportunities across projects and industries.
6. Reducing costs
By early 2025, it’s estimated that the average base salary for a data scientist in the United States will exceed $125,000, with more experienced data scientists earning significantly more.
There’s obviously tremendous and growing demand for data scientists in AI, but few companies hold much hope of attracting and retaining the specialized talent they need.
Truly large LLMs are exorbitantly expensive to build, train, maintain, and deploy, requiring entire warehouses full of highly optimized (and very expensive) computing equipment and massive amounts of storage.
Smaller, purpose-built open models and AI applications are significantly more efficient to build, train, and implement. They require just a fraction of the computing power of LLMs, and projects like InstructLab enable people without specialized skills and experience to actively and effectively contribute to AI model training and fine-tuning.
Clearly, the cost savings and flexibility that open source brings to AI development benefit small and medium-sized businesses seeking a competitive edge through AI applications.
In summary
To build democratic and open AI, it’s crucial to use the open source principles that enabled cloud computing, the internet, Linux, and so many other powerful, deeply innovative open technologies.
This is the path that Red Hat is following to enable AI and related tools. Everyone should benefit from artificial intelligence development, so everyone should get to help determine and shape its trajectory, and contribute to its development. Collaborative innovation and open source aren’t just essential—they’re unavoidable for the discipline’s future.
