It is challenging to envision any contemporary computer system that has not been enhanced by the prowess of artificial intelligence (AI). For instance, when you capture an image with your smartphone camera, over twenty deep learning (DL) models come into play, performing tasks from object recognition to depth perception, all collaborating to assist you in capturing that flawless photo!

Business operations, efficiency applications, and user interactions can all benefit from incorporating some kind of AI, and few other technologies have expanded with the same magnitude, pace, and influence. However, like any other technological innovation, AI comes with its unique set of hazards, which in this situation encompass security and safety concerns, and potentially legal responsibilities. In this piece, we will briefly explore some of these safety and security issues, particularly related to generative AI (gen AI), and how we can cultivate more dependable, secure, and trustworthy AI systems.

Distinguishing Between Security and Safety

Similar to any computer system (hardware or software), AI systems can be exploited for malicious intents, such as unlocking devices, injecting commands, engaging in adversarial training, and other activities. However, AI systems introduce a new dimension to the scenario—the idea of the security of the output data. This is predominantly due to the following reasons:

• AI outputs are frequently produced based on previous training of the model, and the effectiveness of the output hinges on the quality of the data utilized during training. Established models boast about leveraging a vast amount of data, often quantified by the number of tokens employed for training the model. The premise is that the more tokens utilized, the more efficient the model’s training
• The outcomes from the model could be utilized to support business, user, and technical decisions. This poses financial risks as well as potential safety and legal consequences. For instance, there is an abundance of vulnerable code on the web, so any model trained on such data faces the risk of generating insecure code. If this generated code is directly implemented in a software project, it could trigger an entirely new form of supply chain attack

While some aspects of AI security and safety are intertwined, most safety frameworks tend to address them separately. Safety protocols for computers are a relatively recent concept for most organizations, and we are still grappling with their implications.

Safety Factors When Utilizing AI Models

In essence, gen AI models function by predicting the subsequent word in a sentence. Although these models have evolved to be considerably more sophisticated, they fundamentally operate on this principle. This implies there are various intriguing elements to contemplate when discussing AI safety.

Input Determines Output

The principle of “garbage in, garbage out” is a foundational computing principle that still applies to AI models, albeit in a slightly different context. A gen AI model “learns” from a particular set of data during its training phase. Typically, this training phase is bifurcated. The initial phase involves pre-training, where a vast dataset is employed, frequently sourced from the internet. The subsequent phase is fine-tuning, where data pertinent to the model’s objective is utilized to enhance the model’s proficiency in a more targeted task or set of tasks. Some models might undergo more than two phases, based on the model’s structure and purpose.

As anticipated, training a model on unfiltered data obtained in bulk from the internet—devoid of considerations for sensitive, unsafe, and offensive content—can yield unforeseen and unfavorable outcomes.

Models Fabricate Scenarios

I frequently draw parallels between AI models and young children. When children are unfamiliar with the answer to a question, they tend to invent an entirely fictitious, albeit compelling narrative. Models exhibit similar behavior in numerous aspects, but the consequences can be more perilous or detrimental, particularly when models generate responses carrying financial, social, or security ramifications.

Evaluating Safety and Setting Standards

Although the AI sector is still in its preliminary phases, there have been suggestions for benchmarking standards that we find intriguing and worth monitoring:

Establishing Boundaries

Boundary-setting tools and models adopt various strategies to ensure that a model’s output aligns with the predefined safety and security criteria. Several open-source tools and initiatives are available to establish these boundaries. However, a boundary-setting mechanism is essentially another software component and therefore carries its own risks and constraints. It is the responsibility of model developers to institute mechanisms for evaluating and benchmarking the detrimental impacts of their models before operational deployment.

The Significance of Open Source

While the industry is deliberating the definition of an open-source model for AI and the characteristics it should possess, IBM and Red Hat are at the forefront, implementing open standards and open data for the AI models they distribute. This encompasses:

Red Hat is also a founding member of the AI Alliance, a collaborative network comprising companies, startups, universities, research institutions, government bodies, and nonprofit organizations at the forefront of AI technology, applications, and governance. As part of this alliance, efforts are underway to establish a genuinely open, safer, and more secure AI landscape—not just for customers but for the wider open-source community.

In Conclusion

Artificial intelligence is still in its infancy, and it is imperative for us to consider its security and safety now, rather than attempting to retrofit these aspects at later stages. Red Hat believes that in the realm of AI development, open-source platforms and open systems can yield a profound impact.

Discover more about RHEL AI