Nvidia Debuts Open-Source Tool That Trains AI To Stop Spouting Malicious Remarks

[Firewaller]

Artificial intelligence is a rapidly expanding field with potential to improve lives. However, with this possibility comes a significant risk of errors and malicious attacks, especially regarding chatbots. Fortunately, Nvidia has devised a solution to this problem: ‘NeMo Guardrails’.

NeMo Guardrails is a set of guidelines to prevent chatbots from spewing misinformation, toxic comments, and racist remarks. In addition, developers from various organizations can train their language learning models (LLMs) to follow specific rules and safety measures. This ensures that the machines stick to their intended purpose, remain secure, and do not produce inappropriate content.

One of the critical features of NeMo Guardrails is that it enables companies to set security limits, ensuring their LLMs use accurate information when responding to users. Moreover, the safety rules prevent LLMs from making up answers to questions, avoiding toxic responses and malicious codes. For example, customer service chatbots will decline to answer questions about the weather to avoid going off-topic. These guidelines will also prevent humans from hacking into the machine and trying to steer it into making racist or hostile comments.

Read full article

View the full article