Arti Raman Forbes Councils Member
Forbes Technology Council COUNCIL POST
Mar 22, 2024,09:45am EDT
Arti Raman is the founder and CEO of Portal26. She is an expert on managing and mitigating risk for enterprise GenAI and data.
There’s no doubt that generative AI (GenAI) is rapidly evolving in the enterprise, with more than 50% of organizations gearing up to implement AI in the coming year. While the surge in AI is both powerful and necessary, it brings forth a new array of vulnerabilities that demand attention. To stay ahead while maintaining infrastructure integrity and combatting unmanaged AI, it becomes paramount for these companies and their security teams to enact governance policies for monitoring employee usage.
There’s no doubt that generative AI (GenAI) is rapidly evolving in the enterprise, with more than 50% of organizations gearing up to implement AI in the coming year. While the surge in AI is both powerful and necessary, it brings forth a new array of vulnerabilities that demand attention. To stay ahead while maintaining infrastructure integrity and combatting unmanaged AI, it becomes paramount for these companies and their security teams to enact governance policies for monitoring employee usage.
Ways AI Is UnmanagedUnmanaged AI refers to GenAI use within organizations that lacks proper oversight, control or governance. You can’t manage what you can’t see.
Unmanaged AI takes various forms. For instance, organizations eager to stay competitive may overlook security considerations in the rush to adopt the technology amid the AI boom. This rapid adoption, without due regard for security, creates vulnerabilities. Moreover, not fully comprehending AI’s potential in the workplace can result in unforeseen security risks, emphasizing the importance of GenAI training and robust company policies.
Unmanaged AI, be it through trends like Bring Your Own AI (BYOAI), shadow AI or intentional and unintentional misuse, exposes organizations to potential attacks and damage to their bottom line. Consequently, addressing the chronic lack of visibility native to many GenAI tools has never been more crucial for organizations.
Intentional Misuse Of GenAI Tools
While there are many ways GenAI tools can be intentionally misused, there are three particularly harmful ways. The first method is AI data poisoning. This occurs when undetectable inaccuracies are intentionally embedded during the model’s training. Even a small amount of bad data can ruin outcomes in a calculation, projection or analysis.
GenAI’s ability to generate large volumes of data also introduces the potential for malicious use of generated data. This can jeopardize sensitive information and critical assets, which can be challenging to recover from in terms of the trust lost between consumers and key stakeholders.
Finally, there is fake content propagation. A complex aspect of GenAI ethics involves the surfacing of deepfakes and the consequent spread of misinformation. Insufficient governance of GenAI inputs due to limited education or standards may lead to deepfake generation. Organizations then bear the responsibility of identifying and countering the spread of false information, necessitating innovative solutions to detect and combat the proliferation of fake content.
Shadow AI
Brand Contributor to Forbes Sharon Maher defines shadow AI as “unsanctioned or ad hoc generative AI use within an organization that’s outside IT governance.” Highlighting the threat, a 2023 survey revealed a significant 93% of respondents are concerned about shadow AI.
Primarily arising from employees seeking AI-based assistance in their roles with tools like ChatGPT, these tools often turn into shadow AI due to a need for more education among users. For example, a recent study found that 6% of employees utilize GenAI by copying and pasting sensitive intellectual property into tools like ChatGPT. Moreover, the survey indicated over 50% of organizations receive fewer than five hours of annual education and training on GenAI issues.
Unintentional Misuse Of GenAI Tools
While there are many ways to misuse GenAI intentionally, there are instances where we may unintentionally and unknowingly deploy AI irresponsibly. One form of unintentional misuse is AI bias. This occurs when systems generate biased results, reflecting and perpetuating historical and current human biases. AI systems learn decision-making from training data, and if data is mislabeled or over- or underrepresents certain groups or characteristics, the model can produce skewed results, eroding trust in the organization.
Similarly, human bias influences AI models as we unconsciously imbue them with our own experiences and biases. Furthermore, the consequences extend beyond mistrust; according to a Scientific American article, individuals interacting with AI models can unconsciously incorporate the distorted data into their decision-making, creating a cycle of mutual amplification of stereotypes and inequalities between humans and AI models.
The BYOAI Dilemma
BYOAI, or Bring Your Own AI, is a relatively new term that refers to employees using any form of mainstream or experimental AI tool to accomplish business tasks whether their organization approves of it or not. These tools could be virtual assistants like Siri or Alexa, AI writing tools like Grammarly or AI transcription tools like Otter.ai.
The key aspect of BYOAI is that the business does not sanction the tool. Because AI use is not slowing down anytime soon, employers need a forward-thinking approach to managing the BYOAI trend. Recent surveys show that 58% of organizations are already managing more than five tools. While BYOAI tools can help overall employee efficiency, they also inadvertently create security risks by adding unmonitored tools.
Undoubtedly, GenAI stands as an excellent tool for enhancing productivity and streamlining processes within organizations. However, when unmanaged, there are significant risks, such as loss of intellectual property (IP) and sensitive data, inadvertent use of false model output and legal and compliance risks. As GenAI use increases in businesses, prioritizing guardrails that focus on security, privacy and legal risks is paramount.
The spectrum of unmanaged AI risks—from shadow AI, BYOAI and intentional and unintentional misuse—underscores the need for holistic organizational visibility and governance. A thorough visibility and governance platform can aid businesses in effectively managing their AI.
Through rich visibility into usage and the ability to create and enforce GenAI governance policies, organizations can defend against compliance, privacy, and IP-related risks. However, to ensure effectiveness, GenAI ethics and governance methods must be fully integrated and accessible across the entire enterprise.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives.
healthcarereimagined 