‘Failed miserably’: Two lawyers who used ChatGPT in a federal court filing that cited ‘bogus’ cases are now facing serious punishment. Here are some big risks of using AI in the workplace

2. Potential for bias in AI decision making

It’s possible for human biases to be baked into AI models, especially if the underlying training data is biased. These tools were created by humans, after all.

One example of this came to light in 2018 when Reuters revealed that Amazon had scrapped a secret AI recruiting tool that was found to be biased heavily in favor of hiring men over women to fill tech jobs. The company’s AI model was apparently trained to vet applicants based on patterns in resumes submitted to the company over a decade-long period. Most resumes came from men — a reflection of the male-dominated tech industry — which led to Amazon’s system teaching itself male candidates were preferable.

There have been a number of instances of human bias on display in AI, and with that in mind professionals should be cautious about using AI platforms that are not transparent about where the training data was sourced from or how the models are structured.

3. Hallucinations problem

Large Language Models, the breakthrough technology powering AI platforms like ChatGPT, are designed to predict the next word or phrase after a series of words in context. These systems, however, are typically geared toward mimicking natural language and are not necessarily designed to be accurate, which at times has resulted in questionable content. Just ask our two lawyer friends at the beginning of this article.

This issue is so common that AI researchers have even coined a term for it: AI hallucinations. The problem has also created legal trouble for OpenAI. The firm was hit with a defamation lawsuit from a Georgia radio host who says the ChatGPT platform falsely claimed he embezzled money.