Data is critical to credit union operations. Through data, credit unions can better understand their members, spot irregularities in member behavior, catch fraud, and report other suspicious activities. More recently, credit union data has also been used to train machine learning (ML) AI models. AI pulls data from the credit union well to teach it what to look for and how to respond.
In today’s society, data is an extremely valuable currency. Though cash remains king, data is the princess—and this one is in distress. You see, because of data’s important role in cybersecurity and anti-fraud efforts through AI, cybercriminals have realized they first need to take out or manipulate the AI in order to better execute attacks, and what better way than to taint the well the AI drinks from?
That type of cyberattack is what we call data poisoning.
Your damsel in distress
Data poisoning is the equivalent of an old hag—or, in this case, cybercriminals—feeding your data a poisoned apple. Except, instead of the apple killing the princess until a handsome prince can wake it with true love’s kiss, the poison corrupts the data, which is then fed to the AI, infecting it in turn.
There are a number of incentives to alter the behavior of a credit union’s AI programs. For starters, AI has become a huge tool in cybersecurity over the last few years, with AI being able to detect fraud and stop suspect transfers in their tracks, as well as catch unauthorized accounts, signs of money laundering, etc. Furthermore, AI is being used in key credit union operations such as loan decisioning and processing.
Quite honestly, these days, there is very little at the credit union that AI doesn’t feed into, which makes AI—or, more specifically, the data that feeds the AI—the perfect target of an attack. By manipulating the AI systems of a credit union, cybercriminals can make fraud nearly undetectable, alter loan decisioning, and more. It doesn’t take much either. A little goes a long way when it comes to data poisoning, says VPNRanks, whose research finds that only 7-8% of training data needs to be affected to cause “significant failures.”
So, how are the cybercriminals poisoning your data, and to what end?
Types of poisoning attacks
There are four main methods that cybercriminals will use to poison your data.
Backdoor Attacks
Cybercriminals can create a “backdoor” into the system, leaving an opening for them to later enter through or a vulnerability for them to exploit. Essentially, hackers will add a hidden trigger into the AI that causes it to alter predictions when met with the trigger. This is a very elusive type of attack, as the AI operates in its normal capacity and outwardly shows no signs of being poisoned. The only time it acts incorrectly is when it encounters the specified trigger.
This can support hackers and cybercriminals in a number of activities, such as using triggers to bypass spam filters. It can also be combined with label flipping/clean label attacks (described below) to allow fraud to slip by undetected or alter loan approvals.
Data injection or availability attacks
This is a simple but extremely effective type of data poisoning. Here, the goal is not to manipulate data or AI readings of data, but rather to poison the entire lot of data and corrupt the system as a whole. Unlike other types of data poisoning attacks, which require more finesse to remain undetected, data injection requires no subtlety. Instead, cybercriminals merely inject large volumes of corrupted, irrelevant, or random data into the data set to confuse the model’s training with the goal of making the whole system unreliable, thus creating an operational crisis.
Once the AI is poisoned and the credit union is in panic mode, cybercriminals typically proceed in one of two ways. First, they may demand a ransom in order to reveal which data was poisoned and prevent further poisoning. If they don’t go down that route, they could also take advantage of the chaos and resulting blind spots to access the system and cause further harm, gather account information, or mess with fraud monitoring.
Mislabeling attacks
Also known as label flipping, this type of attack can cause the AI to misunderstand/mislabel the input. For example, in a normal scenario, if a photo of a dog were to be input into the AI, the expected outcome is that the AI would label it as a dog. In a mislabeling attack, cybercriminals would “flip” the label of the input, so that the AI registers the photo of a dog as a cat and reports it as such.
This may seem minor, but in the financial world, this could have a large impact. If a credit union uses AI in its loan decisioning to assess credit scores, label flipping could cause the AI to read a good credit score as “bad” or a bad credit score as “good,” altering loan approvals and potentially violating the Fair Lending Act. Additionally, transactions that might typically be marked as suspicious can be instead labeled as safe.
Clean label attack
Clean-label attacks are the sneakiest and hardest to catch, as they don’t actually change the label of the poison. The data injected carries the correct label (and thus evades detection by human eyes) but still subtly impacts the AI’s behavior and training. This attack takes the longest to succeed, as the poison is slowly affecting the AI over time, rather than all at once, to help the poison stay under the radar. Clean labeling is often used to plant backdoor triggers for later attacks and manipulation.
How to protect the princess
True love’s kiss may have broken the poisoned apple’s spell in Snow White, but we’re not looking for an antidote here. Rather, credit unions need a way to keep the apple out of the equation altogether.
In the fight against data poisoning, your first and strongest defense is to always keep a human in the loop. This is part of any good AI policy, but it becomes particularly important when dealing with data poisoning. If the data has been poisoned, your AI won’t be able to tell (it’s been brainwashed not to know), but a human will. Inaccuracies in datasets, reporting, loan decisioning, fraud detection, etc., can all be caught by a human eye, so ensure there is human oversight of AI operations. In addition to these manual checks, cybersecurity experts also recommend continuous monitoring, detecting, and auditing of AI machine learning.
Furthermore, take the necessary steps to protect your data. This includes following best practices for storing and encrypting, as well as determining access controls. Treat your data like a princess and secure it in a tall tower (maybe with a dragon guarding it). Not just anyone should be able to walk into the tower and access the data. Instead, your credit union should have clear guidelines regarding who can access data and how. Employ the principle of least privilege, or essentially, only allow access to those who truly need it for their job. And even then, access and changes to data should be logged.
Finally, we’ve talked about this all month long, but it bears repeating: education. Your staff may know how to spot an inaccurate report from the AI, but it may not know that’s a sign of data poisoning or know where to follow up. As a reminder, your staff should be continuously trained on your credit union’s cybersecurity protocols, red flags to look out for, and how to respond in case of an incident. Your business continuity plan should be tried and tested.
Start assessing your systems now
Cybercriminals are always seeking improved ways to infiltrate the credit union. New technology is typically a good target for bad actors, as many users are still getting familiar with how it operates and may be blind to anomalies that signal red flags or potential vulnerabilities in the system to exploit.
Thankfully, as we learn more and more about data poisoning and how AI reacts to it, we become better at catching and preventing these incidents. But as we work on beefing up industry-wide recommendations and protocols, don’t neglect setting guidelines and controls for your own credit union.
Because once your princess is poisoned, it will take more than magic to wake her back up.
