Introduction
Much of the activity on the internet is automated, and quite a lot of it is specifically due to bots. Bots can be used for many purposes, but in this series of articles we’ll be looking at bots that create and use fake accounts. This first article looks at the motivations behind fake account creation; part 2 will look at the specific techniques used by fake account creation bots.
What are Fake Accounts?
Online systems are designed with specific purposes in mind and, for many, user accounts are an important means of enabling key functions. An average person's list of online accounts provides them with a diverse set of digital capabilities: management of financial accounts, buying and selling goods online, meeting other people and even falling in love. A fake account is one that is either:
- Differs from the intended purpose of the online system. These purposes may be malicious or even criminal, while other uses are more benign.
- Uses fake, stolen or intentionally inaccurate information that does not map to a correctly identifiable individual or entity.
Is it Illegal to Create Fake Accounts?
This is a complex question best answered by legal experts. In many cases fake accounts in general are not illegal, though they might go against the terms and conditions of the platform in question. There may be restrictions such as limiting each user to a single account or requiring accounts to be linked to real life identities.
In specific highly regulated industries like Financial Services, platforms are subject to Know Your Customer (KYC) laws to ensure that all accounts are linked to a real and verified individual. Creating a fake account in such instances would be illegal and therefore, the process of creating an account is onerous and involves verification of user identities and PII (Personally identifiable information).
This is not the case for most online systems where only an email address or phone number is required to create an account. In these cases, using fake or incorrect information to create an account may not be illegal. However, if stolen information is used to create the fake account, then that may have legal ramifications, depending on how this stolen information was acquired and whether the account creator knew the data was stolen.
How Big of a Problem are Fake Accounts?
Fake accounts are a pervasive problem for almost every online platform. During Elon Musk’s well-publicized takeover bid for Twitter, fake accounts were at the core of the legal back and forth that ensued. The monetary value of a social platform like Twitter is driven by the number of people using the platform, because this is how advertisers evaluate the attractiveness of buying ads.. Advertisers want to reach real people that can buy their goods and services, so they care about whether the ads they pay for are being seen by real people or by bots. It was estimated by F5 cyber security expert Dan Woods that more than 80% of twitter accounts were fake. This would materially impact the value of Twitter and how much Elon Musk was willing to pay for the company, to the tune of tens of billions of dollars.
Not only do fake accounts impact website analytics and monetization, but they have been cited as a central tool in major fraud cases1, organized disinformation campaigns2, and numerous other activities. Fake accounts can be powerful tools, especially in online ecosystems where accounts are intended to serve as proxy identities of real users.
Why Create Fake Accounts?
Let us turn our attention to motivation. Why would anyone want to create fake accounts? There are numerous reasons why one would be incentivized to do so. We have split these reasons into malicious and benign reasons.
Malicious Purposes
Criminals and fraudsters need to protect their identities and will often go to great lengths to obfuscate their true identities both in real life and online. As a result, online fraud will necessitate the need for fake identities and accounts that are not traceable back to the criminals.
Fraud
A major use case for fake accounts is to hide the identity of an online fraudster so they can commit various forms of fraud such as the following.
Romance Scams
Fake accounts are used by fraudsters on online dating platforms using fake, stolen, or edited photos and profile information. These accounts are then used to chat with and lure unsuspecting victims to fall in love with the persona played by the fraudster. The fraudster will then use this loving relationship as a basis to extort money from the victim. This is like what was depicted in the Netflix documentary “The Tinder Swindler”. Online dating is a game of large numbers. Automation is used to create a large number of fake accounts, and bots are used to manage the interactions with the victims. These bots will also notify the fraudster of potential victims that have reached a level that will require special manual attention from the fraudster to move the fraud along.
Money Laundering
Fake bank accounts in the name of a fake individual or company are used to facilitate money laundering and the movement of illicit funds. These are created using fake, synthetic, or stolen identities that cannot be traced back to the criminals.
Credential Stuffing/Account Takeover
Fake accounts called "Canary Accounts" are used by attackers engaged in credential stuffing and account takeover operations. These are accounts that are either created by or controlled (previously compromised and taken over) by the attacker. Many security systems will flag low login success rates as indicative of credential stuffing and account takeover activity.
A fraudster must therefore find ways to artificially inflate their login success rate above a given threshold to prevent detection. Because the attacker knows the correct username and password for their canary accounts, they can use them to guarantee a minimum login success rate. For example, if they wanted at least a 50% login success rate, they would log in once into a canary account (guaranteed success) then once into an account from the compromised list they are testing, then again into another canary account (guaranteed success) then another test account. This way they will not be flagged for low login success rates.
These canary accounts also serve a second purpose, like the canary in the proverbial coal mine, to warn attackers when their attack has been detected. Security teams block credential stuffing attacks by returning a failed login response. If an attacker is using canary accounts where they know the credentials are valid, and they receive a failed login response, they will immediately know that they have been discovered and are being mitigated.
Data Scraping
Many platforms have valuable data that can be scraped and sold for a profit. To minimize this, many platforms will require users to be authenticated to access the data and will set up rate limits on the number of searches an individual account can run.
To get around this, attackers will create large numbers of fake accounts. Then using these fake accounts, they will coordinate and distribute the required search load so that each account is below the rate limit but allowing all available data to be scraped from the system. An example of this is insurance rate scraping, which is done by getting a quote for every age, gender, zip code, car type, make and model in the country. These are millions of different combinations which can only be done using automation and large numbers of accounts.
System Vulnerability Exploitation
Fake accounts can also be used to exploit system vulnerabilities that exist due to the interaction of a group of two or more accounts. To exploit such a vulnerability, the attacker would need more than one account in a system.
An example we can cite was a vulnerability at a large bank that allowed an attacker to log into two accounts simultaneously and use automation to send $1 at high velocity between the two accounts. The system had a vulnerability that at very high speeds it would add the next $1 before it deducted the last one and essentially create money that didn't exist before in both accounts. This is an example of the sorts of vulnerabilities that can be exploited using coordinated activity between two or more fake accounts.
For Influence
An attacker creates a large number of accounts to influence online systems that rely on activity trends and popularity. This is one of the main reasons that fake social media accounts are created. Below are some examples of how fake accounts are created to advance influence.
Disinformation
A user with a large number of fake accounts can use them to amplify specific content including misinformation. This is what is seen around general elections in many countries. The fake accounts will coordinate to post, repost, like, comment, and react to the attacker's desired content so that the social media algorithms pick up the content and show it to more users. This amplifies the reach of this content and ensures that more people see it.
Social Media Monetization
A user with tens of thousands of fake social media profiles can sell followers to would-be social media influencers that want to grow their follower count and increase their monetization. One needs to have a specific minimum number of followers, content views, likes, and other interactions to make money on many platforms. The algorithms also only recommend their users follow accounts that their friends follow or that are trending and popular. This causes a chicken and egg problem for new users who need to have lots of followers in order to get more, so they are tempted to buy the first few tens of thousands of followers to seed their growth on these platforms.
Fake Reviews
Reviews are a major part of how people make purchasing decisions online. To make one’s product sell better online, it helps to have a large number of stellar reviews. Users with thousands of fake accounts can sell product reviews and offer to provide thousands of glowing reviews for a product, establishment, or service for a fee. They achieve this by coordinating their army of fake accounts to submit lots of reviews. This army is not just used to provide positive reviews for one’s product, establishment or service, but can also be used to provide negative reviews for the competition or organizations and individuals who the customer wishes to target.
Online Polls and Surveys
A user with large numbers of fake accounts can skew data collected from online polls and surveys to their advantage. This can be anything from election opinion polls, proposed government policies that are open for comments, online polls for reality shows, and other online competitions decided by public votes. People can buy tens of thousands of votes for their cause.
Referral Fraud
Many companies, especially up and coming startups, offer some form of a referral program. This is to encourage their customers to refer their friends and family in return for some form of financial incentive. This is seen as a cost-effective way of acquiring new customers that tend to have higher lifetime value than those acquired through online ads. Due to the poor design of some of these referral programs, they create an incentive for attackers to create lots of fake accounts which can refer each other and allow the attacker to get paid for doing so. Many companies have wised up to this and try to design their referral programs to reduce this risk. However, attackers are innovative and can still find ways to manipulate some of these programs resulting in payouts of thousands of dollars for fake users.
For Incentive Manipulation
This is when users create a large number of fake accounts for the purposes of manipulating existing incentives on a given platform. This is done to either get unearned benefits or to multiply any benefits earned. Below are a few examples of this:
Birthday Incentive Manipulation
Many fast and quick service food restaurants as well as other companies offer birthday incentives. This is when you get a free or discounted item(s) if it is your birthday. Because a user has one birthday a year, these companies do not bother to confirm when your birthday actually is as the once-a-year cost is manageable irrespective of when your birthday is. Attackers can manipulate such a system by creating 365 fake accounts with each account having a different birthday, to cover the entire year. They would then use automation to log into the account whose birthday it was on any given day and attach a given gift card to that account. This would entitle the holder of that gift card to the free or discounted item(s) on that day. Because every day the gift card is attached to an account whose birthday it is, this gift card becomes what is called a “black card” that entitles you to free items every day. These cards are then sold for a profit by the attacker for hundreds of dollars.
Bonus Manipulation
Many online platforms e.g., ride sharing companies offer drivers incentives to hit specific rider targets over a period of time. The benefits of hitting these targets may be higher retention rates of their fares or monetary bonuses. Due to the attractiveness of these financial incentives, some drivers that are falling short of the target may be incentivized to record fake/fraudulent rides or sales (on sales platforms). Users with large numbers of fake accounts can sell the missing rides/sales to allow the person to hit their target and unlock the financial incentives. This makes sense since the value of the incentive is often larger than the gap needed to unlock them. The driver would struggle to find people to pay $50 to take 5 $10 rides, but users with lots of fake ride share accounts would happily take $70 payment for example, book the $50 worth of fake rides (keeping $20 as profit for themselves) if this unlocks a $500 bonus for the driver for example. Most ride share companies have put in place measures to prevent this kind of fraud but many online platforms with target-based incentives are still susceptible to this type of manipulation.
For Unfair Marketplace Advantage
Having a large number of fake accounts can be used to gain an unfair advantage on many online marketplaces. This unfair advantage can be used to make outsized returns for the actor. Below are some examples of such market manipulation.
Online Gambling Cheating
Having fake accounts on online gambling sites can give a user an unfair advantage in being able to coordinate actions that result in an unfair advantage including odds manipulation by placing coordinated bets, controlling more than one player at a poker or other online gaming table which allows an unfair advantage in the game.
Marketplace Power
On marketplaces such as freelancing sites, a service provider is incentivized to create multiple fake accounts that they control. This allows them to increase their visibility on the platform and get access to a wider range of jobs. Each account will be optimized for specific uses e.g. For platforms that have geo limits on who can bid for projects, the service provider will have an account in every location available to ensure they rank as the nearest provider in every location, for example. This gives them an unfair advantage, allowing them to get disproportionately more work than competitors. They can also use multiple fake accounts to engage in collusion and price manipulation by setting the average rates to their advantage. They can also pretend to bid against themselves for a job in a way that ensures there is no real competition, and they end up getting higher rates than they would if they really negotiated against a competitor.
Reservations for Sale
Reservations, tickets, and the sale of other high value items like limited edition sneakers can be manipulated by fake accounts. Fake accounts can be used by ticket reseller bots or sneaker bots to buy large numbers of tickets/sneakers that are resold on the secondary market for a profit as highlighted in our recent reseller bots article. High demand reservations for popular restaurants, public facilities like tennis courts, visa and passport appointments etc. are susceptible to manipulation by bots using fake accounts. Fake accounts are created in large numbers using automation. These accounts are then coordinated using automation to snatch up all available reservations as soon as they become available. These reservations are then sold on the secondary market to desperate buyers. Reservation systems do not typically allow a single user to make multiple reservations, this is why large numbers of fake accounts are needed to do so. This unfortunately has many disadvantages for the other users and the service providers which we will cover later when we talk about the downsides of fake accounts and why you should care.
Raffle/Lottery Rigging
Retailers and other companies that sell high demand-limited supply items like electronic gadgets (e.g. PS5, Nintendo Switch, Crypto mining GPUs etc.) and sneakers, have a problem with reseller and sneaker bots buying up all the inventory before legitimate users can get their hands on them. To prevent these high-speed bots buying up all the inventory within seconds of items going on sale, retailers often use raffles/lotteries to decide who gets to purchase these limited items. This theoretically is to give legitimate humans a fair shot by ensuring that items are not bought by whomever can complete the checkout process the fastest, as this tends to be reseller and sneaker bots. Instead, over a period of time, the retailer will allow users to express interest in buying the item and entering a raffle/lottery. A limited number of applicants will then be selected to purchase the items based on a random selection process. To ensure that they get the sale items, reseller and sneaker bots will create tens of thousands of accounts and enter all of them into the lottery for the sale items. This increases their odds and ensures that a large chunk of the sale inventory ends up going to one of their many fake accounts.
Benign Purposes
There are also benign reasons why a user may want to create fake accounts. Typically, these benign fake accounts are created manually and not using automation, though at times automation can be used.
Kids Avoiding Parents on Social Media
Teenagers typically do not want their parents to know what they do online especially on social media. To avoid parental oversight, they will typically have a normal profile with their given name that their parents and family can follow and monitor, this is called the Rinsta (real Instagram) account. They will then create a second account (FInsta or fake Instagram) which uses their nickname or other handle. This account is only shared with their friends and peers and kept secret from their parents.
Parents Monitoring Kids With Fake Profiles
Parents realizing that it is hard to keep track of their kids online social media activities due to the rise of Finsta and other fake accounts created by their kids, decided to also create fake accounts of their own to follow and track their children’s fake accounts. These accounts are typically created using the fake profile of an age-appropriate peer that they believe their child will want to connect with online.
Catfishing
Catfishing has been defined as “to deceive (someone) by creating a false personal profile online.”3 Users will create catfishing accounts for a number of different reasons including the parent monitoring noted above. On online dating platforms these accounts can be created by fraudsters as covered in the malicious fake accounts section, or they can be created by people that just want to experience what life would be like and how they would be treated if people thought they looked a particular way. These accounts are not for the purposes of defrauding anyone but to get attention and companionship online.
Anonymity
There are many people on the internet that want to express themselves without consequences to their personal, professional, or other reputation. As a result, many people will create fake profiles using pseudonymous information and imagery that is not linked to their real-world identity. This includes individuals with legitimate privacy and safety concerns, but also parody accounts on which one individual claims to be another, typically well-known person using their name and likeness. The rise of these kinds of accounts is one reason why social media platforms introduced verifications to tell which account belonged to the real person and which ones were fake or parody accounts.
Conclusion
Fake accounts are a growing problem for online platforms, with cybersecurity expert Dan Woods estimating about 80% of social media and online dating user accounts to be fake4 and in many cases controlled by bots. There are both benign and malicious reasons for the creation of fake accounts. Benign fake accounts tend to be created manually, while automation is used to drive malicious fake account creation and usage. Benign fake account creation is mainly driven by the desire for privacy and anonymity, while malicious fake account creation is driven by financial objectives using fraudulent means.
In the next article of this series, we are going to dig deeper into why automation is essential to commit fraud and monetize fake accounts at scale. We will also cover the impact these accounts have and how they cost platforms and their users large amounts of money each year. Time will also be allocated in the next article to explore why some businesses do not care about the existence of fake accounts on their platforms and are unwilling to take steps to mitigate them.
Recommendations
For any application on account info, an analysis of how this could be abused ought to be performed. The above material could be used as a template for this analysis.
In future articles in this series, we’ll look deeply into the technical means that attackers use to create and use fake accounts, and provide recommendations for countermeasures.
