How Bots Ruined the PlayStation 5 Launch for Millions of Gamers

Background

The PlayStation 5 was released by Sony on the 12th of November 2020 and quickly became the must have item for millions of gamers and parents around the world.¹ Due to supply chain issues, Sony was unable to supply sufficient quantities of the consoles to satiate global demand resulting in a perfect storm for reseller bots.² Reseller bots scoured the internet acquiring inventory wherever it could be found and reselling the elusive PS5 at exorbitant prices to desperate buyers on the secondary market. F5 protected one such large US ecommerce retailer that was being targeted by PS5 reseller bots. This article is a case study covering the PS5 sale event for a large US ecommerce site and via their native mobile application. This article will review the approaches taken by reseller bots to avoid detection and maximize acquired inventory. We also briefly cover how reseller bots advertised and sold the acquired PS5s.

Overview

The case study will focus on the process of bots attempting to add PS5 stock to their carts for a specific large US ecommerce company’s web and mobile applications for a two month period between January 1^st and March 1^st 2022. Sony’s PS5 was released for sale on January 11^th, hence the period before this date shall be referred to as the reconnaissance period and the period thereafter the sale event period.

During the reconnaissance period there were low levels of automated activity associated with resellers testing and configuring their bots in preparation for the January 11^th sale (see Figure 1). Once the sale starts, the sale event period is categorized by a drastic spike in automation against the “add to cart” flow within the first few minutes of the sale as reseller bots scramble to acquire as much inventory as possible while stocks last. This spike was 200x normal human traffic volumes peaking at over 27 million requests per hour. In typical scenarios reseller bot activity would die down within a few hours once all inventory had sold out. However, due to successful anti-bot controls, the inventory lasted for a number of days. Resellers attempted to retool their bots to acquire the PS5s they could see were still listed. Examining the retooling process gives application defenders a lot of insight into the technical motivation, capabilities, and resourcefulness of the resellers as they attempt to circumvent anti-bot defenses.

Reconnaissance

Prior to the PS5 sale event on January 11^th, reseller bots were active on the ecommerce retail company’s web and mobile applications. The resellers needed to test their bots and ensure that they are able to bypass any anti-bot defenses that may have been put in place, allowing for seamless operation during the sale event. The bots targeted obscure items for their test cases, buying tens of units of small ticket items costing just a few dollars ranging from plush kids toys to microfiber cleaning cloths.

The ”add to cart” step is considered one of the most important steps in the inventory acquisition process as most retailers will temporarily take items out of inventory as soon as they are added to a cart. This is the process that reseller bots will test and optimize the most. In order to beat other reseller bots they use some of the following tricks:

1. Bots do not navigate websites as we would do by browsing for and clicking ‘add to cart’ buttons. Instead, they craft an HTTP request to add the required item directly to the cart. This removes unnecessary steps in the process and saves valuable time.
2. Bots supply all the product specs in the initial ‘add to cart’ request. For instance, they ensure that things like color, size, and optional configurations such as hard drive size are specified in the first request so as to save time and ensure the item can be added to the cart in one go.
3. Bot users will pre-authenticate before the sale event so as to avoid having to do that once the sale starts, as this additional step could waste precious moments.
4. Finally, resellers will pre-run the bots to ensure they have time to deal with and solve any anti-bot speed bumps, such as multi-factor authentication (MFA) and CAPTCHA, that the retailer may have in place. By solving CAPTCHA or other challenges just before the sale starts, this will ensure that the bots will not be challenged again once the sale starts and increase the chances of acquiring the inventory needed.

Because there is no monetary cost associated with adding items to cart, this is the part of the process that resellers will test the most to ensure that they can add multiple items without getting flagged by security controls. Other steps of the checkout process, like submitting payment details and completing the order, do come with a cost (though unwanted items can be returned for full refunds) and are not tested to the same extent. The checkout flow, however, has to be tested fully to ensure that payment methods work and that any per-account, card or delivery address limits are discovered and circumvented before the sale event. The presence and extent of any post-sale analytics is also tested to ensure that resellers can run their bots and buy the required items without any issues.

The Launch of PlayStation 5

Reseller bots will continuously check ecommerce web and mobile applications for the target inventory. This is to ensure that they are the first to know once the products are available for sale. Though ecommerce retailers may advertise when a product will go on sale, the real sale may start earlier or later than actually advertised. This is a risk a resellers cannot take, and they will start monitoring the sites earlier than the announced sale launch time.

Once the sale starts, bots will try and acquire as much inventory as they can while stocks and funds last. Resellers will scale up their network of bots for peak throughput using a wide network of proxy IP addresses, payment providers, shipping address and user accounts. In this particular case involving the sale of PS5, the volume of automated “add to cart” transactions exploded from an average of about 15K requests per-hour during the reconnaissance stage to over 27M requests per hour during the first moments of the sale event. During the first hour of the sale event, there were 11M automated “add to cart” requests from automated reseller bots and only 88K from legitimate customers. That is to say 99.2% of all purchase requests for the PS5 actually came from reseller bots within the first hour of the sale. This gives a sense of the scale of reseller bot activity for high demand product sales.

Expanding the Opportunity

Resellers quickly realized that the PS5 inventory had been listed on the ecommerce site under different stock keeping unit codes (SKUs). These different SKUs included some product bundling by the retailer to maximize revenue by combining the PlayStation console itself with other products. These included, but were not limited to, bundling the PS5 with video games, monitors, controllers, or subscriptions to online gaming platforms. Resellers quickly searched for all available SKUs that contained the PS5s and targeted those as well. For the retailer in question, there were over 100 different SKUs that contained the PS5 console during this event and all of them were identified and targeted by reseller bots.

Retooling

Once resellers tried to scale up their attack, they quickly realized that they were unable to add any of the PS5 SKUs to their cart even though the product was still available and in stock. This told them that their bots were being mitigated by the retailer’s anti-bot defenses. To counter this, resellers tried to retool their bots quickly in the hopes of bypassing the defenses before inventory had all been sold out to legit customers. Traffic logs for this retailer show a number of different reseller bots and how each of them reacted upon realizing they were being mitigated:

Low Sophistication Bots

The low sophistication reseller bots either gave up immediately, made minimal changes, or kept hammering away hoping something would give and they could add items to their cart. The approach taken likely varies depending on the level of experience, sophistication and motivation of the reseller. A small number of the low sophistication resellers tried to change their IP infrastructure in an attempt to bypass defenses by using one or more of the following tactics:

1. Increasing the number of source IP addresses to circumvent IP based rate limits
2. Using more residential IPs to circumvent any ASN-based block lists
3. Using US-based (and less international) IPs to circumvent potential geo-blocking
4. Changing user-agent strings

Medium Sophistication Bots

Initially it is hard to know whether a particular reseller bot is sophisticated or not. There is therefore some overlap in the retooling process followed by low, medium and high sophistication reseller bots. The difference is at what point the reseller bots stop retooling and give up. Medium sophistication bots will retool by following a process that typically goes as follows:

Change or Diversify IP and ASN Infrastructure

Bots often assume that they are being blocked based on their IP address, so the first step for many is to change IP. Initially throttling the volume per IP, then changing ASNs, and finally their geolocation (country) in case geo-blocking is being used. When this fails the bots will try and change class of ASNs. Initially they may use hosting or international ASNs and when this fails they move towards ASNs in the same country as the target and start to use large numbers residential IPs with low transaction volumes per IP.

Change or Diversify User Agent Strings

If randomizing ASNs and IPs does not succeed, the next thing resellers will change is the user-agent (UA) header. However, headers such as user-agent can be used to track, rate limit, and block bots. Therefore, resellers will generate a large number of either real or fake UA strings and use each for very few transactions in the hope that this will bypass the mitigation controls.

Delete or Refresh Cookies and Other Trackers

When changing IPs, ASNs and UAs fails the sophisticated actor realizes that there must be something else they are being tracked and mitigated on. Cookies becomes the next thing they look at. They will either reject or delete cookies after every transactions or run their bot in incognito mode.

Manually Try to Add to Cart and Capture HTTP Transaction, and Perform a Diff With Bot Transaction

At this point when nothing has worked, the reseller manually tests the site to ensure that the site is actually responsive and not offline. A successful manual test tells the reseller that the site is working and the issue is that their bot is somehow being detected. The sophisticated actor will then capture and inspect the successful manual transaction and look for differences between that request and the one generated by the bot.

Copy and Replay Cookies or Tokens from Successful Manual Transaction

Having identified some differences in cookies, tokens, encrypted data packets or other request parameters, the reseller will simply copy these items from the successful manual transaction and attach them to the bot requests.

Randomize Cookies and Tokens

When this fails to yield the required results, the reseller will try and randomize these new cookies, tokens or parameters in the hope that this will bypass any rate limits placed on these.

Try to Execute All JS on Page to Authentically Generate Cookies and Tokens for Each Transaction

When all else fails, the reseller will conclude that cookies, tokens or other parameters might be digitally signed hence need to be generated by the website every time. To this end they will allow their bot to run all JS and other scripts on the page to try and authentically generate the missing cookies, tokens or other parameters.

High Sophistication

The most sophisticated and motivated reseller bots noticed immediately that they were being blocked and moved to retool by:

1. Randomizing headers to prevent HTTP header-based detection and blocks
2. Attempting to target the PS5 bundles rather than the PS5 console units in the hope that security was lower around those
3. Attempting security token abuse by performing an add-to-cart transaction manually, farming out all the security related tokens and trying to replay them using their automated bots
4. Using fingerprint randomization technology to prevent transaction association which would show that all the add-to-cart requests were coming from the same devices
5. Moving from Web to Mobile or vice versa to try and see if controls might be laxer on the other platform
6. Uninstalling and reinstalling the ecommerce retailer’s mobile application to ensure they are on the latest version and have all the correct signals the site is expecting
7. Inspecting the ecommerce web page for changes including new security scripts or technologies that might be responsible for their mitigation by manually loading and inspecting the page and testing if manual transactions are succeeding
8. Spinning up a manual click farm to purchase the required PS5 inventory

At some point even the sophisticated attackers gave up either due to frustration at being blocked or once the PS5 inventory started running out.

Monetization

The monetization process for the resellers was pretty straightforward. All over the world demand for PS5s was super high and there was no shortage of willing buyers. Resellers would list their products for sale on secondary marketplaces like StockX and eBay (see Table 1).
 

The table above gives some insight into the unit economics for resellers. Many resellers made a gross profit of between $150-$200 per PS5 console unit. This translates to a 30-67% gross margin business with a turnaround time of just a few weeks. This is an extremely lucrative business, which is why the reseller bot problem is unlikely to go away and will, in all probability, worsen into the future.

Conclusion

This case study of PS5 reseller bot activity is based on a large ecommerce company that had put in place robust anti-bot protection. It therefore gives a view into how reseller bots deal with being blocked and how they try to circumvent existing defenses. For privacy reasons we are unable to go into more in-depth specifics around this activity, but hope that this case study helps to shed more light on reseller bots. A future case study will cover shoe bots on an ecommerce site without any anti-bot controls and will give some insight into what reseller bots do when they are not being actively blocked.

Recommendations

In order to protect themselves against reseller bots, enterprises have a number of security controls to choose from. Our previous article, Reseller Bots: Defining the Problem breaks down the various methods, controls, and their effectiveness, in combating automated threats such as reseller bots.