by Lisa
It's the last few days before Christmas and so things are flying off the digital shelves. One minute something has 4 left in stock - suddenly it has zero - what's going on?
Some friends mentioned recently that whilst they were both viewing the same hampers on John Lewis, one of them was seeing things as in stock whilst the other was seeing them as out of stock. Coincidentally I then bought some things from John Lewis for the guys at work the next day and was seeing different things at different times.
So how do stock updates work on online shops, and why can it be confusing?
The basics in a simple shop
Scenario A. Every time a page of a website loads on your screen, the website has to make lots of requests to the database where the content for the website is stored. In the case of a simple shop, it asks the database something like "hey, please tell me all the products for this page: their name, price, link to their image, and stock amount remaining". The database replies with the info, including - for sake of argument - that there are 4 left.
The page loads, and you see that information. You then leave that page open on your screen and go and tell the kids to stop shouting / bouncing / being overly excited. You come back and you scroll down a bit, perhaps mouse over the products for a quick view box to appear, but you don't actually click anywhere or leave that page. So that page has been loaded on your screen, without being reloaded, for a few minutes. It still looks exactly the same as when the page loaded for you - and is telling you there are 4 in stock.
In actual fact, those 4 sold whilst you went and spoke to the kids or went to answer the door to Hermes. But your page hasn't been reloaded so the stock is now out of date.
It is possible for your screen to update whilst you're looking it, without you clicking away or refreshing the page, using for example AJAX to speak to the database. But this was just Scenario A, for a very basic shop.
Caching
So if we call AJAX refreshing of the data Scenario B, Scenario C is when there's caching involved.
You might have heard of caching in relation to your browser - when your web browser saves things to it's short term memory to make things quicker to load. Well, websites can do that too.
I mentioned above how web servers can need to make lots of requests to a database when a page is loading. Say you just had 5 requests to make (it can be a LOT more) - but you had 1000 people trying to load that page at the same time. Suddenly you're hammering the database with 5000 requests all at once. One of our websites has, at times, over 70,000 people trying at any one time to access pages. If there were 5 database queries for each person trying to load a page, that'd be 350,000 requests to the database all at once. And that can be a problem for your hosting (to put it mildly!).
So we use caching (and on that site with 70,000 people at any one time, we use lots of other techniques too to manage the database, get in touch if you need a hand managing large scale websites with tonnes of traffic and don't want to have an annoying "join the queue" page). Caching on websites can take many forms. One technique is to have a visitor set a cache when they view the page - so one person visits and the page loads for them, with all the database queries required, and a copy of what they see is effectively copied and stored and it's that flat copy which is then shown to the next visitors. After 5, 10, 30, 60 minutes or however long the cache is set for, the next visitor to visit the website has the page load for them in the "normal" way, and what they see is then copied and stored as a replacement for the previous cache - and the next visitors see what that person saw. And so it goes on.
Sometimes a cache is set by the website knowing to update it's cached (copied and stored version) when a site admin does a certain task, like update a page's content - or perhaps when a product becomes out of stock. You can code it to do whatever you like to suit your site.
But the point is, if you're viewing the website and are seeing the cache - and then someone else visits the website and they happen to see a newer version of the cache and you haven't refreshed your page - you can see different information.
Autoscaling
Scenario D is then when it gets really complicated, and that's once you introduce autoscaling.
Autoscaling is when a website runs over lots and lots of web servers to handle the amount of traffic, and when things get busy, more servers spin up. When things get quiet, the servers die down again. We use AWS when we need a site to autoscale. If you're reading this as a consumer just out of interest you probably won't realise what a HUGE deal Amazon Web Services is in our industry - yes, you know Amazon as the shop with quick delivery, but Amazon power a huge amount of the Internet too.
When a site autoscales and you visit it, you could be actually viewing a different server than your friend down the road who's looking at the same website at the same time. Infact, you could be looking at a different server each time you click to a different page, but that depends on how the autoscaling is set up for that site. And then, depending on how the caching is done, you could be looking at different caches depending on the particular server you're looking at - even if the differences only last for seconds. Suddenly the stock amount being displayed becomes a bit of an estimate.
So I might buy something that doesn't exist?
On a big shop we built last year, that autoscales and that needed to show how much stock is available, we check at the point of you clicking the "buy now" button whether that stock actually exists anymore. If it doesn't, we have to explain politely to the customer that there's now no stock left. Once it's added to a basket, it gets reserved for you for an hour. Sites that do this reservation process can also mean things seem to flicker in and out of availability as people don't check out or abandon their basket and so the stock that was reserved for them goes back into circulation.
At the end of the day though, on a busy site, it's entirely possible for 2 (or more) people to try to buy the exact last thing at the exact same fraction of a second. So busy shops need to have processes in place to check how the stock is attributed to an order and at what point so - ideally - it can tell a shopper before showing them an order confirmation that their item isn't available.
When I placed my John Lewis orders yesterday, I got a couple of emails an hour or so later telling me that a couple of items were out of stock. Whether it takes an hour or so for John Lewis to retrieve this data I don't know - it could be that on an infrastructure as huge as their's the website needs to speak to an external stock management system to know current stock levels, which means information being passed around. Or it might be that the volume of transactions means that checks and subsequent emails go into a queue and so the updates trickle out without overloading the system.
When you order from somewhere like Etsy you might have noticed you get an email acknowledging your order before you get one actually confirming your order has been accepted. This could be for the simple reason that small independent sellers need to confirm manually that they can handle that order, if they're busy or potentially on holiday, or it could be because in a marketplace like that there are external stock management systems in place that need to be checked.
Busy times
2020 has seen a boom in online shopping, especially now at Christmas time. So whilst this is just a whistle stop overview of some of the possible ways stock is displayed on websites, and as such is a little simplistic, hopefully it gives a taster of what goes into keeping a busy shop afloat!
