On Indie Hackers, I often people writing about their concerns of their site being able to handle the sudden burst of traffic, they're hoping to get from a Product Hunt launch. Many times, this is an over-blown fear, but can also be a rational concern.
We've all seen sites submitted to Reddit, crumble under the traffic. I can only imagine how frustrating it can be to suddenly be getting exposure like never before for a new project and then see the opportunity lost due to technical difficulties!
Not only that but popular, fast-growing services often have regularly recurring outages and partial outages. Twitter used to be famous for the fail whale. Perhaps IH users are particularly concerned because IH itself had a rough year while scaling quickly. There were times when it took me 30 seconds to load a non-cached page on a 100Mbit connection.
# The surprise bill
There's another kind of scalability disaster to watch out for. How much will it cost you if your server setup doesn't crash under a heavy load of traffic? The internet is replete with examples:
- Firebase Costs Increased by 7,000%!
- I was billed for 14k USD on Amazon Web Services 😱
- How we spent 30k USD in Firebase in less than 72 hours
- We Burnt $72K testing Firebase + Cloud Run and almost went Bankrupt
# Why I care about this topic
I may be particularly concerned about not just scaling but the costs of scaling for historical reasons.
# My first job
My first software job was at a company in Beijing that was, for a time, the leading platform for educational iOS apps. Basically, we provided an API that 3rd party devs could integrate with to get some additional features and marketing benefits. This meant both tight margins and a lot of people using a lot of apps, all hammering our servers. Being a very junior employee, I was insulated from this but the discussion came up repeatedly in our office.
# The startup that didn't quite make it
Later, in 2017, I was working on a free-to-use app that included video chat. Most of its users were in SE Asia. I was using WebRTC, which is generally peer to peer and free, but some percentage of machines can't establish a direct connection. Of that group, some can communicate establishing a connection with the help of a STUN server but others need a TURN server to relay the entirety of their conversation. This last case is comparatively expensive and consumes quite a bit of bandwidth.
At that time Twilio had a new WebRTC video chat service with very reasonable costs. They charged per API call, as well as a fee for total bandwidth consumed by proxying media via a TURN server. In order to save time, I went with Twilio and got to building.
When I was already well underway building my platform, Twilio increased their cost per API call by 100x and added a per minute fee to all video chats. This meant that I'd be paying Twilio by the minute for users who didn't need TURN, were communicating entirely P2P and not paying me anything.
I was very lucky not to have much traction yet.
# How much will it cost to host your site?
There are a lot of different options at different levels of abstraction. I generally think of hosting options as falling into one of three buckets—a managed PaaS, a cloud VPS or dedicated servers. A managed PaaS is by far the most expensive (and most convenient), while dedicated servers are much less expensive but completely DYI. Cloud VPSes tend to be somewhere in the middle.
Amazon spans from a simple VPS if you're using EC2, to something closer in cost to a managed PaaS if you're using managed databases, autoscaling and other services.
# Just egress bandwidth costs can be considerable
One cost that few apps can escape is bandwidth. Bandwidth is also one of the highest-margin fees on many popular hosts. Most web servers aren't CPU or even memory bound. Game servers are hungry for both, but still tend to consume so much bandwidth that mainstraeam hosting options are untennable.
Here are fees for bandwidth "egress", i.e. outgoing bandwidth, at several hosts:
| Service | Egress bandwidth |
|---|---|
| Heroku | not sold past 2TB limit |
| Render | 100GB + $0.10/GB |
| Netlify | 100GB + $0.10/GB |
| AWS | 1TB + $0.15 to $0.07/GB (depending on volume) |
| Digital Ocean | 1TB + $0.01/GB |
| Linode | 1TB + $0.01/GB |
| OVH | Unmetered |
| Afterburst | Unmetered |
| Feral Hosting | Unmetered |
| ... | Unmetered |
It's pretty easy to see how even without using a service like Twilio, bandwidth pricing makes relatively low-cost VPSes like DO or Linode unworkable for a freemium video chat service. Servers crashing under a spike of users is bad, but a bank account emptying can be even worse.
Interestingly, Kyle Vogt (a cofounder at Twitch) recently tweeted about the new Amazon Interactive Video Service. Immediately following, he added:
Although as an AWS offering it's too expensive to build your own @Twitch or free streaming service. The reason we built a global network from scratch in the first place was the insane prices companies like Akamai and Limelight were charging in 2006. Similar story here.
Sadly, I actually did start out thinking Amazon was the natural solution for "anything that needs to scale". At least in the crowds I followed, it seemed like an accepted truth. It wasn't until after ditching my external dependencies, testing and doing the math that I realized the whole plan was unworkable.
For a bandwidth-hungry service, it probably makes more sense to start with an unmetered VPS and then move on to a dedicated server after proving the concept. LowEndBox is a great place to find both. It has regular announcements of discounts on dedicated hardware, tutorials and many, many links to various hosts. I really wish I'd learned about them years ago.
Another good place to learn about the landscape is Reddit. There are a lot of people on a lot of subreddits trying to do a lot on a budget.
# Is it worth worrying about scaling at all?
In general, I'd say not before you've built anything. Stack Overflow got huge before needing to scale horozontally and that was on mid-to-late 2000s hardware! It's very easy to overestimate how fast modern hardware is. I've known multiple people who have scaled content sites to millions of monthly users on a single mid-tier VPS box.
Your B2B tool or job board probably won't get anywhere near that level of usage. I you're doing any kind of SaaS or membership site, more users will mean more revenue and easy ways to solve the problem unless your site becomes truly massive. On the other hand, if you work on free to use services, especially in lower income countries as I have, then it really can be an issue.
For the record, I'm running this blog for free on Render as a static site and my current side projects on Digital Ocean for very little.