We want to share our recent experience of hosting BigBlueButton on AWS.
Well, we were interested in AWS for two reasons:
- Proximity to users and, hence, lower network latency, and
- Auto-scaling so that we can turn off the instances when not in use
Our client, a school with ~300 students, was based out of India, and we chose the Mumbai region of AWS. Thus, we could reduce network latency by 400 milliseconds, which means a quick start of classes.
BigBlueButton auto-scaling in AWS is tricky:
- Scale In doesn’t work automatically based on say, CPU usage. Even if there is one user on a particular EC2 instance showing very low CPU usage, you can’t Scale In, i.e., remove that instance. Otherwise, the on-going class on that instance would get disrupted.
- Scale-Out is also complicated but doable. You would need an entry into Route53 for the new instance, find out the private IP of the new instance and re-configure BBB to use that, register the new instance into Scalelite, etc.
- As we knew that classes would start at 9 AM and continue till 3 PM, we were able to use Scheduled Scaling for Scale-Out. At 8.45 AM, we will Scale-out EC2 instances from pre-configured API images, and at
We used the C5a.2xlarge instance, which was able to handle ~150 simultaneous users.
Thankfully the client didn’t want recordings, which means we don’t need to worry about running the recording process after classes, which would have made Scale-Out more difficult.
Within two weeks of AWS deployment, we were able to stabilize the system.
Then came the bills shock!
Within 15 days, our AWS bills were hundreds of dollars. Way, way more than we ever estimated!
The biggest surprise of the bills was Bandwidth usage at around 40%.
While doing basic match for AWS billing (https://aws.amazon.com/ec2/pricing/on-demand/), we found C5a.2xlarge instances within our budget. We would use the C5a.2xlarge instance only for 8 out of 24 hours and five days a week with auto-scaling.
As we didn’t need recordings, we don’t need to worry about storage costs.
As a startup, we don’t know how long we would need an EC2 instance in advance. Hence, we can’t opt for reserved pricing.
However, we didn’t account for bandwidth charges. Our current data center gives us unlimited bandwidth for free.
It was one big chaos…within a couple of days, we moved our deployment to our favorite data center, where we host all our other clients, and that offers unlimited bandwidth, among other things.
AWS team waived off 60% of the bill. That was a big relief, and we are thankful to AWS.
Still, we had to pay a large sum to clear our dues.
So that’s our AWS deployment story!
How about your experience?
Have you been burned by hosting charges for BigBlueButton?
What alternative solution did you choose?
Leave a comment to let me know.