How To Scale A Service Company From 2 Million To 10 Million

Decoupling application tiers to prepare for horizon scaling
Choosing amidst the database options available on the AWS platform
Shifting components of workload to appropriate AWS services for ameliorate performance and efficiency
Using Auto Scaling to ensure that y'all accept the correct number of EC2 instances bachelor to handle the load for your application
Automating your software development lifecycle with AWS services

To shut out this scaling serial, I want you to consider leveraging service-oriented architecture (SOA) and AWS application services.

Service-Oriented Architecture

Thus far, I've atomic number 82 u.s.a. on the path of decoupling your application into separate tiers and moving workload sub-components into the most fitting AWS services. How can we take this farther? The answer is to apply service-oriented architecture (SOA) principles to your architecture. Instead of just splitting your awarding by server roles (for example, spider web vs. application), consider splitting resources further past the service they collectively deliver. The benefit of this is that y'all can scale each service independently. Your web and awarding tiers will have unlike resources requirements, and then will your unlike services.

Adopting SOA sounds great, but as a startup you might non desire to spend the fourth dimension or your express resources to develop an enterprise-grade queuing or messaging system. But fearfulness not, developers. The AWS platform makes it really piece of cake for you to achieve a loosely coupled architecture by offering many plug and play generic services. I discuss some of these usually used services in the following sections.

Don't Reinvent the Wheel

Sending emails, queueing, transcoding, monitoring, and logging are generic features that near software requires, but does non add differentiating value. If they don't add differentiating value, why should you spend time to develop them in-house? AWS provides a host of these generic services to help you build infrastructure quickly. The post-obit illustration shows some application services that AWS offers.

application services offered by AWS

Amazon Simple Queue Service (SQS)
SQS acts as mucilage for a distributed and highly scalable system. When a system is stitched together via asynchronous messaging, different parts of the system tin can scale or fail independently. Imagine a job that needs to be processed in 3 steps. If you take independent fleets that are responsible for each step and step two fails, that job could remain in the queue until footstep two is stock-still rather than be canceled completely.

SQS is as well very valuable to startups because it removes the maintenance overhead from your software developers. As a managed service, SQS maintains availability of the queues for you. Every bit a scalable service, SQS allows you lot to create an unlimited number of queues, which saves y'all from having to do chapters planning. Your software developers demand to larn only a elementary RESTful API to starting time using SQS.

Amazon Simple Notification Service (SNS)
SNS is a pub-sub service for sending letters to different AWS services. Subscribers of a topic receive messages published to that topic. For example, you can have CloudWatch burn messages to a SNS topic, and any AWS Lambda function that is subscribed to that topic volition be triggered. While both SNS and SQS are messaging services on the AWS platform, SNS tin can push letters to multiple subscribers, eliminating the need to periodically check or "poll" for updates.

SNS as well has a mobile push functionality that lets you evangelize push notifications to Apple tree, Google, Fire OS, and Windows devices, besides every bit Android devices in Prc with Baidu Cloud Push. With push notifications, an installed mobile awarding can notify its users immediately by popping a notification almost an event, without opening the application. For example, if you install a sports app and enable button notifications, the app can ship you lot the latest score of your favorite team even if the app isn't running. The notification appears on your device, and when you acknowledge information technology, the app launches to display more information. Users' experiences are like to receiving an SMS, but with enhanced functionality and at a fraction of the toll.

AWS Lambda
AWS Lambda runs your code written in Coffee, Node.js, and Python without requiring you to provision or manage servers. Lambda will run and calibration your code with high availability, and yous pay merely for the compute fourth dimension you consume in increments of 100 milliseconds. The code you upload will get Lambda functions, which can be triggered by various events on the AWS platform. Lambda functions tin can be thought of as beginning-class compute resources that y'all tin insert into your AWS architecture. A common use case is to have a Lambda function listening to S3 events and performing custom processing logic when objects are uploaded into S3. AWS Lambda is a fairly new but powerful service on the AWS platform.

Users > 1 One thousand thousand

Let'south assume my awarding has become hugely popular, and it'south now getting close to a million visitors. Serving a million users and more is going to crave that we bring together all the approaches I've discussed then far:

Deploying beyond multiple Availability Zones
Using Elastic Load Balancing (ELB) betwixt tiers
Using Auto Scaling
Using service-oriented architecture
Serving content using appropriate AWS services (for case, local vs. Elastic Block Store vs. S3)
Caching off DB using ElastiCache
Moving country off tiers and so you tin horizontally scale

The following diagram illustrates usage of same approaches.

architecture for over one million users

Users @ Five to X Million

Between five to ten million users, you might run into issues with your database due to contention on the write primary. There are several methods of mitigation, but they essentially crave you to further decouple in the database layer. These methods include database federation, sharding, and using NoSql.

Database Federation

Database federation is virtually separating databases by role. For case, you might choose to store data for your forum, users, and products in 3 distinct databases. Over again, this allows your components to scale independently. This database strategy can complement the SOA strategy. For example, you might decide you want to keep data belonging to separate services in separate databases. On the flip side, this architecture is more circuitous and requires yous to write more than sophisticated queries to fetch information.

Sharding

If your data is still too large to exist managed under a federation schema, you can consider sharding. This is horizontal scaling at the database tier. With sharding, you shop data beyond multiple databases and spread the records evenly. You tin can accept users with last names in the A through M range in one database and the rest in another. Sharding allows you to scale larger than federation, but it requires more logic in your awarding to dynamically change the target database depending on the information you need.

NoSql

Lastly, if you have independent tables in relational database, you lot could consider moving that data to a NoSql database such as Amazon DynamoDB. Relational database has overhead associated with maintaining relationships between data. So why take on that overhead when you don't demand to extract relationships from your data? DynamoDB is a managed, scalable, and high operation key-value store. Like RDS, DynamoDB takes care of authoritative tasks for you so you tin focus on high-value tasks. Data is stored across multiple facilities across a region to achieve high availability. DynamoDB as well scales automatically with the size of information.

Next Steps

I promise this blog series has given y'all some food for thought regarding scaling on the AWS platform. For additional resources, delight refer to our reference architecture section to encounter commonly used compages, read our documentation to stay informed, and go to the startup blog section to see how your peers are using AWS services.

When you are ready to bound on AWS, there is an AWS Free Tier available every bit well.

Happy coding!

Summary

1. Split tiers into individual services (SOA).

2. Don't reinvent the cycle. Leverage AWS application services when possible.

3. Consider database federation and sharding to manage large amounts of data.

iv. Motion to NoSql if and when it makes sense.

Reference

Scaling Up to Your First 10 Million Users