Uteach is an online LMS that helps educators, teachers, and businesses deliver courses, sessions, or other educational and training programs. It provides a comprehensive all-in-one platform to build, sell, deliver and manage training content while automating a lot of the learning tasks to save time and remove the operational overhead of maintaining an online teaching business.
Uteach is a cloud-based SaaS LMS, which removes the deployment and maintenance overhead associated with on-premise self-managed LMS solution. They take care of handling the underlying technical complexities, so educators and creators can focus on building their training content.
Infrastructure Overview
Uteach underlying infrastructure is hosted on AWS. It uses EKS for running the application workloads and ELB for enabling network access. Additionally, they use Cloudflare CDN for caching content on edge locations closer to the users for optimizing performance. Each tenant got their own dns hosted zone in Cloudflare.
Uteach is a multi-tenant platform where different creators can register and join. Each creator on the platform can build his own website which is hosted by Uteach and includes his course contents and other data. Creators also get a separate dedicated URL so that students and learners can get access to the creator's content. This URL can be automatically provided by Uteach as a subdomain under the Uteach domain, or there's also an option to connect your own custom domain and use it through CloudFlare dns zone..
Challenge
Now, although each client (educator on the platform) tenant is isolated, this is only logical isolation on the application level, but the underlying infrastructure is shared between the clients. This is a common design in building multi-tenant SaaS solutions where creating dedicated resources for each tenant is not possible due to extremely higher costs or other resource management constraints.
The challenge with this design is that when there's a resource overload on one of the users, it can impact the whole infrastructure, causing disruptions for all the platform. A recurring scenario that was happening frequently with Uteach is the occurrence of targeted DDoS attacks that overload one of the clients on a specific URL, when this happened, the flooding network traffic was actually hitting the underlying cluster nodes which also host other clients, so all the clients got impacted and experienced the service disruptions.
These service disruptions degrade the application's performance and reliability, hurting business reputation and putting business continuity at risk.
Goals
Uteach wanted to improve the reliability of the platform and avoid such service disruptions. This required a solution that can:
- Protect against targeted DDoS attacks that overload the system
- Improve the scalability of the infrastructure to be able to automatically handle sudden increase in the load
Solution
Uteach reached out to Aland Cloud to explore and evaluate the possible solutions to overcome these challenges and achieve the desired goals. We were able to efficiently address these requirements by implementing the following:
Block disruptive Traffic with WAF
To mitigate the DDoS attacks, we added AWS WAF to the design and configured the appropriate rules for blocking suspicious traffic and applying rate-limiting. The WAF sits in front of the loadbalancer to prevent unwanted traffic from reaching the origin servers.
We configured a web ACL and associated it with the Uteach web application resources on the AWS WAF. This web ACL includes a set of rules for allowing or blocking specific traffic. When the WAF receives an incoming request, it inspects the request and evaluates its content against the rules that are set, when it finds a matching rule, it applies the required action that corresponds to the rule by blocking or allowing the traffic. There's also a default rule set for traffic that doesn't match any of the explicit block or allow rules.
Configure Auto-Scaling for EKS
To enable the infrastructure to scale dynamically with varying load, we implemented autoscaling for the EKS cluster. This autoscaling will automatically add more nodes to the cluster when additional resources are required for the Pods to continue running. We used Karpenter as the autoscaling service, it monitors the cluster and detects when there's a resource shortage, then it automatically provisions more nodes to handle the load.
Autoscaling improved the performance and reliability of the system by avoiding service disruptions caused by load spikes. It also provided a cost-efficient solution as it scales down the nodes again when the cluster load goes down, so it automaltically right-sizes the resources according to optimal balance between performance and cost.
Results & Benefits
Overall, Uteach reports that implementing these improvements with WAF and autoscaling has provided a lot of benefits including:
- Increased availability and uptime for the platform, 99.99%
- No more service disruptions caused by DDoS attacks that impacted SaaS platform business and their revenue's. Many businesses reported losses during continues DDoS attacks.
- More optimized system performance with less required manual intervention and operational overhead
Conclusion
We were able to satisfy the client requirements by improving the cloud architecture and adding critical infrastructure components that enhance the security, performance, and reliability of the system. We implemented AWS WAF with the appropriate set of rules to block malicious traffic and mitigate DDoS attacks preventing the backend infrastructure from being overloaded and avoiding service disruptions.
We boosted the solution even further by configuring auto scaling capabilities for the EKS cluster to enable automatic provisioning of additional nodes when the resource load increases, and automatic removal of nodes when the load decreases again. Thus, striking an optimal balance between cost efficiency and performance in an automated way.
These improvements addressed the client's challenges and strengthened the overall system reliability, building more trust and positive experience for the platform users.
