Scale, a word that has to do with size, and ability, another word that gets you thinking if an event is possible or not. We could stop right here and have you doing some scalability testing, but there’s much more to it than knowing the term’s meaning. Regardless of how good looking a system is, it could still fail scalability tests! You see, the tests in this variety of analyses have nothing to do with form and feature. They’re strictly looking into how a system would respond when various loads exert onto it.
In this post, we’ll take you through some of the areas that make up a solid test and wade into the various thought processes behind the idea of scalability testing. You should acquire the ability to confidently discuss critical aspects of the topic and perhaps even be able to suggest tools and procedures when setting out the actual tests.
What Is Scalability Testing?
Let’s take a step back before diving headfirst into a dictionary definition here, shall we? As has been seen above, the entire idea here is to look into matters bigger than if an application does what the functionality documents dictate. It’s safe to assume that during design (and development), only a few users are made for each user group in order to make progress. However, thousands more are desired when the system is finished. In fact, companies tend to get creative with marketing just to get user base figures to brag about to (funnily enough) other potential users. Without successfully completing scalability testing procedures, those users could just spell doom for said imaginary companies.
Before any system can ask for users, we must determine if it won’t succumb to the pressure of too many requests coming through at the same time. There you go. That’s the founding perspective behind scalability testing. What you’ll notice, however, is that when conducting scalability tests, not only will the ceiling be sought after, but solutions to such snags should be suggested and tested as well.
It’s well worth mentioning now that scalability testing falls under black box testing.
What’s the Point?
Since the concept of scale is bidirectional, the ability of a system to shrink on command should also be determined and reported. When presented with scalability testing results, decision-makers can then plan for sustainable growth (in any direction) with more certainty.
So far we’ve been easy on the jargon. The “how” part should bring itself out of hiding as you read on. What you need to accept at this point is that scalability tests, unlike load tests, consider the max and min performance attributes of a system. They then go ahead and seek remedies to the limits discovered. Load tests will just seek to find levels of (high) pressure under which a system folds. Keep this in mind as we delve into the crux of the matter.
Scalability Testing Attributes
We could peg as many as a dozen test points (attributes) for a system. That number is often determined by the complexity of the said application. There would be no use testing how a system intended for only five to 10 users performs when 1,000 users make concurrent requests for resources. Neither would there exist grounds for testing database-related issues where an application is static. Such project management edges should help you save time. Quartering the size and complexity of your system aside, you could consider the following attributes:
- User-number-related performance
- Response time
- Memory usage
- Throughput
- Load-bearing performance
- Network usage
- Screen transition
And now for brief explanations of these five scalability testing attributes.
User-Number-Related Performance
As you increase the number of users communicating with a system, the system tends to show behavioral changes. If (as expected) it slows down, there comes a point where it just freezes. Corrective action at that point will allow the ability to perform smoothly, even with the new load. Reports produced for this attribute should show what resources were added to beat the bottleneck. Recordings are made for small, intermediate, and excess amounts of users.
Response Time
At varying loads (number of users accessing the system), the wait time for a request to be processed tends to increase. Results here allow for changes in resources that determine speed to be made available in time to maintain a good user experience.
Memory Usage
Memory usage often stems from the application source code’s potency. When running a program, the amount of time that the CPU takes to compile code directly affects throughput. This then touches several other attributes adversely. Standards should then be set to minimize the choice of programming language’s effect on the overall scalability of a project.
Throughput
Many tools exist solely to measure the throughput of a network-reliant application. Changes that can mitigate slow throughput can be in the form of different packet policies among others. As loads increase, the time it takes to respond to requests to and from the server through the network (throughput) should not be compromised.
Load-Bearing Performance
The load is the moving variable when conducting most of the tests above. Small loads will provide a standard, or starting measure, of performance. As the load increases, taking logs of how every other resource is affected, along with mitigation measures to attain stability, will be useful.
Network Usage
Logging network usage is a good way to ensure your infrastructure can withstand high traffic volumes. The fewer congestion you encounter during network-related tests, the better your chances of successfully ascending to new usage levels. Ideally, every congestion your network encounters should be studied and mitigated before it actually happens when new users come on board.
Screen Transition
Testing how long your system transitions from a single interface to the next across the possible access devices is a good gauge of how ready it is to scale. Slow screen transitions are often a sign of code that is not optimized. In the case of resources being requested in order to load a screen, optimization of the assets through compression would be wise before adding the number of concurrent requests for it. This way, your application works smoothly regardless of how many users are looking to access the same assets. Failure to optimize screen transition could lead to long wait times, and hordes of dissatisfied users.
Advantages of Scalability Testing
Let’s discuss why a company may be better off conducting scalability tests after all. For one, trying to add users to a system while oblivious of its capabilities under varying loads can prove a perilous endeavor. Upon systems failure, stakeholder confidence can wane, along with any hope of rising again.
Secondly, you save a lot of money by setting reasonable growth rates that ideally pay for themselves. Lastly, even when selling a company or facing foreclosure, reducing resources while cutting costs can be a controlled event. And this potentially saves the reputations of everyone with the ailing business on their résumés.
How to Start Scalability Tests
With all the knowledge you’ve been equipped with, it only makes sense that you know how to get started with the process of scalability testing. Industry best practice entails that you should determine the scope of your project first. That comes before you even pick out teams and tools for the job.
Prerequisites to look at based on the testing tools used during this primer process include the following;
- Capability. The testing applications should have the ability to look into the attributes you will have deemed essential for the study.
- Environment. The operating system in which to run the testing suite should be ready beforehand.
- Memory and Processor. In order to simulate the loads on your network and applications, provide the testing agent with enough storage and processing resources.
Assuming all these are in place, the following checklist should guide you from idea to actionable findings in the form of condensed reports.
- Agree on the scalability attributes to focus on.
- Discover and acquire the tools for the tests.
- Set up your test scenario (including reasonable test controls).
- Carry out scalability tests while recording all findings.
- Determine remedies for scaling purposes at all load intervals.
- Extract and present your findings.
While it can be a routine procedure for big companies such as Google, Apple, and Amazon, smaller companies often only carry out scalability tests when essential. They usually lack as many tools, hands, or the budget to carry out scalability testing at the scale that these giants do (see what we did there?). As such, it’s best to have tools that make the entire process less difficult.
Among a plethora of others, tools like Scalyr allow engineers and users to carry out scalability testing. All without the familiarization process gnawing into the actual testing.
Vertical Scaling vs Horizontal Scaling
Findings from your scalability tests often lead to improvements in the application and the resources around it. There are two ways you can scale the infrastructure around your applications; vertical or horizontal.
Vertical Scaling
An option to improve capacity and processing speed before increasing load is to add components to your infrastructure. This could be in the form of installing better processors to the same racks that host your applications. Cloud or on-premise, it is possible to scale vertically by allocating more space to the same servers.
While cheaper than horizontal scaling, the downside to vertical scaling is that you cut into the performance as the components pile up. Compatibility issues also creep up on your infrastructure, along with complexity.
Horizontal Scaling
In the case of horizontal scaling (or expanding out), a new server to share the load with the incumbent would be introduced to the network. This is a quick way to double or better the performance and capacity of your infrastructure.
An obvious downside to this approach to scaling is the cost that comes with duplicating your loadout each time you need to expand. Another caveat is the fact that you’d need more attention invested in maintaining the new equipment appended to your current set up.
Tip – How To Scale Fast
Let’s plunge into the most crucial tip for scaling expeditiously; use the cloud. Most of the results you’ll discover after running tests will point at the requirement for space or better processing power. As such, it makes sense to have your applications hosted in an environment that has virtually infinite amounts of both. You ought to use the cloud in some shape or form in order to cut corners and expenses without negotiating on performance.
Hybrid cloud models have been known to improve security, local accessibility of progress crucial resources while allowing wider deployment surface area for tests and scaling. Here’s a breakdown of how you can do this.
Conclusion
If you read every word above as it was meant, you should be ready with enough of the talk to do the walk. You know enough about scaling applications the right way. Now it makes sense to run quick tests on your current application environment with Scalyr to confirm or exhume aspects of your system that need improvement before you scale.
