Best practices for testing and deploying microservices

Transitioning from a monolithic architecture to microservices is an intricate, time-consuming task. It demands both strategic foresight and meticulous execution.

In this 10-part series, we’ll guide you through the most common challenges faced during monolith to microservices migration. Last week we published the fifth part of our series, on monitoring and observability in microservices. This week, we are going to dive into testing and deployment strategies in microservices architecture.

We'll publish a new article every Monday, so stay tuned. You can also download the full 10-part series to guide your monolith-to-microservices migration.

Strong testing and deployment strategies brief intro

The mesh of simultaneously communicating services creates unique challenges when testing and deploying in microservices. Not only do you have to ensure that each service is functioning internally, but that it’s interacting effectively with other services. And then, you have to make sure the whole web of microservices is functioning together as a system.

With multiple layers acting and interacting simultaneously, you need strong testing and deployment strategies to maintain the quality and stability of microservices. That’s what we will talk about in this article.

Testing strategies

You can’t test all layers of a microservices architecture with a single type of test. Your team has to test your microservices at four different levels to ensure it is stable at each layer of functionality. You start testing at the granular, or unit level and work your way up to testing the whole system.

Unit testing

Unit testing in microservices is fundamentally the same as in monoliths. Your team tests individual components, or services in isolation to make sure they meet specified requirements. Frameworks such as JUnit, NUnit, and Mocha are popular choices in their specific language spaces. The main difference with microservices is that with microservices you’re dealing with multiple frameworks within your app. So, you’ll have to use different testing stacks for each microservice based on the technology used to create it.

Contract testing

Once you’ve ensured units are working well in isolation, you can begin to test how they are interacting. Contract testing is a constrained version of system testing. Focusing on only two units, consumer and provider, contract testing ensures that both have a shared understanding of the API contract. In other words, the consumer can check for aspects like response format, status codes, etc., while the producer can check that its response is in accordance with the agreed contract. Tools like Pact, Spring Cloud Contract, and Swagger can be used for contract testing.

Integration testing

At the next level up, you’ll have to test the interaction and communication in the system as a whole. During integration testing, you’ll ensure that each of those units, which you have tested and shown to work effectively in isolation, are communicating effectively with other units to create a cohesive system. Tools like Postman, SoapUI, and REST-assured can be used for integration testing of RESTful APIs, for example.

End-to-end testing

Finally, end-to-end testing validates the entire system from the user's perspective. Unlike integration testing, which validates how microservices work together, E2E tests execute the entire set of calls involved in any user action, testing the end result to validate that the full feature is working as expected to fulfill business requirements. Tools like Selenium, Cypress, and Cucumber can be used for end-to-end testing.

In a dynamic system like microservices, you should be continuously running tests to make sure issues don’t creep in through updates or patches. Practices like Continuous Integration (CI) and Continuous Testing automate the testing process and catch issues early in the development cycle.

Deployment strategies

When your team is rolling out your new microservices structure, they’re not just rolling out a single project. They’re rolling out multiple independent services, each one dependent on the others. That complexity requires a careful, well-thought-through deployment process to minimize problems and the potential knock-on effects throughout the system.

The right deployment strategies help you do just that. Below are four effective approaches to consider for a smoother rollout.

Blue-green deployment

In blue-green deployment, you run two identical production environments, i.e. "blue" and "green", during updates. One of these environments, in this case blue, runs the old version while you deploy the new version of your microservices on green. During the update, all traffic is directed to the stable (blue) environment. Once the green environment (i.e. the updated version) is tested and validated, you switch traffic over to it. This approach allows for quick rollbacks in case of issues and ensures that there is no downtime during deployment.

Canary deployment

With canary deployment, you gradually roll out a new version of a microservice to a small subset of users or servers. The old version runs alongside the new version (the canary) while traffic is diverted to the canary subset by subset. If the canary performs well, the rollout continues until all traffic is shifted to the new version. If there is an issue, you can roll back the update by switching all users back to the old version. This strategy allows you to test the new version in a production environment with real traffic, without risking all users.

Rolling update

Rolling updates are performed in batches, where a portion of the instances are updated while the remaining instances continue to serve traffic. Once the updated instances are stable, the next batch is updated. This process continues until all instances are updated. This system minimizes downtime and allows for a quick rollback in case a problem is found, so you can minimize the number of users affected by it.

Serverless deployment

Serverless deployment uses serverless platforms like AWS Lambda, Azure Functions, or Google Cloud Functions to deploy and run functions or small units of code, which can be used to implement microservices. Essentially, this splits the microservice into smaller, individual parts. Serverless platforms abstract away infrastructure management and automatically scale these functions based on demand. This approach simplifies deployment and allows for granular scaling of individual functions, which can be orchestrated to form a microservices architecture.

Every app needs to be continually updated to stay healthy and up-to-date. So, organizations often adopt Continuous Deployment (CD) practices to streamline the deployment process. CD pipelines automate the build, testing, and deployment steps to make frequent and reliable releases easier.

Ensuring durable deployment with rigorous testing – Netflix's testing and deployment strategy

When Netflix rolled out their microservices architecture, they had to completely change how they updated the app. In response to the change, they developed a suite of tools and practices they could use to test and deploy their updates without interrupting service to their customers

Netflix’s testing & stress testing strategy

Netflix had to develop new testing strategies to make sure they understood what was happening at all levels of their app, including unit testing, integration testing, and end-to-end testing. They chose Junit, Mockito, and Spock to test at the unit level and REST-assured for integration testing of their RESTful APIs.

Netflix also chose to stress test each microservice through a process called Test Annihilation. Essentially, what they do is purposefully inject failures into their microservices during testing to see how the update deals with the faults. This helps them ensure the resilience and fault tolerance of their microservices.

Deploying updates

Netflix pioneered the use of Canary Deployment and Red/Black Deployment (similar to Blue-Green Deployment). By using a combination of strategies, they were able to ensure continuous service to their subscribers. In the same way, they developed Spinnaker, an open-source, multi-cloud continuous delivery platform, to automate their deployment pipelines. Spinnaker allows Netflix to safely and efficiently deploy microservices across diverse regions and cloud providers.

Planning for the unexpected

Before any update is deployed, Netflix stress tests each microservice again with Chaos Engineering practices. They developed the tools Chaos Monkey and Chaos Kong to simulate failures and disruptions in their infrastructure so they could see how their updates dealt with unexpected scenarios and failures. This gives them the ability to proactively build resilience into their architecture, ensuring that their microservices can handle unexpected scenarios.

Ready to scale reliably

The massive scale of Netflix’s reach requires testing to failure before risking the next update. To do that, Netflix adopted comprehensive testing strategies, automated deployment pipelines, and chaos engineering practices. This way, Netflix has built a highly reliable and scalable microservices architecture that can handle the massive scale and complexity of their streaming service.

Looking ahead

Next week we’ll publish the next post in the series: “Understanding the security and access control requirements of microservices environment”

Or, you can download the complete 10-part series in one e-book now: "Monolith to microservices migration: 10 critical challenges to consider".