Async vs Sync Benchmark (.NET)

Tools

  • .NET 5 Web Api application (test target)
  • Azure Sql Database
  • Azure App Service (hosts the application)
  • Azure App Insights (to gather metrics)
  • locust framework (to simulate user load).

Configuration

Experiment schema
  • a user from the locust host 1 hits the synchronous endpoint of the App Service 1, gets the response, stays idle during 0.5–1 seconds (the exact time delay is random). Repeats till the end of the experiment
  • a user from the locust host 2 behaves exactly the same, with the only one difference — he hits the asynchronous endpoint of the App Service 2.
App Services code

Metrics

  • requests per minute — shows a number of requests that the application actually processed and returned a status code
  • thread count — shows the number of threads the app service consumes
  • median response time, ms

Experiment #1

  • number of users: 75 (per service)
Experiment #1. Requests per minute
Experiment #1. Thread count
Experiment #1. Summary

Experiment #2

  • number of users: 150
Experiment #2. Requests per minute
Experiment #2. Summary

Experiment #3

  • number of users: 200
Experiment #3. Summary

Conclusion

The use of asynchronous operations instead of synchronous doesn’t help to improve performance or user experience directly. First of all, it improves stability and predictability under pressure. In other words, it picks up the load threshold, so the system can process more before it starts degrading.

Appendix #1. Technical details

  • Azure App Service: B1, 100 ACU, 1,75 Gb Memory, A-Series compute equivalent
  • Azure Sql Database: Standard S4: 200 DTUs, 500 Mb storage
  • Sql Connection settings: Max Pool Size=200

Appendix #2. Notes

*In order to achieve the cleanest test result I should have run tests from 2 VMs located in the same network where the target App Services sit. But I assumed that a network lag would impact both app in a similar way, therefore it cannot jeopardize the main goal — compare how asynchronous and synchronous methods behave.

Appendix #3. Bonus Experiment

What did I hack in order to force synchronous endpoint to perform almost as asynchronous and plot the graph below (experiment conditions are the same as in the third one — 200 users)?

Bonus Experiment. Requests per minute

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Artem Mikulich

Artem Mikulich

64 Followers

My goal is to eliminate technological barriers and help businesses to grow and change the world to the best