This won’t come as a shock, but engineers who are on the hook to develop revenue-generating software are quickly moving away from traditional, monolithic architectures and delivering code faster than ever. This trend is especially evident among Scalyr customers, but we wanted to understand how it is generally unfolding and how it affects DevOps observability. We surveyed 155 software development practitioners in DevOps-focused organizations over the last couple of months, and just released our State of DevOps Observability Report. In this blog, I’ll give you the Cliff’s Notes version of what we found.
We confirmed that organizations really are shifting away from traditional, monolithic architectures. Three-quarters of survey respondents say they deliver at least some of their applications, and more than one-third deliver most of their applications, as microservices. They are also delivering software rapidly, with 71 percent of engineers pushing code into production at least weekly and nearly one-third doing so at least daily. Looking at all of our survey findings through the lens of these two trends, we realize that modern software delivery is putting pressure on DevOps observability.
Here are some of our findings:
Companies are delivering software in a modern way.
- Three-quarters of respondents deliver some and more than one-third deliver most of their applications as microservices.
- 71 percent of engineers push code at least once per week, and nearly one-third push code at least once per day.
Companies rely on many tools for observability.
- Nearly half of respondents have five or more observability tools.
- 58 percent of respondents in a DevOps role have five or more such tools.
Engineering teams spend a lot of time troubleshooting.
- 40 percent of respondents say their companies’ engineers spend most of their time troubleshooting software issues.
- This percentage increases to 62 percent for respondents who deliver mostly microservices and 73 percent for those who push code at least once per day.
Ad-hoc query speed is respondents’ top log management requirement.
- 54 percent of respondents care most about ad-hoc query speed in log management.
- This number jumps to 61 percent for respondents who push code at least once per day and 68 percent for those who deliver mostly microservices.
- Other top requirements include the ability to parse and operate intelligently on complex logs as well as alert speed.
Despite query speed being their top priority, respondents who deliver software in a modern way spend most of their investigation time waiting.
- One-fourth of respondents spend most of their log management investigation time waiting for queries to complete.
- This number jumps to 53 percent for respondents who deliver mostly microservices and 81 percent for those who push code at least once per day.
The common thread among these findings is that, as organizations make the shift from more traditional architectures to microservices and deliver code more frequently, they spend more time troubleshooting and debugging software, working to understand complex data sources, and operational visibility is paramount. Especially when it comes to centralized log management, speed is critical. In addition to speed, our observation from interacting with Scalyr customers is that they both need and value the ability to perform searches that are not only ad hoc but also unstructured and flexible (free text, regular expressions, wildcards, etc.). Our takeaway is that organizations that are experiencing this shift to modern software delivery need to take a hard look at their observability tools and processes to make sure they can keep pace and evolve to support modern, high-speed, distributed software engineering practices.
Are you delivering software in this way? How are you keeping up from an observability standpoint? Tell us about your experiences in the comments.
