Observability is a somewhat recent addition to the ever-growing list of buzzwords in the tech industry. However, unlike many of those items, observability does refer to an important concept—namely, the ability to understand a system’s internal state by looking at external signs it emits.
This post is about how to achieve observability, including tools you can leverage, but we can’t talk about that without defining it and explaining why you would want to achieve it in the first place. So, that’s exactly what we’re going to do. Let’s dig in.
Observability Is Important and You Should Care About It: Let’s Learn Why
Once the “what” and “why” are out of our way, we’ll be ready to tackle the “how” of observability. So, let’s start with some basics.
What Is Observability?
Guessing the meaning of the word observability isn’t that hard, given the way the English language works. Observability is the capacity that a thing has to be observed. In our context, the object of observation is a system. More specifically, a system running in production. So, what does it mean to observe a production system? To answer that, I’ll borrow the definition from another post on the Scalyr blog:
Observability helps you understand the internals of your production system by asking questions from the outside.
In a nutshell, system observability is the practice of making your system emit signs all the time so you can track those signs from the outside to identify problems and fix them.
Why Should You Care About Observability?
You’ve just seen that observability gives you the capacity to understand how your system works internally by observing the external evidence it produces. And that’s not all. Observability also involves instrumenting your systems to ensure those external signs are created in the first place.
So, it seems that observability is a reasonably simple concept to grasp. So, with the “what” out of our way, our attention naturally turns to the next obvious question: why? Why is observability so essential for production systems?
The Kinds of Problems Observability Solves Are Good Problems to Have
The problems observability solves are good problems to have. They are somewhat inevitable side effects of how software development is done today, which is superior to the way we did it decades ago.
Software development in the past was simpler. Not necessarily easier, mind you, but simpler: There were fewer components. If you had something like a basic web application, some basic monitoring would typically suffice to ensure everything was working properly.
However, today things are different. We live in the era of the cloud and distributed systems. We have containerization, micro-services, Amazon AWS, and more. Some people might argue all of that has added more complexity to the software development process. I won’t dispute that. But I argue that despite the added complexity, the net result is still positive due to the benefits that these new tools and approaches bring to the table. For instance, a tool like Docker greatly improves the software development pipeline, facilitating testing and making the “it works on my machine” problem effectively go away.
The complexity is real, though, and it does add difficulties. Since there are more moving parts interacting with one another in many ways, there are more opportunities for problems to occur. When they do happen, detecting and fixing them might be harder than it was in the past.
Logging, Monitoring, Observability: What’s the Relationship?
How is observability different from monitoring? How does logging fit into the puzzle? In some ways, you can think of logging and monitoring as observability tools. But the relationship is more complex than that. Let’s answer those questions now.
Logging Is a Pillar of Observability
We mentioned earlier that one of the fundamentals of observability is to instrumentate your systems so they generate signs to the external world about their behavior. You can then track those external pieces of evidence to understand what failed, how it failed, and then hopefully fix the problem. If that makes you think of logging, that’s not a coincidence: Logging is indeed one of the most powerful techniques you can use to make your systems leave a trail of breadcrumbs behind.
In other words, the difference between logging and observability is that logging is an enabler of observability. While observability might be considered the overarching concept, logging is one of the stepping stones that might help you get there.
Monitoring Is an Action
And what about monitoring? I’ll argue there are two main differences between observability and monitoring. We could say one of them is philosophical or semantic, while the other is more practical.
The semantic difference is that monitoring is an action. Observability, on the other hand, is an attribute. Your system can be more or less observable. Having adequate monitoring in place certainly improves the observability of a system.
But the practical difference between the two is that monitoring always answers the same questions. If you’re monitoring CPU and disk usage, then you’ll always get answers about CPU and disk usage. Leveraging observability gives you the ability to always ask and answer new questions, venturing into the territory of unknown unknowns.
Same Purpose, Different Layers
So, to answer the question in this section’s title, logging, monitoring, and observability serve the same purpose, but we could say they are at different levels of abstraction.
Monitoring is at the tactical level: It’s an action that you do.
Logging is also at the tactical level. It’s one of the fundamental building blocks of being able to glimpse behind the curtains of your system, so to speak. But we could argue that logging is a little bit above monitoring since it exists in various levels of your IT infrastructure, from the hardware to the database to the application and a whole lot more.
Finally, observability is the big picture final goal, which you can accomplish by bringing the lower levels together in a smart way—for instance, by leveraging log aggregation.
Observability Sounds Great…But How Do You Get There?
You accomplish observability by leveraging adequate observability tools and techniques. Here are some of the tools, tool categories, and techniques that can help you on your way to observability:
- Use logging frameworks. Logging is one of the building blocks of observability, but you shouldn’t try to roll out your homegrown logging solution. Creating a complete logging framework from scratch is harder and more involved than it might seem at first glance, and doing so isn’t likely to be your organization’s core business. Instead, leverage one of the available logging frameworks for your current tech stack.
- Follow conventions and best practices. You should follow best practices and conventions around monitoring, logging, and the other building blocks of observability. For example, caring about log formatting—both to make them easily parsable but also human-readable—will net you dividends in the future.
- Leverage log visualization. Log visualization lets you see the insights from your logs in the forms of charts and graphs without the need for hard-to-write queries and the like. There are many use cases for log visualization, and it definitely facilitates the journey into observability.
- Adopt a log management solution. A log management platform is also a huge facilitator of observability. For instance, a tool like Scalyr gives you the ability to centralize your logs from disparate sources into a single location, parse them, and then perform powerful and amazingly fast searches. Give Scalyr a try today.
To Observability…and Beyond!
As tech professionals, we’re acutely aware of how infatuated with buzzwords our sector is. The tech industry loves buzzwords. It seems to eat them for breakfast. So, separating the chaff from the wheat—that is, identifying which buzzwords are just ephemeral fads and which ones refer to concepts that are here to stay—isn’t always an easy task.
The existing evidence seems to support that observability has a firm place in the wheat category. It’s definitely not a passing trend. On the contrary, it’s a concept you and your organization need to take seriously if you want to survive and thrive in the brave new world of cloud, DevOps, and distributed systems. You simply can’t afford to ignore it.
Happily for you, in this post you’ve learned more about observability. You’ve learned its definition, the problems it solves—and why they’re good problems to have—and how it relates to similar approaches. More importantly, we’ve covered the steps you need to take in order to achieve observability, including tools that can ease your way there.
Many organizations in the past dismissed things like automated testing or version control as unimportant. Many of them are no longer around. Don’t join their ranks. Leverage observability (and observability tools) and reap the benefits of doing so.
Thanks for reading, and until the next time.
