Observability with Eduardo Silva

There are hundreds of observability companies out there and many ways to think about observability, such as application performance monitoring, server monitoring, and tracing. In a production application, multiple tools are often needed to get proper visibility on the application. This creates some challenges. Applications can produce lots of different observability data, but how should the data be routed to the various downstream tools?

In addition, how can the data be selectively sent to different storage tiers to minimize costs? Calypdia is a service that helps manage observability data from source to destination. Eduardo Silva is the founder and CEO of Calypdia, and he joins us in this episode. This episode is hosted by Lee Acheson.

Lee Acheson is a software architect, author, and thought leader on cloud computing and application modernization. His best-selling book, Architecting for Scale, is an essential resource for technical teams looking to maintain high availability and manage risk in their cloud environments. Lee is the host of his podcast, Modern Digital Business, produced for people looking to build and grow their digital business, listen at MDB.fm. Follow Lee at softwarearchitectureinsights.com and see all his content at LeeAcheson.com.

Eduardo, welcome to SoftFringingering Daily. Hello, thank you for inviting me to be here. I'm glad you're here. First of all, is it Clapia or Clapia?

How is it pronounced? We pronounced the company name as Calypdia. Calypdia. Calypdia.

Okay, so it's actually close to the first time. That means it would be other times. It's always the struggle of mine, right? It's figuring out how to pronounce some of these, and some people's names too.

So are you an observability company or do you work with observability companies or both? How would you describe yourself? Okay, I would describe Calypdia as both as an observability and a security company that sits in the middle to move and transfer or reduce data between sources and destinations. So you're playing both the observability space and the security space.

Is the idea? Yeah, actually, if you abstract for what it means to play with data, right? We are very agnostic. And when we say it about security and observability, there are different type of outcomes, right?

Or data by distraction that you have for different type of users. But all of them comes from the same data. Yeah, the data is the same as just different use cases for the data. Yes, different solutions, different use cases, and different needs around them, right?

Right, right. Yeah, so data that's critical for one use case is not necessarily critical for another, but something else is in its place. That makes sense. How do you compare yourself?

Let's look at first observability companies specifically. So let's talk about companies like let's say Data Dog or New Relic. How do you compare to them? You're not a traditional observability company or for that matter to the security company.

Where do you fill in that niche? Yeah, that's a really good question. Actually, everybody who understands something for observability is a description that has been set by the biggest players, like Splunk, a data dog, or others, like observabilities, my platform where you sent all the data and I provide you the right tooling for analysis, insights, and everything you might need for security, for observability, or any other type of case. Now, when you think about what the user is trying to accomplish, at the end of the day, they just want to analyze data, get insights, right?

And when they take the decision, they go one step back and say, how, right? And how means, oh, I might need to centralize all the data in one platform that has all the capabilities to strike and run analysis or reduce my data or do some kind of math around it. And then you start thinking about Splunk, you start thinking about data dog or elastic open search for any other major player. But where we say Kalithya, right?

Kalithya is a company that specializes on collecting, extracting data from multiple sources, multiple destinations, and integrates them with Splunk, with data dog, with open search, with elastic, Amazon S3, we have more than a hundred connections between sources and destinations. So while we don't replace Splunk, we don't replace data dog. Actually, we provide a better experience for users that are relies on this platform. So we have very, very agnostic from the back end place.

We call it back end means where we send the data for analysis. So we sit in the middle. Got it. So while companies like Data Dog, New Relic Splunk, et cetera, all of those companies generate data and also provide a way of explaining the data, what you do is you collect, aggregate, and promote the data.

Promote may not be a good word, but you collect, aggregate, and make available for use the data. Yeah, there's a new concept around, but has been there for a couple of years, what is called data pipelines or telemetry pipelines, right? If you think that they're back in as like the ocean, right, you have some way to move some water out to that ocean, right? And also the telemetry pipelines.

And Calibta, we focus on fully integration with different type of sources. And when you say sources could be application logs, application metrics, system metrics, for example, firewalls messages from security aspects that they need to be sent to a central place for analysis, because as I said, the user wants to do data analysis. But this is a constant challenge. Every company is generating 20 to 30% more data every year.

That's one fact. The other gardener just published that by 2026 in three years, at least 40% of the companies will have a solution, something like Alitia in order to control, collect, enrich, and transfer the data before sending the data to these specialized backends. And there are a couple of reasons for that, right? The challenge is that arise when people start generating data with application is like volume.

First of all, you lose control of the volume of amount of data that you have. And your backend provider will tell you, hey, please send me all the data. I will take care of it. Yeah, but maybe you are analyzing 20% of the data by your paying for the 100%.

This is a kind of a traditional approach for data processing, sending everything, and I will process it and give you the insights. Now, while we are proposing this, Alitia, and it's a new trend in the market, it's like, hey, move certain processing capabilities and certain type of analysis to the process that runs before the data gets ingested. And this is a data pipeline. So while you are collecting data from different sources or receiving this data from the network, you have the ability to say, I'm not interested in this data.

I wanted to send it to Splunk, maybe for real-time analysis, or maybe just want to send a fraction and everything else to Amazon is three, which is a cheap as a storage, right? But it's not for real-time data analysis. So all the users started a journey now that they don't think about just observability dating. What is my strategy to accomplish a good and scalable observability in the future?

And if you want to scale in all areas possible, you need to take control, control and measure. And this is our data pipelines, and Kalyta provides the best one in the market for that. Got it. So once again, companies like Data Dog, et cetera, they generate data, and rather than keeping data internal, we pass it out into a generic pipeline.

And that's where you come in. And your purpose is essentially the filter, the aggregation, the determination of where the data should go and where it should go and how it should be processed, sending out to the back ends, which might be Splunk. So you're aggregating the data, maybe it may not be processing at some level, but you're definitely filtering and deciding which data goes where. Actually, we do processing.

We can do processing in motion, whether there is some flight. For example, when you use case from one of our customers from the security space, they have tons of firewalls, and those firewalls chips don't have syslog messages. And they use Splunk to do real-time data analysis. But they found that from 200 messages, maybe five are useful for them.

So they are not sending the 200 messages per hardware to Splunk. They are sending only one that are relevant, and with Kalyta they are able to detect, oh, this is information that I really care about. So I'm just going to send this information to Splunk. Now, data sources could be a firewall, could be something that applications that you have in your own cloud, your own BMs that are generating in for logs, right?

Logs in majority of use cases, logs tells you what's the state of application, what's happening inside the application, informational errors, wire means, and so on. It makes perfect sense. There's an open source offering that does the sort of data pipelines that I'm aware of called Fluentbit. Now you're related to Fluentbit.

Do you want to tell everyone how you're connected with Fluentbit? Yeah, Fluentbit is my baby. I created Fluentbit like almost. I think it's a semi-dissimal Fluentbit.

Yeah, I created Fluentbit. The first part is called FluentD. So FluentD was created by SADA. I created Fluentbit.

I was part of the FluentD team. So Fluentbit and FluentD both project solves the problem of how to collect data from multiple sources, send it to multiple destinations, but also deal with different formats. It's not the same syslog, and an Apache log, web server, or engineer, that's all different, but at the end of the day, you need to sustain processing capabilities in order to structure the data in a way that you can do an efficient analysis of the data. FluentD and Fluentbit allow you to do that.

Fluentbit has been a very successful project, has been around for almost seven years, as I said. And I think that one of the biggest traction in the market, the primary solutions to create data pipelines, is that those projects when we work at a previous company called Trisha Data, we donated those projects to the CNCF. And I think that FluentD joined it and Fluentbit like the fourth or fifth project to the CNCF. CNC was pretty new.

It was just Kubernetes, Prometheus. I think that OpenTrace and what Jerry was there. So we are from the early days. This project is starting to get a lot of adoption and primarily because people are generating more data.

I think it's one plus one, right? Every year people generate more data. So they need to analyze it. They need to move the data.

And they found that Reliant of FluentD and now Fluentbit was the best solution for that because they were scalable. They have a plug-able architecture. And I think that one of the most important points for big corporations is like they're really vendor neutral. They have with the CNCF.

It's not like, oh, I'm an Eurelic and I own this or I'm Splunk or DataDock and I own this technology. And if tomorrow I change the rules of the game, I don't know, something could happen, right? And you might be in trouble. But by being a vendor agnostic place, this open the possibility for companies to trust and rely on this technology.

And when it's a company, for example, today Microsoft, Google and Amazon runs Fluentbit at a very, very high scale in their infrastructures. And Fluentbit has an open support. It has all the integrations possible between sources, back and vendors. And after a couple of years, I decided when my microphone there today, I'm right there.

It's time to start the next chapter of the FluentDock system. And this is Calitia because FluentDock has messy adopted in a few weeks as we were talking before, it's we're going to approach 10 billion dollars in total, which is insane. It's amazing for an open source product. Yeah.

Yeah. And yeah, so the connection between FluentBait and Calitia Calitia is just the enterprise chapter of Fluent. You know, enterprise has other needs around security, performance, out to healing, or sometimes cloud native solutions, more integration around cloud native scalability, secret management, and those type of things. So do you take FluentBait as a basis and add things like support and enterprise-grade capabilities on top of it?

Or did you take FluentBait as a starting point and create a new product for the enterprise? Oh, it's totally new product. Actually, Calitia, what we call Calitia core is a full platform built on top of FluentBait. So we're not selling a FluentBait++.

We're selling a whole platform that allows you to create data pipelines, not just for aggregation, but also remote pipelines. You can collect fleet management, where remotely you can create pipelines on nodes that can collect process and send the data directly to any place without the need for centralization of aggregation. So for example, if you have, I don't know, you have a hundred BMs and you need to process all the information from those being collected pre-processed or maybe reduce information, you can deploy Calitia core and remotely use the core agent that is based on FluentBait. And our platform through the simple UI allows you to create a data pipeline and say, this is how I want to go with the data.

This is a logic that I want to process my data and then send the results to one or multiple destinations. Now, if you do aggregation, have this concept of each pipeline versus centralized pipeline or aggregation. In aggregation or centralized pipeline is a very scalable way because it's not just one pipeline. So you can scale it up until hundreds if you want.

It depends on the load of data that you want to receive. And maybe the next question is, hey, can you do maybe the same thing with FluentBait? Yes, but might take you a couple of weeks and a couple of faults from your engineering team to manage and maintain this in a consistent way, while with the product, you can have a running in 10 minutes. Now, you're also a SAS application, right?

So people value as a service versus as a downloaded software product. Is that correct? I know there's an agent and everything, but forget again, the core of your product is a SAS application. It's a hybrid mode.

Let me explain you how our control plane leaves in the cloud in our cloud and the control plane doesn't store any customer data. So when you do you deploy core on your own instance in your own data center or your own computer, even laptop, if you want, core gets up and running it connects to our cloud and register itself and say, hey, I'm here. Now from the cloud, you can start creating the data pipelines that gets deployed in your own computer, in your own environment. So the data never goes through our cloud.

The data always flows in the way that you tell it to flow. So the data flows, the pipeline itself is all in the customer's premise or wherever they install the agents and the management of it is what goes on in the cloud. Yes, but there's another version that we just shipped a couple of weeks ago. We have a couple of customers who runs in private cloud.

Private cloud, you might think about finance and government and they have a special needs around a we cannot talk to the internet. And for them, we ship a version of Pallity Court that runs with all the components in their own environment, so they don't talk to our cloud. So you have these two type of deployment mechanism, hybrid in a shared cloud environment, but everything or your data is on-prem and the other side is fully be basic with your private cloud. Got it.

Okay. So there's an installation process to getting the product working. What do you do to make that a smooth process for people? Oh, we just provide two commands, a shell scrape or a health command and you're ready to go copy-based.

It takes two minutes to get installed. It depends on your network bandwidth, right? But just all the other get up the artifacts and everything that's ready to go. And if you're in Kubernetes, you can get a better experience.

Actually, as I gave you our solution, it's really interesting for how do we look at the future of deployments for platforms that support different type of workloads. Our solution is fully based on Kubernetes. But even if you don't have Kubernetes running, you can deploy Clicor. I would install it with recognize, oh, you are not in Kubernetes, that's fine.

I'm going to spin up a single classic Kubernetes for my own as a single process and I'm going to start doing all the magic. And if you're in a real Kubernetes cluster, it's going to solve all the components and manage all the balancing that working for you automatically. Cool. Okay.

So there's an on-premise component. There is a you're back in component. So you do the management of your and but all the data plane is entirely with a customer premises. So there's no shared data that goes on.

Okay. So that makes you very different from many observability companies like the new Relics and the data dogs in the world were very much there in control of the customers data. Yeah, they are in control when they send the data to them. We provide control for the users and send customers back in their own environment.

And at the end of the day, this is not going to get better to others. Actually, we found that our customers get a better experience with their own back in an E-ray or Splant because at the end of the day, they are sending the data gets processed in a more efficient way with the right structure. The user doesn't need to wait until all the data gets ingested into the back end and get indexed in order to do some processing or analysis because you can do this in parallel, right? And in a distributed fashion, while using Kali Techor technology.

And there's a huge difference between waiting 30 minutes to process your data instead of doing one second or less while the data is being collected in real time. Makes sense. So you know, when I think about observability, and I know you want to talk about observability and security. So let's focus on the observability side for a moment.

I think that's the broader covering, at least from this conversation. There's multiple types of observability data that you capture and that you can or can be sent through a pipeline, like such as yours. And so when I think of observability data, I think of three major classifications of data. I think of event data, which includes like log files and things like that.

There are things that have happened to your application. Here's a notice that it's happened and you log it, you do whatever you need with it. The other one is metrics. So these are point in time values.

This is what New Relic and Data Dog historically have been very good at. What's the CPU of the server? What's the amount of free memory, any virtual memory stack in this part of the application? How much free memory do I have?

How much have I used? All those sorts of things are point in time values that are part of an observability story. And the third one is tracing, which is managing a request end-to-end request all the way through a potential multiple service application and all the way back and data from multiple sources, like you're getting it together and sending it back to create that entire trace. So when I think of observability data, I think of those three separate things.

Now, different companies, I think, specialize in different types of observability data. The data dogs in the world really came from a metric background and have added tracing and logging and things like that. And you have companies like Splunk, which came from a logging space and added all these other ones in later. Few companies do them all well.

Most companies do one or two of them well, but not all of them well. Do you focus on one or two or do you have capabilities and do you focus on helping in all three of those? Or do you focus on one or two of those categories? A great question.

Actually, our platform supports logs, metrics, and traces. For example, we can collect logs. It's been historically our major support, but for metrics, we can do, for example, Prometheus scraping and send those metrics as open telemetrics. We have all these conversion layers or receive open telemetrics and send them or export them as Prometheus exposure.

So we can compare between different formats through different layers. We have a done that for a couple of years already in the open source, where we are leveraging these features, right, from a product perspective with some simplification. Now, you're right, you're saying observability care about logs, and for example, security cares about data sources and destinations like a scene period. Now, for major, we support processing capabilities too.

And now I agree with that, that everybody has been focusing on one or the other. But now we are seeing that in the market, everybody's trying to converge on that. We need a solution that gives us the possibility to correlate all the data together. They realize you need all three in order to have a full solution.

I think that need versus ideal is case by case, right? They're companies that with logs, they're fine, but just with metrics. But for example, in our case, we're finding we are going to ship a new feature in our product that allows us to convert logs to metrics. This doesn't exist everywhere.

So for example, imagine that you're receiving 1,000 logs, right? And you know the structure, you know the value that they have. But you don't care about the logs, you just want to know, for example, how many of them belongs to certain territory, how many of them have a specific value, for example, HTTP queries or so on. So it's more valuable for the user to have the ability to take those logs and write in the pipeline, be able to process them and ship a single metric.

That's like, hey, this is a value. And then you can take some action. For example, alerting, hey, our things going well, no, you said the threshold and you can do whatever you want with that. Most of the packets provide you those type of features by processing logs, but why after they have been adjusted.

I would say if I talk about where I see from all the perspective logs, make some traces of the journey to have a single platform for everything, I would say that we're still in an early stage in the market for that. But I'm glad to see that all vendors are jumping into that journey because everybody's coming up with really interesting approaches for data analysis. Now, the biggest problem is the volume of data for them. So I hear you as far as the aggregation between like converting logs to metrics and vice versa, because metrics can be changed to events by triggers.

That's essentially what a trigger is, is a notification of something happening. But one of the things I know that's always been a challenge, I know things are getting a lot better now with many open source products, but at least in the early days of tracing, request correlation was one of the hard things to do. I know when New Relic first started working on tracing, I was still in the relic at the time. That was always a hard thing to figure out how to do is how do you do the correlation successfully without adding huge complexity to your application to your infrastructure?

Is that sort of correlation something that like you can help with? In the way that we helped, for example, with traces specifically, we can receive open-to-linear payload, open-to-linear standard for tracing as Prometheus or Matrix. That's the standard. So we get the traces.

The ability that we have is to reduce the content of the traces. Sometimes they are very noisy. The lifespan has information that are not necessary, or certain spans are not useful. Same in logs, we have a lot of noise, same thing happens with trace.

Actually, trace is very, very noisy. So our capabilities is not around to do correlation, it's around data transformation on the spaces. And we let all the correlation to happen in New Relic or data. Or any platform that the customer is using.

Okay, so correlation from the standpoint of relating one piece of independent data to another piece of independent data and to find out the commonality between them and the interactions between them. You'd leave those two in the relics of the world. But when it comes to actually creating an entire trace and merging all the data that goes along with that and determining what parts of that trace are relevant and what parts are not, that's something that you can handle within the pipeline. Yes, what the user does, for example, is that the instrumented applications, for example, with open telemetry and they say, oh, this is the endpoint to send my traces.

And they use CaliCORE, we reset the traces and they created on rules to say, I want to discard certain spans based on certain content. So those spans doesn't flow to the back end. But that's all thing that we do for tracing. And the backend said, oh, I get all these spans that we tried to correlate in, oh, this is a full trace from endpoint.

But now you have less data, right? You have something that gets easier to process. This kind of reminds me of a quote I saw or that I wrote down that I saw on your website and I wanted to mention to you and I just see it on my notes here now. And that quote I saw on your website was transform complexity into insights.

And that caught my attention because IT complexity is a huge topic of mine and I've even written a book, a Riley book specifically on that topic. And so I care a lot about complexity. And I didn't really know how that applies to your product, but I think you're kind of explaining some of that now. Not correlation but aggregation is really part of that reducing complexity message that you were trying to get across with that statement, right?

Yeah, and there are a lot of complexities in this space. For example, the moment that the users need to deal with different applications with different sources plus different formats and they want to have a unified experience of data analysis, that is really, really complex. And the way that we solve this problem, we started this journey 10 years ago with Vue and D, right? Everything that we're building now is like building blocks on top of experience where we want that instead of a user focusing on how to collect data, how to transfer the data, how to send it, which is a very heavy task for many teams and we see this in a day to day with all the cells called all the PLC, yeah, people struggle with this because you want to do the analysis, right?

Nobody is waking up in the morning and saying, hey, I just want to manage my logs. No, it's not like that. They just want to strike a value from there. But manage logs, manage metrics and traces are very complex and heavy tasks, even more while now the environments are growing, right?

Growing means a number of games and that's, you know, it's like a core complexity around how hard it's for me to attack the problem in a very efficient way, right? And the only way to do it is divide and conquer, right? And we build this technology to simplify how to divide, how to conquer easily and level up the user in a way that they don't need to touch a unified anymore. They just can do a drag and drop, click a couple of buttons and you get everything that used to do in a couple of days in just minutes.

And that's simplicity for us. And honestly, we pay a lot of attention in our user interface, how the user interact with it, if we have any extra button that is not needed or extra action that will make things hard to understand, we don't want that, right? And most of our customers are really happy with the interface because it's very intuitive and they can accomplish and extract value right away without taking 10 hours of training. So I think that, yeah, so for us, we're using this complexity and we see that our customers are getting insights faster and are hitting their goal in a better way.

We have very customer focus. That makes sense. And certainly that's one of the dimensions of complexity is the complexity of the user experience. That makes perfect sense.

But I also sense another complexity offering in your mixes as well, there too. One of the things I talk about in the book is I talk about, you know, there's a trade off between depth of data and complexity. The more data you get, the more analysis you can do, the more complex the system is. The less complex it is, there's a correlation to keep a system simple often means losing analytics means losing visibility into how it works.

That's the way you make a system simpler. If you want more visibility and more noticing what's going on within the system, that typically means a more complex system. So one of the things, you know, observability in general, the idea of opening up a system so you can see more of what's going on inside is by its very nature, an increase in complexity. But in some cases, huge increase in complexity.

Using a large scale application, using fullest capabilities of like a data dog or a new relic could be very complex just to even wrap your mind around what else going on. But what's valuable is finding a sweet spot, right? Adjusting things, finding a sweet spot between how much data you want, how much observability you need into your system, and the level of complexity that goes with that. So I see a product like yours is something that's able to fine tune that adjustment, if you will, change the amount of observability data and therefore change the complexity.

As kind of a feature of what you do, users can change that. And you know, I want to send this level of detail to this application, you know, I'll store all the integrated detail in S3 for later analysis if necessary, but real time, I just need this high level summary and for learning, I need a high level summary, et cetera, et cetera. Those sorts of things, you're essentially making decisions on trading off detail for complexities, what you're doing. And do you see it that way or am I missing something in that analysis?

Yeah, there's an interesting piece in your analysis of what you said, what the user wants and what the user needs. What we see in our daily work is that we know that they went to analysis data, but when it's about the what, most of you still doesn't have any idea how much information they have. That's a reality. Unless you start doing some inventory of control, okay, how much data have we have on customer that they had a lot of built in house tools to collect the data from their applications, and they thought that they were processing a couple of three hours per day.

So we changed the logic, we put in the product, and now they get surprised because hey, now we're collecting more data that we used to before. Yeah, you were losing data. You didn't know that you have more data before. This is what you have, this is for real.

And now when they get into this discovery process like, hey, I take control, I understand what I have. Now I can take some decisions on what I need. Right? So the need that you have before taking control might not be the need after you put some tool in place and start analyzing, hey, this is all what I have.

Because the moment that you decide to implement a telemetry pipeline solution could be for security or for observability, you get more control, you are able to process and better more data, and you have a more holistic view of why you really have your environment. And you're always against time because you know that months over months, your data is growing. The amount of data per minute is more than the last month. Right?

So if you don't stop and you train yourself and put a solution, it will be hard. So everybody has an idea until they get a bunch of the face, right? There's something like that. And I think that in this space, having the same data, we think we have.

Yeah, yeah. And you know, the other more data doesn't correlate with value. That's the biggest problem. The more data you have, it's hard to find the value.

So you have to clean it up. You have to make sure that all these data is completely argument is, yeah, it was too complex. A complex system can't be analyzed easily. And so if you have so much data that makes your system so complex, you don't know what the data is, it's not useful to you anymore.

Yeah. And you know, you might be surprised how much money companies and user space to store all this data that, oh, I need to store it because maybe I need it, right? But if you go one step forward and put some control, some solution before that, you might be in a better shape for what's coming from a data grow, data volume perspective in the future. But yeah, there's a lot of story from different users.

But it's interesting. It's interesting, probably the same problem that we used to have 10 years ago, we have it again with volume, it's hard to start the stuff. But I think the work cannot rely on the working that's used to working from a couple of years ago. I think that everything is shifting more to the left where the left is where the data is being collected, where data is being processed.

There's a ton of value on that. Yeah, I'm happy to say that. Yeah, I've decided that more emerging companies also try to keep this space with their own approach. There definitely is.

Yeah. Yeah. And it's like what you were saying to about wasted data. I don't think there's any more expensive five words in English language than maybe I might need it.

Those five words have been cost a lot of people a lot of money often. And so, you know, I agree with you. When I worked in the reality, I used to say that most applications have significantly more analytics data than they do business data. And that always shocked people.

And I always thought afterwards with the same, you don't realize that in some cases, it's even in order of magnitude more analytics data than they do business data. It's huge quantities of data that you're collecting. You may not even know about it. And it's just what you do with it all is a complexity, you know, and obviously the reality has their solutions for doing it, etc.

But when I was working there. But the point being people were always shocked to hear that most companies have more analytics data than business data. And it's a significant difference. Yeah.

And I see that most people discovered this by practice in a couple of years. I think it's an interesting journey. It is an interesting challenge every day around data, getting the data, getting the insight. It's really hard.

But everything in the lab is to companies to decide are they going to take a strategy to fight the back or not? And if they're not, yeah, we'll have to do it in the future at some point. But there's a lot of- That makes sense. Yeah.

There's a lot of interesting journeys about ability operations, you know, and security. I know we're running short on your time. And I appreciate your hard stop coming up here. But we haven't yet talked about security.

So I'm wondering if we could talk a little bit about security. Now, I was going to talk to you about the difference between SEAM and SOAR, because I know you play a little bit in both places. But you're primarily a SEAM offering. Is that correct?

Now, just for everyone who's listening and if you're not familiar with those terms, SEAM is let's see, security, information, and event management. It's the events that happen that can cause your business harm from a security standpoint. While SOAR is security orchestration automation in response, I guess, is the R. But that's more about the systems that you build up that manage the security of your applications and what the processes are for doing that.

So SOAR is more of a preventive and SEAM is more of a notice of detection. Do you consider yourself more of a SEAM company or more of a SOAR company? Or do you kind of play in both? Oh, it's a bit of a we're both.

We play both. The thing is, is the following. We have our base product that has two ways to be consumed for security and operations and for stability. For example, if I can do a comparison between one of the other, for example, in security, you care about security sources, like firewall, security messages that you should send to a SEAM.

If you end up in the space, maybe you are a devops person or SRE, but you just care about the logs, make sure some traces. You just care about that and that's it. In the other site, for example, in security, you care about parts of four firewalls, windows logs formatting, processing, everything that's around processing to save costs on your SEAM. While in observability, you care about data enrichment, metadata, removing traces, devops logs, which are, you might say, oh, it's kind of similar, but a different person, a totally different, for example, in security, the buyer is a CISO, right, the chief of security.

While in observability is usually the VP of engineering or now there's VP of observability is a new title. And the problem is like, for example, in security, if you want to ask am I security or observability in security, because the major problem is like, oh, I cannot connect my data sources. I cannot extract this information from this application because this is relevant, because it could be a security threat or if it's been hacked, that information is there and I'm not be aware of that, right? It's really intensive about connectivity or integrating different systems.

Well, in observability, if I cannot connect my data, the problem is different. I cannot serve my end customers. I might not have my high available services or so on. So the spectrum of problems and person are totally different.

But yeah, data is there, right? The way that you consume it, in the way that you want to strike value might be different based on your own needs. So from security system, what is different? It's different, but the data is still the same.

Yeah, well, data is there, just a round of bytes. So real quickly before we stop, you list a ton of integrations on your website. Do you want to mention some that you think are the most important for your customers or the most critical ones? Obviously open telemetry.

But which ones are the ones that the people listening are going to be more likely to be interested in? Yeah, for example, I would say the common one is Amazon S3, which is a very cheap object storage that is common across majority of our customers, they use it. Well, at least if they are not in the BBC, they don't have access to that, they might not use it. The other are Splunk, very highly used.

Most of our customers use Splunk, they're mid-sized company, big-sized companies, HTTP to receive events over HTTP and send over HTTP. Now, open telemetry, just for tracing, but I would say it's not that hot as logs in our case. I think majority of these cases that we're solving today are around logs and metrics, few ones with traces. So still things like syslog integration is more important than open telemetry in most cases?

Yeah, the thing is that you have to solve the problems for the problems that you have today, not for the problems that you want to have in the future. Now, a lot of companies might read into open telemetry to use to do logging with open telemetry and do metrics with open telemetry beside traces, kind of a standard. But the industry is still running, we are very honest, to move a logs is fluent, to move and process metrics is per meters and traces of inter- And there's still you have thousands and thousands of machines running syslog or five-walls sending messages on syslog, you have to connect those devices. And now when we see that companies work in hybrid mode, they have their old system, but also they use the cloud.

So they want to leverage and be able to collect data from every single source of information. If you have, well, as I said, five-walls a couple of times already your devices, application logs, or even if you're running an instance on AWS, you might want to extract application information logs and enrich them with metadata. Like, oh, this was running on AWS server with this label or with this tag. So then you can correlate information based on the type of environment where this was running before.

So yeah, it's a bit of everything. Anybody who's just trying to say we're going to standardize on this protocol and our solution is just for this protocol. Yeah, that's fine. Maybe you're looking for customers in five years.

But today, everything is about syslog, HTTP, you know, it's planned, even elastic, you see it a lot too. So there's a mix of everything. Great. Well, thank you very much.

So my guest today has been Eduardo Silva, who is the founder and CEO of Cliptia. Cliptia, right? Cliptia. Cliptia.

Let me tell you why. If I can tell you why, because the original word is called Caliptia. Caliptia, right? T-H-E, okay?

And Caliptia is a genius of hammingbirds. You know, that's going bit, low is a hammingbird. So Caliptia is a genius of hammingbirds. But the way you might pronounce Caliptia in Spanish sounds Caliptia.

That's why we should just Caliptia. Yeah. Makes sense. So Caliptia.

Okay. Yeah. Perfect. Anyway, Cliptia is a company that helps you manage your observability and your security data.

And once again, it's Eduardo Silva, who's the founder and CEO. Eduardo, thank you so much for joining me today on Software Engineering Daily. No, I appreciate the invitation. I would be happy to have another conversation anytime soon.

Thanks again.