Using Telemetry to Test Your Application Across Customer-Representative Environments

Chuck D'Antonio and Kaylee McHugh
 | 
Nov 28, 2023

Our Director of Solutions Engineering, Chuck D’Antonio, walks through how to use Replicated’s Instance Insights (telemetry) to set up a GitHub action that tests a vendor’s application across their customer-representative environments. Watch the video or read the transcript below.

Chuck: The Replicated platform has a lot of great features, but two of them that I spend a lot of time talking about lately are the Compatibility Matrix and our Instance Insights, which are driven by the telemetry we can collect from the clusters that your customers are running your applications in.

When you log into the Replicated Vendor Portal, you will see the list of customers that you have and you'll see information about each of their clusters.

When we click into any one of these, we can see details about the cluster: like the type of Kubernetes distribution, what version of the replicated software they're running, information about your software they're running, etc.

In and of itself this is pretty cool! One of the things I love to use it for is to quickly understand “What is this customer that's having trouble really doing? What version are they on? What environment are they running?”, and all those questions that I might spend a bunch of time asking in order to help them out. 

The other feature that's really cool is the Replicated Compatibility Matrix. 

You can see I've got a bunch of clusters running here and they're all different types of clusters: there’s GKE, there’s OpenShift, there’s AKS, there's even one in here that looks like it's K3S. So, this is great for doing your testing and understanding whether your application's going to run the way you expect it to when it gets out there to your customer environments.

But, what really makes these two things powerful, is when we bring them together. I'm going to show you how I did that here by looking at this GitHub action.

In this GitHub action, you can see I'm doing something very simple. I'm packaging my chart, and then I'm running some tests that I'm calling “customer representative testing”. In the middle there, though, I have this step: “get customer instances”. What I'm doing when I get customer instances is I'm going out to the Replicated Vendor Portal, calling the API, and I'm getting those distribution and version numbers that are showing from my customer telemetry. Then, I'm using the matrix feature here in GitHub actions. 

A lot of different CI solutions are going to have something similar. I know GitLab does. I know Concourse does. I'm sure all of them do. And I'm passing the results of that into that so that I'm testing against the types of clusters that my customers have.

If I click into this one, you can see I tested EKS 1.25. So, I've got a cluster that's running EKS 1.25.

If I click into this one, you can see I'm running K3S 1.24.1. So I'm running what the telemetry is telling me I need to in terms of test environments and I'm testing my application in all of those. 

The best thing about this is that it's going to surface things like this K3S 1.24.1 that I would not think of when I was coming up with a test plan.

When I'm coming up with a test plan, I'm going to use things like GKE, AKS, EKS, probably OpenShift, maybe Tanzu, and then I'm probably going to drop off there and I'm probably only going to use one or two versions and those are going to be the newest and latest versions of those.

But here I have a customer on a fairly old cluster of a distribution that, in my initial thinking of a test plan, might seem like a little bit of an oddball and my tests are just automatically running against it because that's what my telemetry says.

It's a really powerful way that these two features come together and provide more value from the Replicated Platform for you.

Interested in trying this out yourself? Sign up for a free trial or request a demo anytime.