Demo, Monitor and instrument
a Spring boot application.
So in this demo, we'll be setting up
a digital ocean Droplet.
we'll be installing Prometheus, we're installing profana,
we gonna make some configuration changes,
we gonna deploy and build the application using Docker.
So it does a lot of exciting stuff,
that we gonna do in this chapter.
So alright, let's get started.
First up, we gonna start by creating a Droplet.
In case you already have a Droplet running,
you can skip this part where we actually
start to configure it and you can follow from there.
But in case you haven't installed the Droplet yet,
I'm gonna show you how to do it.
So first up, we go to the digital ocean site.
And we gonna do Create Droplets,
we gonna select
T 16.04.6 six version.
We gonna use this one,
we can also use this one doesn't really matter,
but I'm just gonna use this one because it's a bit faster.
I'm gonna deploy it in Frankfurt, you can choose a region,
which is closest to you and public key here
and we're gonna call this instance brought me tears.
And now we gonna create a Droplet.
There we go
and our Droplet has been created.
Let's copy the IP and then do SSH.
root at the IP address dash i to provide the SSH key
dot SSH dot id underscore rsa.
And this will use my private key and public key
to actually connect to that instance.
Yes, please.
And there we go, we've logged in two times on our machine.
So the first step as always, is to get to our course
GitHub page and git clone our repository here.
So let's grab that.
So I'm gonna do clone, change this to HTTPS.
There we go.
Go back here.
And then we're gonna do a git clone.
We go now we got to training here,
So that contains all our files, even our source code.
But first up, we're gonna go to scripts.
And you see there are a lot of files here.
First up, we're gonna install Prometheus.
It's gonna run this script
'cause this is a completely empty machine.
There we go now it's installing Prometheus.
To root power and easy.
Another thing we're going to do right now
is to install Profana.
Another thing we're going to do is install Docker.
And then one thing left for us to do is to add
some configuration changes, but we'll do that later on.
It's just to not confuse you.
So we chose these scripts because this eliminates
a lot of headache when you're trying to debug something
when something doesn't run or work for a student.
So in case we see an error, we can adjust those scripts
and then it works for everybody.
That's the reason why we use a script,
you can always check out what's in there.
They're pretty basic, but nevertheless like,
if you see here, for example, the Docker script,
just make sure that we use the same Docker version
for all our students in case something goes wrong,
we just adjust this whenever the student reruns a lap,
it will be fixed for everybody.
That's why we chose to do it that way.
For now we can start with another interesting part.
That's the application.
So prometheus course,
going to the Spring boot promethues.
And as you can see if you're familiar with Java,
you'll see here a lot of files and some testing
and some POM files.
First up, let's start with the POM file.
So basically, we get a group Id, artifact I'd
and get the snapshot dissolve, it looks familiar.
We got a parent which is spring boots, as you can see here.
But the version two dot two dot six, nothing special here.
We get some dependencies and look at the Spring boot
actuator, the micrometer registry prometheus
and a Spring boot Starter tests.
So that's not all,
super important, but it's just that you know how this works.
As you can see here, those are the two important things
for this demo.
The rest is just needed to do and start our Java app.
So what is also a little bit different is that I chose
to create this with a Docker file.
So don't be scared it's not.
If you're not familiar with it, it's actually super easy.
So basically, I'm gonna build the application
with a Maven container.
As builder as you can see here, we're gonna create
that and then end the runtime container.
I don't need all those tools.
So then we can just use an open jdk 11 slim containers,
so it's super fast and super lean, it's not bloated
with all that Maven stuff in there.
So it basically, you build your jar in here.
This when you build a container, will fetch the jar
out of this build step, build it in here, package into one
nice small container and runs it.
So all right, that's another important thing I wanted
to mention, Let's open the readme file now.
I made a beautiful readme file.
So maybe it's best that I go to the prometheus course
here as well to our Spring boot course.
There we go.
And what are we gonna do is,
so basically we're gonna build the container first.
I got a one liner here, if you don't wanna wait.
So basically it builds a container
and it runs it afterwards,
but you can also do it in two separate steps.
So maybe that's not a bad idea.
So let's start building it in the meantime,
I can explain how this actually all works.
As you can see here, we got an application.
Nothing fancy here, just the basic application
and there we got an application controller.
So as you can see here, I added some io micrometer things,
the annotation times, instruments counter,
instrument metrics.
So we get a lot of stuff here.
In here, what we're actually gonna do is, the first thing
is to actually create a custom metric.
This is just to give you an example of a custom metric.
And this is actually a runCounter.
So we can count how many times this API has been invoked.
Simple, it gonna lock something and it's going to return
Hello world, so nothing fancy there.
In here, we got the same thing, we got an API delayed demo,
but this is delayed because I'm gonna wait, a random amount
of time before returning the actual message.
So this is just to give you some variation and to draw some
beautiful graphs because else it's not that interesting
because it locally, it responds super fast
and it's hard to graph something that is so fast
so it would be kinda stupid to display it.
That's why I chose another route API slash delay demo,
so you can see actually, how it looks when it's slow down.
All right, so this is basically our application.
Another interesting thing is the application of properties.
So in here, you got a management metrics tax application.
So this is basically gonna put a tag on the application
prometheus demo I put it hard coded due to it can make
a variable like you want, you add a name here.
So this is basically, so we can use our default dashboard
in Grafana this one,
so this's very important that we can use
that because it's a standard JVM dashboards,
it gets a lot of metrics out of the box.
But it needs some proper tagging,
in order for those functions to work.
That's basically the kind of workaround.
Then we got some management endpoints,
which are enabled by default.
And we're gonna set that to true default, it's false.
So it's not gonna display your, any endpoints.
Then we're gonna expose some of those endpoints.
We're gonna expose actuator, expose prometheus
and we're gonna expose the health endpoint.
There we go, so actually our container is almost built.
So I see is finished with building the jar.
So it basically, it's now pulling the open jdk container,
so you can insert a jar in that target
and leave it all right, great now let's head over
back to our readme file.
So we built a container,
now we are going to run it.
There we go, so we're gonna run the container.
And there we go.
And it's actually running, that's a good idea.
Gonna copy an endpoint, be sure your container is running.
And gonna show you what is actually export.
Quit that.
Then you do a curl.
You see, I ran a curl on that endpoint, it says,
Done waiting, waited so many microseconds
and you're gonna display it in the log here as well.
So let's run that a couple of times just to
generate some graphs.
We're gonna also use the not delayed demo,
so you can see the difference.
So it's super instant, that's why I chose to make a random
function out of the delay tone because also it's really
hard to graph.
Alright, so now we generated some metrics.
Let's check the actuator endpoint.
Actuator slash Prometheus.
There we go.
And as you can see here, this probably looks familiar.
This is actually the micrometer translating all those
actuator data points into something
Prometheus actually understands.
So in this directory, if you go back to the scripts
directory here, I also added the add Spring boot app.
So let's check what's in there.
So basically, we're gonna add a Spring boot demo app
job name, we're gonna scrape every five seconds.
The metrics bot, it's like I just showed you,
is the actuator slash prometheus.
And the targets are localhost 8080 and then it's gonna
append this to that path
and then it's gonna find those files.
We're gonna do an echo of it
and put it into configuration file.
This is just so nobody makes a typo here,
like I explained earlier.
So let's run this.
So I'm gonna add spring boots app,
super fast.
What did I actually do, I'm gonna cut the file
and as you can see here just abandoned it here.
Now the only thing for us to do is to restart prometheus
plus, prometheus doesn't refresh it's configuration file
automatically, so we're gonna do service prometheus restart.
So super fast, it can also do a status if you wanna see
what it's actually doing, as you can see,
there were no configuration file errors or whatever.
It just started the active running so everything is okay.
So basically now we can check the machine, we're gonna do.
So as you can see here, opens on the url,
Port 9090.
I've opened prometheus and as you can see, it's running.
Let's check the targets.
So as you can see the endpoint is up and running.
And we're getting metrics alright.
Now in order for us to actually display something
we can graph here, but let's move over to Grafana
and do it directly over there.
First up, let's log into Grafana,
this is also freshly installed admin.
Admin,
new password.
Save the password and now let's add a data source.
Gonna do promethues.
It's HTTP.
Localhost in this case Port 9090.
We're gonna call this promethues everything is default,
Save and Test and the data source is working.
So that's always good.
And then we're gonna import a dashboard.
So in the url I showed you earlier,
we gonna take this dashboard.
So we can just go profana.com slash dashboard, slash 4701.
And we gonna do load.
As you can see JVM micrometer.
This all sounds familiar, we're gonna import
it in the folder General, unique identifiers,
prometheus data source will be prometheus
and now everything is in order to import.
And as you can see here, right out of the box,
we got so much info.
That's why we needed to set that label correctly.
See here prometheus demo, that was the label
that we set correctly over here.
Let me show you again.
Spring boots prometheus.
Here in this file,
see as the prometheus demo.
So as you can see, I've entered a couple requests here
and some, took some longer time like I said,
some are really fast.
There's not a lot of data here of course,
but you see the points coming in right?
Like it's just that it's up since really slow,
so let me put it in the last 30 minutes
or so which actually
can see something and as you can see here,
the duration was small and some slower and some super fast.
So but that mainly gives you a point of the actual
JVM metrics that is collecting.
So there's a lot of here, feel free to browse around,
and see what actually can be done.
But it's just to give you an overview of what you can do.
You can also see the CPU, the load on the machine and so on.
So you can make a really cool thing about this.
Now we're not here for that,
we're also gonna create something new.
Just discard this and let's create some actual other data.
So now I'm gonna create a graph.
So because we are now interested in something else,
so I'm actually interested in amount of requests
that we are getting per second over all our API's.
And so first up, we're gonna use a graph.
That all looks alright.
Okay I'm then setting alerts or going here
and we're gonna add it up promQL query.
As you can see here, let's edit it.
I'm gonna set that to the last 15 minutes,
so you can see something.
And as you can see here, we got some few requests per second
coming in actually, so it's not much because the API
is not used and it's using it for all the endpoints.
So spring put demo app, as you can see
the actuator slash prometheus.
So this is actually promitheus querying itself
this is why it's going up while we're not running any API's.
Also this one as you can see here and also this one.
But actually there's not a lot of traffic,
I only hit it a couple of times,
so it's not gonna display a lot.
But this gives you an overview of what you can actually do.
Now if you're more interested,
let us generate some more loads.
So let me find a curl again.
API demo, API demo.
Let's hit it a couple of times.
It's gonna display the Hello world,
and you can see they delayed
one as well.
So there's not that much to see.
So it'll just wait around the time before,
returning the message as you can see,
right now let's head back over here.
And now we actually are going to add something here.
So basically, we're gonna open and close the curly brackets.
And we're gonna use the uri.
So I'm only interested in a certain path, for example,
being the API demo.
And I'm only interested in the status.
As you can see, it's super easy that it order and place it.
Has a status of 200.
Only I'm interested in the successful requests per second.
Let's check this, up as you can see here, the API dot demo,
I get some quite some data coming in right now,
because that's me running it.
As we go see here in the delayed slash demo,
can see the same thing,
same pattern, because I also use that.
But in the application, we also integrated something cool.
And I need to show you.
In the application controller,
we implemented,
the runCounter dot increment here.
So basically as you can see here,
we're gonna instantiate the runCounter with this,
which is a metrics dot counter,
which runCounter.
And actually, we're gonna increment this value.
So that's super easy and super useful to actually check
so you can actually implement some business logic there
but you can also send to prometheus to act on
when something strange is happening,
some large amounts of money getting wired, some large orders
getting done in middle of night, something like that.
You can choose whatever.
So in order for us to actually do that,
it's super familiar than what we've done earlier,
and basically we're gonna,
let us see we're gonna do the run.
As you can see, we already completed it again,
to runCounter total and let's check it out.
So basically, we also detected that it's a counter.
So that's super easy.
But you can add a rate for example and then you can see
how many times the counter has been hit per second
over the last five minutes.
This is not very useful,
in our case, we just want a counter for example.
Then you can see here that we actually started running it,
running it, running it and it displays the actual counter.
So you can also add it and add some alerts to it
if something goes too high or whatever.
Another interesting thing to graph,
is in the previous one, we did we wanted to know
what the counters, as you can see those count.
And see that's always in the going in an up line.
But it's just like the amount of HTTP requests
that come in per second.
But Prometheus also will display the seconds,
request seconds some.
And that's also an interesting one, as you can see here.
So basically it will show you the requests,
which are coming in per second.
And it's something really useful.
So for example in our API delayed demo app,
you can easily see how the API is behaving
over the last two minutes for example.
Like that and then you can see how many requests,
what the duration was of the request,
in the last five minutes per second,
so it's also very useful that you can query this way
because then you can see how your API is behaving.
I hope this gave you some info about how to use it,
how to instrument it and why you should do it.
The demo application is also on our GitHub page.
So it's really useful that you can check it out
and start playing with it, start adding some metrics
and just try around.
Also please make sure to look at the micrometer
documentation and the Java Spring boot documentation
on how to implement metrics and what makes sense.
This is just to provide you a framework that works
so you can start experimenting and gain some time
on how to actually use this.
Alright I really hope you enjoyed it
and see you in the next lecture.
Bye.