Prometheus Querying.
Querying Metrics Introduction.
Prometheus provides function expression language
called PromQL.
It provides built in operators and functions.
It does vector based calculations, just like excel.
And it does expressions over time-series vectors.
Another important thing to remember
is that PromQL is read only.
So you can't insert data via PromQL.
Here's an example of a query.
I will go into that deeper in the next slides.
Prometheus Querying Expressions.
In Prometheus, you got an instant vector.
That is a set of time-series
containing a single sample for each time series,
all sharing the same time stamp.
An example for that is node cpu seconds total.
A range vector is a set of time-series
containing a range of data points over time
for each time series.
Here you see the same node cpu seconds total.
But then, with a time mark behind it.
Like this is for five minutes.
You also got scalar.
It's a simple numeric floating point value.
Example, minus 3.14.
You also have a string.
You also have a string. But this is currently unused.
Example, foobar.
PromQL delivers a very rich querying language,
which is operators.
Arithmetic binary operators
like subtraction, multiplication, division (mumbles),
power, exponential.
You got also comparison binary operators
like equal, greater than, less or equal.
You get the picture.
You can also, logical set,
binary operators like the and, or, unless.
But you could also a very rich aggregation operator.
So you got the sum, the minus, standard deviation,
standard variation, the topk, the quantile account,
and many more, which you will see this in the demo.
As there is no better way of explaining
how to query Prometheus is by actually querying it.
The proof is in eating the pudding.
So we're going to do that.
Let me switch to our Prometheus server.
So, we navigate back to our Prometheus server.
We go to craft, and then we type in a query.
So, lets start with returning all the time series
with the metric http request total.
So we going to go http underscore request,
request total, here you go.
And lets execute this.
Here you see all the metrics that it has.
With the value behind it, so how many time it got called.
You can also easily graph it.
And here you see the legend.
Another cool metric is up.
Execute.
This just shows one,
but this means that Prometheus' job is still running.
But as you can see in the legend,
also the node export was running.
So if you wanna say, I only want to see the Prometheus job,
it's easily done.
You just job equals Prometheus.
Execute and there you go.
Now you only have the up metric for the job Prometheus.
You can also use regular expressions.
You could select time-series
only for the jobs who matches certain criteria.
Like we did with the up metric, but now,
lets search for all the http request total.
Http request total.
Where the job ends with etheus in the name.
So we're going to job equals till the quote quote
and then we're going to do beginning, all characters etheus.
So lets see if it can find something.
There we go.
Now we have filtered everything that ended up etheus.
We can also return a whole range of time.
In this case, five minutes for this same vector,
making it a range vector.
So, I'm going to do, Prometheus,
And we're wanna see the method.
Get. And lets show only five minutes.
Execute.
Oh, made a typo.
As you can see here, this is not an error.
Because, you define a range of values
and this is not drawable.
So we need to switch the console,
and here you see all those values in the time we know.
So there's no way to actually draw this,
but you can extract the info that way.
In the next example, we're going to select
all http status codes except the two .x.x.
So, all the values request, we're just gonna leave them
and we just want to see everything that is not two xx.
So, this can indicate, lets say if you are monitoring for
500 errors, you can easily do it this way.
So I'm going to check the code.
So this is the code here.
Exclamation mark till quote quote.
Now we gonna do, two, whatever value, whatever value.
And lets see what pops up.
Oh, here we go, we got a lot of 400 requests.
And you can see how beautiful it is drawn here.
So lets say we don't want to see everything else
except the 400.
You can do it that way.
There you go. Now you see everything that is not a 400 code.
Now I'm going to discuss using function and operators.
So, to return the per second rate of all time-series,
with the http request total metric name.
As it's measured over the last five minutes.
So, to do this, we gonna do rate and we're going to do http,
http request underscore total over the last five minutes.
And it doesn't find any data points here,
but that's because I made a typo here.
So lets do that again.
Here you go.
So here you see all the http requests.
And we're assuming that the http request total time-series
have all the correct labels.
We can also sort them by job.
So, lets do sum by job.
And put that between break like that.
Here you go.
Now because we only have one job running,
we can easily see how many http requests
are total in five minutes.
It's getting,
as you can see here, we started to our laps here.
So, this is the data.
You can also apply a binary operator
like discussed in the beginning of this section.
So in following example,
we're going to apply a binary operator.
So we're going to calculate
the actual memory that is being used.
Node memory MemTotal bytes,
so we got the total amount of bytes.
And now we're gonna subtract it from, let me copy this.
Node MemFree underscore bytes.
If I recall it correctly,
can always see what is available here.
MemTotal btyes.
Yep.
And, we're gonna do that. Like that.
And then we're going to divide by 1024
and divide it again.
So we go from bytes to kilobytes to megabytes.
So, you know, how much of memory
is being used on your server.
Here you go, so yeah.
It's not that much.
So yeah, the machine is still okay.
Lets say, we want the top three CPU users
grouped per application.
You also got operator top.
Topk three sum rate,
now I'm gonna do node cpu seconds total
over 5 minutes and you can sort that by up,
remove this.
'Cause we only have one app running,
you only see only one of course,
but this normally should show you more than one.
If the label is applied correctly.
All right, so I encourage you to go to
the Prometheus website.
Look at all the options you got
and try to match a query on your use case.
There are some really good examples on that website,
so makes sure to take a look at them and try them yourself.
As I said in the beginning of this section,
the proof is eating the pudding
and it's really easy to learn
when you're actually doing it yourself.
I hope this help you to use the PromQL query language.