Prometheus client libraries.
Before you can monitor your codes,
you should instrument your codes.
Prometheus is a great project to do this
as it has a lot of official libraries,
like Go, Java, Scala, Python and Ruby.
There are some unofficial libraries,
but they're supported by the community
for Bash, C++, Erlang, Haskell, Lua,
.Net, Node.js, PHP, and Rust and many more.
If you're using something,
and there's no client library available,
you can implement it yourself in one of the supported
exposition formats.
You get two exposition formats,
depending on what version you're using.
In our version, we used in the labs,
the protocol-buffer format is no longer supported.
So it should rely on the simple tech space format.
You see an example of what an exposition format is.
So you got a metric name, a label name, label value.
You've got for types of metrics.
You've got a counter, gauge, a histogram, and a summary.
Counter is very simple.
It's just a value that only goes up.
Think of visits to a website.
Gauge is a simple number value that can go up and down.
Think of CPU load and temperature.
Histogram, this samples observations
like request of duration or response sizes.
And these observations get counted into buckets.
This includes underscore count and the underscore sum.
Main purpose is calculating quantiles.
A summary, similar to a histogram,
a summary samples observations.
Think of request durations, or response sizes.
A summary also provides a total count of observations
and the sum of all observed values.
It calculates configurable quantiles
over a sliding time window.
So for example, when you wanna calculate the latency,
you need two counters,
the total requests and the total latency of those requests.
You take the rate of the total requests,
and the rate of the total latency,
and divide them and you get the average latency.
Prometheus Instrumentation-Python.
As you can see, the client library's also part
of the Prometheus project and can be found on Github.
It's officially supported.
You can easily install it using
pip install prometheus_client.
The supported metrics are Counter, Gauge,
Summary and Histogram.
Here's a little Python Example.
We start by importing the Prometheus client
in order to be able to instrument this application.
We created a counter
to count the total number of HTTP requests,
a gauge which can go up and down
to monitor the total number of in progress requests,
and a histogram to measure the latency of the HTTP requests.
This some standard flask route stuff
and this is a helper annotation to measure
how long amount it takes,
and save it as a histogram metric.
This is a helper annotation to increment a gauge
when entering the methods and decrementing when leaving.
And all metrics can have labels.
This allows grouping of the related time series.
This is the end point which exposes the metrics
to Prometheus whenever it comes by ascribes those metrics.
Prometheus instrumentation with Go.
As you can see, Golang is also
officially supported language.
It's also supported by the Prometheus project,
and you can also find it on Github.
So it's super easy to implement.
It just import client_golang
and on with the simple function,
you can already expose some basic metrics from Go.
The supported metrics are also Counter, Gauge,
Summary and Histogram.
As you can see, some custom metrics are created
via importing this library.
Just an example for a gauge, for the numbers of jobs,
currently in queue.
To include the new metric in the list of exposed metrics
in HTTP handler, it must be registered
in the default registry.
Don't forget this as you can easily spend a while
figuring out why a metric's missing from the end point.
In a previous example, there were no labels.
But as this is best practice, you can best add a label.
So you can do it by this, by a new gauge, back
and then you add the string job type.
The instrumentation must now specify the values
for the label in this case the job type
before incrementing or decrementing the gauge.
If specifying multiple labels,
the label values must be passed in the same order.
And this is important because all the times here,
it gets mixed up, as the labels were defined
when creating the metric.
A histogram, measuring the duration of things
happens inside of application is usually a good way
to get some insights of the performance of your application.
You can easily detect anomalies with it.
Duration should be analyzed throughout our distribution.
So not the averages, which means the ideal metric type
for this is a histogram.
Histograms observe all the values and count them
into bounded buckets.
These buckets will be defined for the histogram
as creation time and should be tailored
to what is being measured.
Otherwise, yeah, the metric will have very
too little to use.
The typical profile for a job run by this application
is in the order between one to five seconds.
Think of a batch job that calculates invoices for example.
And it usually takes between one to five seconds.
Good buckets could be one second, two second, five second,
10 seconds, 20 seconds or even 60 seconds.
The histogram is then used inside the code
to observe the duration of each job.
A summary,
in creating a summary, it's somewhat similar to a histogram.
The difference is that Prometheus.NewSummary
must specify which quantiles to calculate
instead of specifying those buckets.
In the next chapter, I'll be talking abut pushing
metrics using Prometheus.