5. Management APIs

In this lesson, we have taken the habit to start over Prometheus Server and the AlertManager every time we changed the configuration files. This is fine in our case because we do not need constant monitoring but in most cases, Prometheus operations need to carry on.

Prometheus actually exposes Management APIs to do so:

To enable reloading, we have to launch the Prometheus Server with the --web.enable-lifecycle`.

For example, to reload the Prometheus Server configuration file, we could have done:

# in one console
prometheus/prometheus --web.enable-lifecycle --config.file=prometheus/prometheus.yml
# in another console
curl -X POST http://localhost:9090/-/reload

References

If you want to retrieve easily some information to make your own Prometheus Server and Alert Manager, the official links are below:

• Prometheus configuration file
• Recording rules reference
• Alerting rules reference
• PromQL reference
• Prometheus exporters and clients

We have not dealt with the security issues either: Prometheus handles basic authentication as well as more secure means. The references cover these topics.

6. Conclusion

Prometheus is a very interesting tool when it comes to monitoring multiple services. As Data Science projects dwells more and more into micro-service infrastructure, it becomes vital for a Data Engineer to be able to monitor APIs, Databases, Web servers, applications in a simple and efficient way.

Thanks to its PULL mechanism, Prometheus requires a lot less engineering than other tools with the same purpose. Moreover, we can find a lot of Exporters that can fit in with most of the applications that Data Scientists use. Its client library are also very useful to customize and expose your own metrics.

With the Alert Manager, important moments in your infrastructure are notified right away and we can deal with intricate amount of information without getting flooded.

One of the downsides of Prometheus is its lack of a good dashboard interface. It seems that it is by design as Prometheus chose to make it integrate very well with Grafana instead.