Monitoring Linux Logs with Kibana and Rsyslog

Monitoring Linux logs is crucial and every DevOps engineer should know how to do it. Here’s why :

• You have real-time visual feedback about your logs : probably one of the key aspects of log monitoring, you can build meaningful visualizations (such as datatables, pies, graphs or aggregated bar charts) to give some meaning to your logs.
• You are able to aggregate information to build advanced and more complex dashboards : sometimes raw information is not enough, you may want to join it with other logs or to compare it with other logs to identify a trend. A visualization platform with expression handling lets you perform that.
• You can quickly filter for a certain term, or given a certain time period : if you are only interested in SSH logs, you can build a targeted dashboard for it.
• Logs are navigable in a quick and elegant way : I know the pain of tailing and greping your logs files endlessly. I’d rather have a platform for it.

Python Modules and Packages

There are actually three different ways to define a module in Python:

1. A module can be written in Python itself.
2. A module can be written in C and loaded dynamically at run-time, like the re (regular expression) module.
3. A built-in module is intrinsically contained in the interpreter.

A module’s contents are accessed the same way in all three cases: with the importstatement.

Absolute vs Relative Imports in Python

A Python module is a file that has a .py extension, and a Python package is any folder that has modules inside it (or, in Python 2, a folder that contains an __init__.pyfile).

What happens when you have code in one module that needs to access code in another module or package? You import it!

Memory Management in Python

Memory Is an Empty Book

You can begin by thinking of a computer’s memory as an empty book intended for short stories. There’s nothing written on the pages yet. Eventually, different authors will come along. Each author wants some space to write their story in.

Since they aren’t allowed to write over each other, they must be careful about which pages they write in. Before they begin writing, they consult the manager of the book. The manager then decides where in the book they’re allowed to write.

Since this book is around for a long time, many of the stories in it are no longer relevant. When no one reads or references the stories, they are removed to make room for new stories.

Notificaciones de Grafana en Slack

Como ya tengo algo saturada la bandeja de correo, acabé decidiéndome por enviar las notificaciones de Grafana a Slack :D.

En primer lugar tendremos que crearnos una cuenta en Slack si no la tenemos todavía. Crearemos un espacio de trabajo (workspace) y un canal al que en mi caso he llamado #grafana. Todas las notificaciones de mi servidor Grafana irán a parar a ese canal.

System metrics with Docker, Telegraf, Influxdb and Grafana

We’ll use Docker, for the quick deployment our monitoring system, also it’ll gives us a freedom to use any software with dependency free and keep our system clean after. Topical, You can look at Docker from this side too, as a cross-platform package system.

Also, we’ll use part of Tick stack, namely an Influxdb base to store our metrics and Telegraf, like an agent on remote system, for nice and pretty graphs we’ll take Grafana.

Docker in Action

With Docker you can easily deploy a web application along with it’s dependencies, environment variables, and configuration settings - everything you need to recreate your environment quickly and efficiently.

We’ll start by creating a Docker container for running a Python Flask application. From there, we’ll look at a nice development workflow to manage the local development of an app as well as continuous integration and delivery, step by step …