Part 67: Track airport departures with Grafana's Infinity data source

Here's something different to leave my Generative AI thing and overall Zabbix theme for a moment. 

Recently I've been playing with Grafana's Infinity data source. It's an extremely flexible tool for fetching and parsing data, let that be JSON, XML, GraphQL, HTML, or many other supported formats. As in addition to Zabbix I have Grafana running on my Raspberry Pi 4, I did let my imagination to fly and tried out a simple task with Infinity: show Helsinki airport departing flights. That's easy, there must be an API, right? No, in this case the content is in HTML table, see the page here.

Let's do this together -- in the end you'll have a dashboard like this, visualizing the departures on text table, showing the top destinations, and a Sankey diagram of flights, connecting their details in a metro map like thing.

Install the plugin

For us self-hosted Grafana users, just install the plugins with 

grafana-cli plugins install yesoreyeram-infinity-datasource
grafana-cli plugins install netsage-sankey-panel

and restart your Grafana. 

Grafana Cloud users, just find the plugins Infinity and Sankey from Grafana plugins and install it by clicking on Install.

Done!

Setup the Infinity plugin as new data source

In Grafana, go to Connections --> Add new connection, find Infinity, click it, give it a name, done.

Start using it

Next, create a new dashboard and add a new visualization. Choose Infinity as your data source, and just fill in new values.

In case the picture is hard to read for you: 

• Type HTML
• Parser Default
• Source URL
• Format Table
• Method GET
• URL https://airporthelsinkivantaa.com/en/departures
• Rows/root tbody tr
• Selector td:nth-child(1).as Departure format as String
• Selector td:nth-child(2).as Boarding format as String
• Selector td:nth-child(3).as Flight format as String
• Selector td:nth-child(4).as To format as String
• Selector td:nth-child(5).as Check-in format as String
• Selector td:nth-child(6).as Terminal / Gate format as String
• Selector td:nth-child(7).as Status format as String

Refresh the view and you should see something like this, though at this point without any additional colors.

What does that do?

Now that you filled in the details, you might wonder how that wizardry is working. 

When Infinity is in its HTML mode, the Rows/root selects the element from your page it should concentrate on. As this page has only one table I could just put tbody tr as my values . If it would some nth table on the page, I would have put tbody:nth-child(2) tr or so.

In HTML, tr stands for table row, and now Infinity goes through all the rows in the table.

Next, you need to parse the table columns, which is td. For that, we can then pick all the columns we need with the td:nth-child(x) and map them with a name.

And, you are done!

Add some colors

For adding some visual cues about flight status, in Grafana's panel editor where you already are, go to Value mapping and add the details like this with the color scheme you fancy.

Save your new panel. There you have it, a nice text table. But we can do more!

Show top destinations

Duplicate your panel, and switch the panel type to Piechart. Edit the panel, and click on Transformations tab. Add the following transformations.

Filter data by values

Filter type: Include

Conditions: Match any

Field: Status Match is equal Value Departed

Group by

To Group by

Status Calculate Count

Sort by

Field Status Reverse [x]

On Value options, choose All values and limit the number of results if you want

And here you have it, your top destinations in a nice pie chart.

But there's still more -- the Sankey diagram. How to use that? 

Go with the diagram flow

Duplicate your panel again, edit it and change the type to Sankey. Next, go to Transformations tab and add the following transformations.

Add field from calculation

Mode Reduce row

Operation Status

Calculation Count

Alias Flight status

Group by

Flight Group by

To Group by

Terminal / Gate Group by

Status Group by

Flight status Calculate Count

Organize fields by name (and order them to this order)

Flight

Terminal / Gate

To

Status

Flight status (count)

Limit

Limit 15

Refresh the view, and you should have a nice Sankey panel in front of you, illustrating you which flight number went to which destination, what was its departure gate, and ultimately, also the flight status (departed, cancelled...). 

If you want to see every flight that does NOT have its status as Departed like this ...

... then go and add a Filter by values transformation, too, and exclude the values where Status is Departed.

Feel free to try out tweaking the Sankey visuals via its settings.

Of course, now that you have the data, you can show it in any other Grafana panel and merge the data with for example weather conditions. But, that's a story for another time.