At Inviso we have Alteryx workflows gathering data from all our internal systems to our datalake. The workflows are scheduled on our Alteryx server. But how do we ensure that these flows run smoothly? And that someone will react if that is not the case? To insure this we use our Alteryx server, the Alteryx CReW Macro Pack and Tableau of course!
The List Runner in Alteryx
The Alteryx tool “List Runner” is a CReW macro that can be used to run a series of Alteryx workflows. The CReW macro pack contains serveral usefull macros supporting Alteryx and is availabe for free download right here: https://community.alteryx.com/t5/Engine-Works-Blog/Crew-Macro-Pack-2016-Q2-Release/ba-p/26482.
The list Runner tool outputs a dataset containing the information of the executed workflows. This dataset is appended to a “flowlog” table in our database every time the workflow runs and is used to monitor the flow in Tableau.
The Log Parser macro from the CReW macro pack is used to parse the log. The message tools are used to look for any warnings or errors. The workflow below is an example of how the tools can be used.
These message tools in the workflow will tell you if there are any errors or warnings running the workflows. You can have Alteryx send an email if there are any errors, but if not, you will only see them if you open and run this flow. This is not needed when using the scheduler. Instead there is another way to monitor the flows. Furthermore, if the workflow with the list runner has not run at all, you won’t get alerted using this method as the flow has to run in order to send you the mail with the error.
When the List Runner is set up, you can schedule the List Runner workflow to run as frequently as required. This can be done either by a scheduler or on your alteryx server. The “flowlog”-data generated from the list runner workflow can then be used to monitor the alteryx workflow operations using Tableau.
Monitoring of operations in Tableau
The “flowlog”-data is loaded into Tableau. The output contains information on any errors and warnings from the list of workflows that were run in the list runner job. In the below example, one workflow failed due to one error.
This data can be used in Tableau to build dashboards that monitor and show the status of the Alteryx workflow.
The dashboard below shows the status of the different Alteryx workflow executions as well as the processing time. This dashboard is ideal to monitor execution of my workflow on a screen in the office. Below you see one red bar that indicate a failed workflow with an error. As soon as I see this happen, I can respond immediately!
But what if I am not watching the dashboard all time? I want to make sure that if anything unintentional happens when the workflow is run someone will react. The below dashboard shows the latest workflow start time compared to a chosen KPI for workflow latency. As my workflows run every hour I have set a KPI for 3 hours (180 minutes). The longer it has been since the workflow was executed, the smaller the green bar is.
When the dashboard is in place on the Tableau server you are able to define alerts based on these KPIs. You can choose any alerts that meet your needs. In this case I want an alert in an email if any of the workflows have not been run within the chosen latency KPI.
To do this, I choose the KPI axis and select the alert bottom, then I select when and how often I want to be alerted. In this case I want the server to email me when a workflow has not been run within the chosen KPI (here 3 hours), so it is above or equal to 0.0, and I choose to be emailed daily.
I now have a completely smooth and fully automated Alteryx workflow job execution and data output, updated regularly. The log-data is used to monitor the Alteryx workflows using Tableau and configured to alert me if there is any issues. If anything unintentional happens, I am reminded to react.
Try it out! It is easy, effective and timesaving! If you want to do the same with your workflow execution, we would love to help!
To get you started I’ve made the Alteryx workflow into a macro, use a Text Input tool containing paths for the workflows as datainput. The macro is uploaded and free to download from the public Alteryx Gallery here: https://gallery.alteryx.com/#!app/ListRunnerFlowlogMacro/5b98ee0d8a933708c4c9a3ae.
Furthermore, I’ve uploaded my Tableau workbook to Tabelau Public with a dataextract and a hardcoded date so you can see and try it for yourself. Go to https://public.tableau.com/profile/charlotte.ponikowski#!/vizhome/MonitorAlteryxworkflowsinTabelau/Overview to download the workbook.
Enjoy!