How to send a message to slack channel on Pentaho Data Integration (PDI) in 5 minutes
Today I’m going to write about how to send a message to slack channel in Pentaho Data Integration. I write this topic because it is a problem that I face during the executing of scheduled data integration jobs on Pentaho (Getting the notification immediately) if there is a problem. And besides that, my company are using slack as a day to day communication channel.
Okay, lets go to topic. The first part that we need to setup is the slack part. We will going to use the “incoming-webhook” feature in slack. In order to set it up, please follow the steps below.
Prerequisite
if you haven’t install the Pentaho Data Integration, Please see the step to install it on my previous article https://medium.com/@gembit.soultan/how-i-setup-my-pentaho-data-integration-pdi-and-solving-the-problem-on-running-it-on-mac-osx-6f0cc7f3b97c.
Slack Incoming-Webhook Setup
- Go to the settings & administration — Manage apps
2. Click on the button “Browse the app directory” and type “Incoming webhooks”
3. The next steps is click the button add to slack app.
4. Choose the channel or you can create a new channel for the incoming webhooks and dont forget to click the green button “Add Incoming Webhooks Integration”
5. As you can see, the slack will generate you the webhooks url. and some instructions like the json body parameter to post the message to slack.
6. Okay, before we go to the Pentaho lets try to test it with Postman API to test the webhook url.
if you post the url and see the response status ok, then your incoming webhook slack setup is successful.
Slack Integration Jobs
The next steps is creating the pentaho jobs to test the incoming webhook features. In this case we want to read the text from the file and send it to slack.
- Create a folder in your home directory or whatever you want with the name of “Slack-Job”. Well the folder name is up to you. but for this example I want to name it like this.
- Create a text file and named it “hello-world.txt” (file name is up to you). and content of the file is “Hello world” like the image below.
3. Create a new Job, named it “Send Text To Slack”. type on the search box at the tab design “start”. A step start is used as an entry point of job.
if you're not familiar using the Pentaho, “Job” is used to execute several transformation, retrieve files from the web server, place files in a target directory and job it self can be run based on the schedule. You can read it on the pentaho documentation below. https://help.pentaho.com/Documentation/5.2/0J0/0C0/030
4. Create a new Transformation to read the file from text and to send the message to slack channel.
5. Add a step “Text File Input”, you can search it on tab design by type “Text File”.
5. Edit the Text File Input step by click right and choose edit and then Browse the file you want to read the content and click “Add” button.
Since the content of the file is just only simple “Hello World” text. so we need to configure the tab content and fields. In the content section, Change the separator from (;) to (|) and uncheck the header fields.
In the Fields section, add name of the field you can type it “label” and data type as string.
6. Add the “Modified Javascript value” step, we need this step because the data format to post message to slack is Json value. We are going to convert the hello world to json value like below :
{ “text” : “Hello World” }
7. Edit the “Modified Javascript value” step, and on the script editor, we want to create payload object, with the “text” properties and convert it to JSON with JSON.stringify method. Please see the “Incoming-webhook” setup insturction below on sending Message part.
after you create payload object don’t forget to click the “Get Variables” button to convert it to the payload fields.
6. Add the “Rest client” step to transformation. Add connecting the “Modified Javascript value” to Rest Client.
7. Configure the “Rest Client”. Add the Generated URL from the “Incoming-webhook” that we already created on the beginning, change the HTTP method to “POST”, fill the Body field with “payload” and change the Application type to “JSON”.
Okay, you can run it to test your Transformation. if it is successful, you should see the “Hello World” message in your slack channel.
To link it to the Job, Go to the Job tab that we already create it on the beginning. And add the Transformation step.
Edit the transformation and link it to your Transformation file that we already created previously. I name it my transformation file with “Slack”.
Save the Job with the name “Slack”. and this is my structure file after doing all the steps. There will be 3 files, the “hello-world.txt”, Slack transformation “Slack.ktr”, and “Slack.kjb” file for the jobs file. and try to running it once again from the Job tab.
if it successful then it should be a new message on your slack channel.
Hopefully, this article is really useful for you guys and easy to understand. if you need the file, you can download it from my github repository https://github.com/gsoultan/Slack-Pentaho.
Thanks and Cheers, Keep the spirit to share the knowledge to the others.
Keep productive during the WFH (Work From Home).
