What is the Pipeline Directory?
The Pipeline Directory is a quick and easy way to create outputs for your data. Whether to produce widgets, triggers, to kick off a process or all of these, Pipelines provides versatile functionality for you to accomplish this.
With the ability to import the data you need and the environment to manipulate it to your requirements, the pipeline can be very useful in allowing you to present information in a way that would benefit you and your communities.
How do you create a Pipeline?
Firstly, click on the icon in the top left corner of the screen and under Tools, click ‘Pipeline Directory’. The first page you will see will be the main directory where you will see a list of all of the Pipelines you have created and/or have access to. On the left bar, is a tree-structure view where the Pipelines are divided by location. You can easily add a new location or edit a location by clicking on the ‘+’ icon to add, or the pencil icon to edit.
When adding a Location, you can add a name and description, and also add the communities that this location can be visible to and edited by. If you add a prefix, it’ll be added to each Pipeline you create within the location. Editing a location will allow you to edit each of these.
Once you have set up the location of where you want to store your Pipeline, clicking the create Pipeline button at the top of the left bar will prompt you to add in the necessary information. The communities you want this Pipeline to be visible to adds a layer of security for which communities have access and which can modify it.
So now you are all set up to start configuring your Pipeline. The Pipeline is divided into four separate lanes, and the most important lanes are the Data and the Do, which are required for your Pipeline to work.
A quick overview of what each lane is used for:
Data:- This lane is where you import your data, this can be from a Sabisu Data Object (SDO).
Filters & Calculations:- You can add any filters or calculations to manipulate your data.
Analytics:- This section is where you can set up some more complex analytics to run.
Do:- What you want to do with your data or how you want to visualise it.
A few handy tips:
The Test button – This can be used at any stage of creating the Pipeline , which shows the output of the data.
Versioning – After saving the Pipeline , a new version is stored and the option to backtrack or revert back to previous versions is available.
There is no limit on the number of data sources you can add.
Opening the Data options by clicking the ‘+’ icon will show a list of SDO’s that available to you. Selecting an SDO will open a panel on the right for you to configure exactly what you need to extract.
Within this panel, you will see a list of the datasets and views that are available for use. If there are any pre-defined queries set up on the SDO, these will also be displayed here. Or, if you need something more specific, there is also the option to write your own SQL query, which supports Sabisu $ commands.
The pre-defined queries that have been set up in Premise Admin should allow you to easily get the data you want from the SDO you have selected. Be sure to give your pre-defined query a name, and add any tags you may need.
Datasets & Views:
You can directly pull out data from the database by selecting a dataset or view. Each will then display all of the columns along with some example data from the table or view you have selected. Once you have the columns you want to use in your Pipeline , you can then add any filters to the data you may need. You can imagine this to be like adding the WHERE clause to a SQL query.
When you have filtered your data to your needs, the next step will show you an overview of the filters you added in the previous step, the columns you have selected and the opportunity to change anything before you add it to the Pipeline . You are also able to view and check the results of the query the wizard has constructed for you. Take note that you should always check that you are receiving data from your data source before you add it to the pipeline.
Once you are happy with the data, you can then add it to your Pipeline. Each data source added will be outlined in its own unique colour as a reference.
Make use of the ‘Test’ button to check your output at any time, this will be especially useful in the subsequent lanes, where the data manipulation will take place.
2. Filters & Calculations
With your data source set up, you can then move on to start manipulating your data.
There are a number of filters and calculations you can choose from, and each has its own unique configuration. Each manipulation will be displayed as an extra column in the output, which will be populated depending on the calculation. You can see this by clicking the ‘Test’ button.
An example of a filter or calculation is the ‘Basic Math’ option, here you can select the available inputs from your data sources and insert your calculation. for example, you could insert a tag and multiply it by 0.6. The test button will give you an example of the data it may output. Once the calculation has been added to your Pipeline, you can hit test to see the results.
Each manipulation added to the Pipeline will show a coloured dot on the upper right corner, which is a reference to the data source it is reading from.
The analytics lane is for more complex manipulations, such as anomaly detection or flatlining. This allows you to run more complex calculations without having to do a lot of the work.
If you add anomaly detection to your Pipeline, it will also add another column to your data, this will be shown in 0’s and 1’s to show where an anomaly has been detected.
The Do lane is the final step of the Pipeline and is where you can use your data to create the output(s) you require. Options include creating a chart or widget or setting up a trigger for a note or a process.
To create a widget, you can either create a chart or use a Sabisu template. Charts are very straightforward to create, the config allows you to select the data to include and what format to display it in, for example, a line graph.
Sabisu templates are a selection of pre-made widgets with a professional user interface, all you need to do is just add data.
When you have selected a template to use, you are then able to edit the template to display your data. Within edit mode, the following tools may come in useful:
Show Editable :- When checked, this will show boxes around each field you can change.
Render Data:- When checked, this option shows the data from the Pipeline. Unchecked will show the names of each field.
Refresh Data:- Will refresh to show any changes made.
Each input can be configured to show either data from the Pipeline or free text, such as headers and titles. Be sure to refresh the data after each change and once you are happy with the template, you can then save it to your Pipeline.
Triggers are useful when you need to know if or when an input meets a specific criteria, these can be based on when an input is less than, more than or equal to a specific value.
You can use triggers to set off a process, or to post a note to a community.
To post a note to a community, simply write up the note and add in the requirements for when the note should be posted, ie, when a value reaches below 100. It is also recommended to add a cooling off period to your trigger so it isn’t fired too often. This can be set to 0 if you don’t require any cooling off period.
When setting up a trigger to kick off a Process, the process name is required along with the criteria of when the process should start. If you need to create a process, these can be set up in Actions.
Deploy/Add to Page
When you are happy with your output, you can then deploy your Pipeline and add the widget to your page, Be sure to open the page you require your widget to be displayed as the default action is to add it to your current page.