Concept | Plugins in Dataiku#

Dataiku natively lets you connect to data, process data, train models, and more. To expand the power of Dataiku, you have the ability to add additional packaged functionality through plugins.

In other words, you can tailor your Dataiku instance to include the specific features you need to enhance your data pipelines and workflows.

Use cases#

Plugins vary from the very simple to the very complex. For example, you can retrieve Wikipedia page views, connect with Salesforce data, or get weather forecasts by location.

Let’s take a look at the US Census plugin, which consists of six components: three visual recipes and three dataset connectors.

A Dataiku screenshot of the plugin store page for the Census USA plugin, highlighting its six components.

We can use the visual recipes from this plugin to enrich a dataset with one of the hundreds of socio-demographic variables from the US Census Bureau. We can also use the dataset connectors to build and use the US Census data directly within Dataiku.

Note

Plugin documentation is available on the plugins page of the Dataiku website. The plugin’s page also includes a link to its source code on GitHub. Note that many plugins from the Dataiku Plugin store are open source.

Next, we’ll learn more about what plugin components are and how they work.

Plugin components#

A plugin in Dataiku can contain one or more related components. Each plugin component consists of a graphical user interface (GUI) wrapper around code, and it exposes a single type of Dataiku element, such as a dataset, recipe, webapp, processor, and more.

A slide with Dataiku screenshots showing different plugin components such as a dataset, recipe, webapp, and Java preparation processor.

Note

Although the process of plugin installation and management differs for users on Dataiku Cloud and self-managed instances, usage of plugin components is the same.

Using a visual recipe component#

The plugin’s visual recipe components can be accessed by clicking the +Recipe button in the Flow, or by accessing them in the right panel.

A Dataiku Flow highlighting how plugin recipes can be found either in the right panel or from the + Recipe dropdown.

The plugin’s visual recipe works like any other visual recipe. That is, we can select input parameters in the recipe settings and then run the recipe.

Using a dataset connector component#

To use one of the dataset connector components, Census USA, we can use the + Dataset button to access new dataset connectors for connecting to US census data.

A Dataiku screenshot of creating a new dataset through a plugin connector by clicking the + New Dataset button from the Flow.

This plugin will let you input values for parameters, such as:

  • States for which we want to get data

  • Geography granularity

  • Fields to extract

In the backend, Dataiku will fetch the data from the US Census website to create this dataset.

Using a processor component#

The Zipcode geocoding plugin includes a processor component called zipcode geocoding that is accessible from the Prepare recipe.

A Dataiku screenshot of a Prepare recipe with the processors library open showing the Zipcode geocoding step.

Using this processor, we can extract geographic coordinates from location data such as country and zip code. This processor step works just like any other in the Prepare recipe!

Note

You’ve just learned about a few different plugin components that are available in Dataiku. To see the full list of plugin components, visit the page on Plugin Components in the reference documentation.

What’s next?#

As you can see, plugins are integrated seamlessly in Dataiku to help you accomplish a wide variety of tasks. If you want to extend the functionality of Dataiku even more, try developing your our plugins!