Concept: Developing Plugins¶
This content is also included in a free Dataiku Academy course on Plugin Development, which is part of the Developer learning path. Register for the course there if you’d like to track and validate your progress alongside concept videos, summaries, hands-on tutorials, and quizzes.
In this lesson, we’ll explore the structure of a plugin and the steps involved in creating a development plugin.
There are three ways to create a development plugin:
by writing code to define a new one;
by converting certain Dataiku DSS elements, such as a code recipe or a webapp, into a development plugin; or
by cloning and modifying the remote git repository that stores the source code for an existing plugin.
Write Code To Define a New Development Plugin¶
To create a new, empty development plugin:
Go to the plugin store.
Select Add Plugin > Write your own.
Name your plugin and begin adding components to it.
Add a Preparation Processor Component¶
An example use case for developing a plugin is to extend the capabilities of the Prepare recipe. You can do this by writing a custom processor that will be available as a step in the Prepare recipe. To create the custom processor:
Go to your Plugin’s Summary page.
Click + New Component.
Select the Preparation Processor.
By adding the processor component, Dataiku DSS updates the plugin’s directory structure with two main elements that define the processor component:
a JSON file that defines allowable parameters such as the input and output for the processor; and
a Python file that defines the plugin’s functionality.
You can then modify the contents of these files to customize the processor’s user interface and functionality.
Convert a Dataiku DSS Element to a Development Plugin¶
You can also create a development plugin by converting a Dataiku code-based element, such as a code recipe, to a plugin.
Add a Recipe Component¶
To add a recipe component to the development plugin:
Open a code recipe and click the Convert to plugin action from the sidebar.
This action allows us to either create a new plugin that consists of the plugin recipe or to add the recipe component to an existing plugin.
Add the recipe component to the existing development plugin.
By adding the recipe component to the existing development plugin, Dataiku DSS makes the appropriate additions to the plugin’s directory structure, along with some starter code that you can modify as needed.
Modify the Plugin Component’s Starter Code¶
To create a good quality plugin, you should follow good coding practices when modifying the starter code. Some practices to consider include:
using good naming conventions;
writing clear descriptions;
checking for errors in user inputs, etc.
Clone a Remote Git Repository That Contains a Plugin’s Code¶
The third option for creating plugins is by cloning and modifying the remote git repository that stores the source code for an existing plugin.
Click Add Plugin from the plugin store’s home page.
Select Fetch From Git Repository.
When cloning the repository, you can clone in “development mode” which allows you to modify the code and later push back changes to the repository, provided that you have “write” privileges on the remote repository. You can also clone the entire repository or clone a subfolder (if the repository contains multiple plugins).
Create a Dedicated Code Environment for a Plugin¶
Every plugin needs a code environment. By default, the plugin will use the DSS builtin code environment. However, it is good practice to create a dedicated code environment that includes dependencies, such as all the required code libraries (or packages) for the plugin to work.
To create the code environment:
Click +Create a Code Environment from the plugin’s Summary page.
You can define the dedicated code environment within the plugin’s directory structure with files that contain the specifications for the desired packages and the code environment’s characteristics.
The code-based components in a plugin will use its associated code environment. However, there are some exceptions, such as in the case of custom algorithm components. Custom algorithms use the code environment that you specify in the visual ML tool.