In these topics, you will learn how to write your own plugins with programming languages, such as Python and R.
Naming is an important area of plugin development that is often overlooked. Dataiku requires that plugin names and identifiers meet certain policies.
Benefits of a naming policy¶
It can be difficult to tell which of your installed plugins are developed internally, externally, or from Dataiku. One way to keep all your ducks in a row is to apply a plugin naming convention. In addition, when you share your plugins, your colleagues will be able to understand the purpose of the plugin.
Plugin display name and identifier¶
The plugin summary displays information about the plugin and its components. You can edit the display name and identifier for the plugin in the Edit tab.
Component display name and identifier¶
You can also edit the component display name and component identifier in the Edit tab. Adding a description is a good idea because it documents the reason why you created the component and lets others understand its usefulness and purpose.
Policies and guidelines¶
A naming error happens when the name of the plugin, or any of its components, labels, or tags, does not meet plugin naming policies. Luckily, you can avoid these errors by following the plugin naming policies below. Most of the policies are required while a few are recommended.
Though the Dataiku interface has a built-in code editor, you may have a preferred IDE. You can set up your IDE for interactive editing of your plugin code.
This enables you to develop plugins in your favorite coding environment. You will also be able to test that your plugin is working as expected by interacting with Dataiku through its APIs.
This guide covers setup with Atom and Sublime Text 3.
This guide covers only interactive
All steps in this tutorial happen outside of Dataiku.
You must start by setting up a virtual environment in your workspace. In Python 3 you can use
python3 -m venv dataiku_dev_env.
Activate your environment by running
To install the
dataikuapi package you can run
pip install dataiku-api-client.
You must now install the
dataiku package, please follow the instructions on our page dedicated to the installation of this package.
Set up the kernels¶
We will now take care of the interactive environment. Run
pip install ipykernel to install Jupyter.
You need to create a kernel specification in your user space so that your IDE can find it later.
python -m ipykernel install --user --name dataiku --display-name "Python (Dataiku venv)"
Set up your code editor¶
Install the package Hydrogen by going to Settings (ctrl+,) -> + Install.
You can now open any Python source file and use shift+enter. This will start a kernel in the current file.
In Sublime Text 3¶
Install the package Hermes (use ctrl+shift+p to open Package Control).
When you start a Hermes kernel, an output tab will appear. It is recommended to split your view in two panes and have that tab in the right pane.
Open your Python source file in the left pane and add
# %% to define your cells. A “Run cell” phantom button will appear next to them!
Develop a plugin¶
Within plugins, it is recommended to develop libraries that will provide the core functionality you are looking to add. Additionally, you should provide some recipes that use your libraries.
When developing locally you can use a test file to check that your library is working as expected. Here is a typical file structure:
custom-recipes/my-recipe Recipe folder recipe.json Recipe definition recipe.py Recipe contents, runs within DSS python-lib Library folder mylib.py Library contents README.md plugin.json test.py A local file to test your library
Installing an existing plugin is useful if you simply want to use the plugin. If you want to contribute to the development of an existing plugin, you’ll need to clone the remote Git repository (repo) where the plugin source code is stored.
Access to a Dataiku instance that has been set up to work with remote Git repositories.
Connect to a remote Git repository¶
Navigate to the Plugins Development area and click +New Dev Plugin.
Select Retrieve plugin from Git as the source.
https://github.com/dataiku/dss-plugin-sample-correlationsas the URL of the remote.
Click on the change tracking indicator and select Add remote.
Pull the changes from the remote
In the plugin editor, you should now see the contents of the remote repo in your local repo. If you have write privileges on the remote repository, you can push changes as described in Tutorial | Plugin version management with Git.
You can convert a plugin from the Plugin Store to a Dev Plugin. Use caution. You will not be able to get updates from the store or from your own repositories afterward.
One of the golden rules of plugins is to never use “import dataiku” in a library. With few exceptions, plugins do not depend on Dataiku unless it is an administrative plugin.
Plugins turn custom logic into components that can be used by anyone, giving less technical users the ability to collaborate and technical users the ability to standardize the way data is processed. Plugins can easily be shared by the community of users in an organization and even shared across organizations. Since most plugins are open source, anyone can view the underlying code and make contributions to it or adapt it.
Plugins range from the very simple to the very complex. For example, plugins can be used to create a connector to read or write data to databases or connect to external APIs. Some examples are connecting with Salesforce data, or get weather forecast by location. You can find more plugins in the plugin store. The possibilities are limitless!