Concept | Architecture model for databases#

Watch the video

Database integration#

../../_images/connections-support-levels.png

By integrating with data sources, such as the SQL databases in the previous figure, Dataiku is able to directly read from a database, write to a database, and process data using a dedicated database engine.

Importing a dataset#

Dataiku allows you to import a dataset through an existing SQL connection by selecting the table that you want to import from a list of the tables in the database, and then creating the dataset.

Writing to a database#

You can directly write data from Dataiku to an SQL database by using any visual recipe, for example, a Sync recipe or a Prepare recipe. You can also write to an SQL database by using certain code recipes, such as a Python recipe or an R recipe.

../../_images/recipes-move-data.png

When creating the recipes, specify the database as the storage location of the output tables, by using a connection, such as a PostgreSQL connection.

../../_images/store-output-data.png

Note

SQL datasets in Dataiku are pointers to database tables. Therefore, the data is only written in the SQL database. Database concepts of rows, columns, and column storage types also apply to the SQL datasets.

Database connection settings#

The settings of the SQL connection determine the database and schema where the tables are created. Dataiku also sets the names of the database tables by adding a prefix to the names of the datasets in Dataiku.

../../_images/connection-settings.png

Finally, Dataiku sets the column types, so that you don’t have to write CREATE TABLE statements.

For more information, see SQL databases in the reference documentation.