Concept | Architecture model for databases#

Database integration#

By integrating with data sources, such as the SQL databases in the previous figure, Dataiku is able to directly read from a database, write to a database, and process data using a dedicated database engine.

Importing a dataset#

Dataiku allows you to import a dataset through an existing SQL connection by selecting the table that you want to import from a list of the tables in the database, and then creating the dataset.

Writing to a database#

You can directly write data from Dataiku to an SQL database by using any visual recipe, for example, a Sync recipe or a Prepare recipe. You can also write to an SQL database by using certain code recipes, such as a Python recipe or an R recipe.

When creating the recipes, specify the database as the storage location of the output tables, by using a connection, such as a PostgreSQL connection.

Note

SQL datasets in Dataiku are pointers to database tables. Therefore, the data is only written in the SQL database. Database concepts of rows, columns, and column storage types also apply to the SQL datasets.

Database connection settings#

The settings of the SQL connection determine the database and schema where the tables are created. Dataiku also sets the names of the database tables by adding a prefix to the names of the datasets in Dataiku.

Finally, Dataiku sets the column types, so that you don’t have to write CREATE TABLE statements.