Tutorial | Getting started with datasets#

Create the dataset#

When you start with a project, you’ll want to upload a dataset.

Objectives#

In this section, you will:

  • Upload a local CSV file to Dataiku.

Prerequisites#

To complete this tutorial, you’ll need:

Create the project#

To create the project:

  1. From the Dataiku Design homepage, click + New project > DSS tutorials > Core Designer > Create Your First Project.

  2. From the project homepage, click Go to Flow (or g + f).

Note

You can also download the starter project from this website and import it as a zip file.

Use case summary#

Let’s say we’re a financial company that uses some credit card data to detect fraudulent transactions.

The project comes with two data sources (tx and merchants) and you’ll add a third one (cards).

The table below describes each of these three datasets.

Dataset

Description

tx

Each row is a unique credit card transaction with information such as the card that was used and the merchant where the transaction was made.

It also indicates whether the transaction has either been:

  • Authorized (a score of 1 in the authorized_flag column)

  • Flagged for potential fraud (a score of 0)

merchants

Each row is a unique merchant with information such as the merchant’s location and category.

cards

(to be uploaded)

Each row is a unique credit card ID with information such as the card’s activation month or the cardholder’s FICO score (a common measure of creditworthiness in the US).

Import data#

Dataiku lets you connect to a wide variety of data sources, but for this tutorial, let’s start by uploading a local file. Each row is the latitude and longitude coordinates of a unique credit card holder.

  1. From the Flow, click + Dataset > Upload your files.

    The New Uploaded Files Dataset page opens.

    Note

    If the project was empty, you would have seen a blue button + Import Your First Dataset on the project homepage.

  2. Click Select Files, and choose the cards.csv file.

    Dataiku displays a preview of the cards dataset. As you can see, the data is in a tabular format, with columns (features) and rows (records or observations). Dataiku has correctly set a default dataset name cards based on the file name.

  3. Leave the name cards and click the Configure Format button above the preview.

  4. In the Schema tab, click on Infer Types From Data, and then Confirm so that Dataiku tries to guess the correct storage types based on the current sample.

    A dataiku screenshot showing the Preview on the upload dataset page.

    Note

    Once the dataset is uploaded to your Flow, you can check its schema any time from the Settings > Schema tab, and then infer the types.

  5. Since the result is OK for us, finish importing the dataset by either hitting the Create button or using the shortcut Cmd/Ctrl+S.

This procedure creates the new dataset and lands you on the Explore tab of the cards dataset.

Dataiku screenshot of the Explore tab of a dataset.

Explore the data#

Once you’ve imported a dataset, you’ll want to start exploring it through the Explore and Charts tabs!

The Explore tab of a dataset provides a tabular view of your data where you can start to examine it, while the Charts tab has a drag-and-drop interface for data visualizations.

Objectives#

In this section, you will:

  • Learn about the sampling method for a dataset.

  • Compute the row count of a dataset.

  • Analyze a column.

  • Visualize your data using charts and a pivot table.

Dataset sampling#

As an environment capable of handling large datasets, Dataiku shows only a sample of a dataset when you are working interactively.

You can see the sampling method in the top left of the Explore tab. By default, the sample in this tab includes the first 10,000 records of the dataset.

  1. To view the total row count of your dataset, select Compute row count (the cyclic arrows icon).

  2. To change the sample settings of a dataset, select the Sample badge, which opens a panel on the left.

A Dataiku screenshot showing how to open the Sample settings panel of a dataset.

See also

For more information, see the Sampling article in the reference documentation.

Storage type and meaning of dataset columns#

In the dataset, beneath each column name, Dataiku indicates:

  • The storage type (in gray)

  • The meaning (in blue)

Here, Dataiku detects a meaning of text for the id column, based upon the fact that most values in the sample for customer_id are strings.

Dataiku screenshot of the data quality bar of a dataset column.

The data quality bar shows green for all columns, which means that all rows are valid.

Note

In this dataset, we do not have any NOK (Not OK) or missing values. But if we did:

  • NOK values would be represented in red in both the data quality bar and column cells.

  • Missing values would be in gray in both the data quality bar and column cells.

See also

For more information, see the Schemas, storage types and meanings article in the reference documentation.

Analyze window#

You can analyze the content of each column using the Analyze window. Let’s analyze the content of the cardholder_fico_range column. To do so:

  1. Click on the column name cardholder_fico_range, and select Analyze from the dropdown menu.

    The Analyze window opens, showing the analysis of the data sample.

  2. To extend the analysis to the whole data, select Whole data instead of sample in the dropdown next to the column name, then click Save and Compute.

    Dataiku screenshot of the data quality bar of a dataset column.

  3. Click the arrows next to the column name to switch from one column to another.

    Important

    As we are analyzing the whole dataset, we have to click Compute again at the top of the window to display the analysis for the whole data of the new column. It would be automatically computed if we were just looking at the sample.

    Dataiku screenshot of the data quality bar of a dataset column.

  4. Close the window when you’re done with the analysis.

Charts#

You can use charts to explore a dataset. For example, we might want to know which reward program is the most frequent for specific age ranges.

Here, let’s use two types of charts and a pivot table to show you different approaches.

Add a vertical bars chart#

  1. Click on the Charts tab (or use the keyboard shortcut g + v).

  2. From the Data panel, drag and drop:

    • Count of records as the Y variable.

    • reward_program as the X variable.

    • cardholder_age to the color droplet field.

    The chart reveals that the cash_back reward program is the most popular, whatever the age of card holders.

    Important

    The Sample badge at the top of the chart reminds you that the chart is sampled like the dataset. Clicking on the badge allows you to change the sampling method if needed.

  3. Open the dropdown menu of the cardholder_age variable and check the Adjust bin size for nicer bounds option to have five nicer bins for the age of card holders.

    Dataiku screenshot of some options used to customize a chart.

  4. On the chart, hover over any of the orange bars to filter on card holders aged between 40 and 60. Once the menu is displayed, click on the bar to keep the menu on and select the drill down icon (the down arrow) at the right of the cardholder_age variable.

    Dataiku screenshot of the contextual menu on vertical bars in a chart.

    This action adds a filter to the Filters section of the Setup tab of the left panel.

    Dataiku screenshot of filters added to a chart.

  5. To go back to the default display, extend the age range from 18 to 100 in the Filters section of the Setup tab.

  6. In the left panel, go to the Format tab to customize the chart:

    • Under X Axis, open the Title dropdown, and enter Reward program in the Axis title field.

    • Under Color, select the built-in Pastel 2 palette.

    Dataiku screenshot of some options used to customize a chart.

Add a pie chart#

Let’s add a second chart.

  1. At the bottom of the screen, click + Chart.

  2. In the chart type dropdown, select Pie.

  3. From the panel on the left, drag and drop:

    • Count of records in the Show field.

    • reward_program in the By field.

Dataiku screenshot of a pie chart.

The chart confirms that the cash_back reward program is the most popular.

Add a pivot chart#

Lastly, let’s add a pivot table to represent in a different way the distribution of card holders per age across the different reward programs.

  1. At the bottom of the screen, click + Chart.

  2. In the chart type dropdown, select Pivot table.

  3. From the panel on the left, drag and drop:

    • cardholder_age in the Rows field.

    • reward_program in the Columns field.

    • Count of records in the Value field.

  4. In the dropdown menu of the cardholder_age variable:

    • Enter 5 as the Number of bins.

    • Check the Adjust bin size for nicer bounds option to have five nicer bins for the age of card holders.

Dataiku screenshot of a pivot table.

See also

For more information, see the Tutorial | Charts and pivot tables article.

What’s next?#

Congratulations! You’ve created your first project, imported your first dataset, and built your first charts.

We recommend you to check the Tutorial | Join recipe, where we join the three datasets from this project to enrich each unique transaction with data about the information on credit card holders and merchants for that transaction.