Concept: Sampling

Sampling allows for immediate visual feedback while exploring data no matter how large the dataset. There are a number of different sampling methods available, aside from the default first 10,000 rows. The same sampling principle applies to visualization (Charts) and data prep (Prepare recipe).

../../../_images/sampling.png

Exploring very large datasets can be unwieldy, as even simple operations can be expensive, both in terms of computational resources and time. The approach DSS takes to solving this problem is to display only a sample when exploring and preparing data.

The default sample for any dataset is the first 10,000 rows. Although it is the fastest method, the sample may be biased depending on the composition of the dataset. Depending on your needs, many other sampling strategies, such as random, stratified, or class rebalancing, are available. The tradeoff for a potentially more representative sample is the time needed for DSS to make a full pass or sometimes two full passes of the data.

The main purpose of sampling is to provide immediate visual feedback while exploring and preparing the dataset, no matter how large it may be. This means that because DSS is only viewing a relatively small sample of the data, you can very quickly sort the sample by a column, apply a filter, display column distributions, color columns by values, and view summary statistics.