Tip | Choosing container sizes

The quota included in your subscription determined the largest container available. That is, if your quota is 10 CPUs and 80 GB of RAM, the largest container available in the drop-down menu will be CPU-10-RAM80Gb.

This is true up to CPU-14-RAM112Gb, the largest container available in Dataiku Cloud.

Using a large container by default isn’t recommended as it can exhaust available resources and prevent others from executing their jobs. Try starting with the smallest container available and increasing its size if need be.

There are generally two cases when to increase the size of the container:

• The execution failed with an out of memory error because the container is too small. In that case, it’s recommended to increase the container size so as to allow more memory.

• The execution is too long and the execution can be parallelized, such as with hyperparameter search in visual ML.

Tip

When working with a large dataset, start by executing the job on a sample of the data and the smallest container as a way to test it.