Reference | Managing containerized execution configurations for standard compute or (GPU) accelerated compute#

The Usage & Monitoring panel of the Launchpad lists the containerized execution configurations.

Containerized execution configuration for standard compute#

Dataiku provides certain default configurations according to your elastic AI quota. However, the space admin can create specific configurations by clicking + Add Configuration in the following situations:

  • Create a configuration larger than the default options. This can be the case if you have purchased Multi-Purpose Credits that enables to access compute above your elastic AI quota.

  • Restrict access to certain configurations to specific user groups. To do that, set Usable by to Allowed, and then select the allowed groups. Note that the default configurations have no restrictions.

  • Restrict access to certain workload types (user code or visual recipe or both).

Dataiku screenshot of how to create a new containerized execution configuration.

Note

The following rules determine the maximum CPU / RAM number:

  • If you don’t have credits available at the time of the configuration, then the elastic AI quota defines the maximum.

  • If you do have credits available at the time of the configuration, then the maximum is the maximum that Dataiku Cloud can offer: 46 vCPUS / 368 GiB RAM. Please open a support ticket if you need more.

Tip

Users will choose a configuration based on its name. Hence, a good practice is to standardize the configuration name in a self-describing manner, such as CPU-$NUMBER-RAM-$NUMBER with RAM in GiB.

Containerized execution configuration for (GPU) accelerated compute#

Note

You need to have purchased Multi-Purpose Credits to be able to create this kind of containerized execution configuration. To learn more, contact your Dataiku representative.

In Usage & Monitoring > Containerized execution configurations, click on +Add a GPU Configuration.

  • Choose one GPU instance type from the list. Note that the GPU instance type list is different according to the Cloud Service Provider region hosting your space.

  • Choose a name and restrict access to this configuration with Usable by to Allowed as described in the standard compute chapter.

  • Review the content and confirm.

The allowed users will now be able to use this configuration for their workloads. For notebooks, the time to initialize the kernel will be slightly longer with GPU configuration (up to 5 minutes) than with standard compute configuration.

Note

If, among the listed GPU instance types, you can’t find a type that’s matching your workload need, open a support ticket.