Reference | LLM connections

Dataiku provides connections to a variety of Generative AI services, both hosted and self-hosted, that are required to build GenAI applications.

The connection can then be leveraged with Dataiku’s visual recipes, such as Prompt Studios, Prompt recipe, Natural Language Processing (NLP) recipes, and the Embed recipe, or via the Dataiku Python API for usage in code recipes, code studios, notebooks, and web applications.

Instance administrators can configure connections to LLMs in:

• The Connections menu > Add a connection > Read Only Data Sources from the Launchpad of Dataiku Cloud.

• The Administration menu > Connections > New Connection > LLM Mesh in an on-premise installation.

Hosted LLM connections

Supported connections to Generative AI services include models such as OpenAI, Hugging Face, Cohere, etc.

Connection settings vary depending on the requirements of the providers.

Hugging Face connections

Users can host, manage, and use open source Hugging Face models locally with Dataiku’s Hugging Face connection. This connection includes containerized execution and integration with NVIDIA GPUs to make the model execution simpler. Users can start experimenting with Hugging Face models without downloading the model into the code environment’s resources.

Administrators can configure which Hugging Face models are available for different tasks, and other settings such as containerized execution, in the connection settings.

Custom LLM connections

Users who have developed their own LLMs with API endpoints can integrate these custom services into Dataiku via the Custom LLM connection. Setting up a Custom LLM connection requires developing a specific plugin using the Java LLM component to map the endpoint’s behavior.