Concept | Visual agents#

Important

Building agents requires the Advanced LLM Mesh.

An agent is a system powered by Generative AI models that can use tools to interpret and process data, make decisions, and perform actions.

These systems go beyond simple question-and-answer use cases such as Retrieval Augmented Generation (RAG) and chatbots and can work independently using multiple input sources and tools.

For example, you might use RAG to build a chatbot that can answer questions about your company’s expense policy using the employee handbook and other documents. This is useful, but an agent can do so much more!

You can add tools to allow your agent to query data or perform actions, even in external systems. For example, this agent could also review receipts from employees, check expenses against company policies, flag problems and send alerts to supervisors, then follow up with employees about the status of their expense request.

In Dataiku, agents can be built through a fully visual interface or with code. Agents are objects in the Flow that you can use, modify, and share with others.

Screenshot showing visual and code agents in the Flow.

Building agents#

To build agents with the visual interface, you need three elements:

  • A connection to the Large Language Model (LLM) of your choice to power the agent.

  • A prompt that gives context and instruction to guide the agent in its tasks.

  • One or more agent tools, which are components that can be used by the agent to perform actions or make decisions. The tools are powered by your datasets, APIs, machine learning models, and knowledge banks.

Dataiku screenshot showing an agent Config page.

Agent tools#

Dataiku provides a number of built-in agent tools that you can configure and combine to build custom agents.

To list a few examples, Dataiku’s managed agent tools can:

  • Look up records from a dataset or documents from a knowledge bank.

  • Make predictions based on a machine learning model.

  • Send email alerts.

  • Search the web.

Dataiku screenshot showing options when building agent tools.

If the managed tools don’t provide what you need, you can also build custom agent tools by:

  • Using inline Python tools to write custom Python code directly in the interface.

  • Developing plugin tools for easy templated reuse across your organization.

Visual agent tools apply the ReAct — or Reason and Act — approach. With ReAct, the agent first “reasons”, using its LLM to analyze the input and determine the most appropriate “action” to take (i.e. which tool to call). The agent then observes the result and decides if further actions are needed, and continues this loop until the task is completed.

Using agents#

Once you create an agent, it becomes available as a “virtual LLM” in the LLM Mesh. This means that everywhere you can connect to an LLM in Dataiku, you can connect to the agent in the same way. This includes:

Connecting to an agent in the Prompt recipe via the LLM Mesh.

You can also turn any agent built in Dataiku (visual or code) into a tool to compose multi-agent architectures.

Guardrails and governance#

Because agents are created through Dataiku’s LLM Mesh, you can test the performance of an agent with multiple LLMs and adapt as new models become available.

Agents are also governed objects in Dataiku Govern, which allows for review and pre-deployment sign-off.

You can apply Dataiku’s guardrails to an agent, to ensure that queries and responses are safe and correct.

Screenshot showing the guardrails in an agent.

Next steps#

Build an agent and tools in the Tutorial | Visual agents.