Tutorial | Visual agents#

Get started#

AI agents can be useful across many industries to automate repetitive tasks, increase efficiency, and help non-technical users to access information from data.

Keeping up with AI market and research news is itself a challenge! Let’s build an AI agent to help an industry analyst with some of those repetitive tasks, such as searching the latest news and research, generating reports, and emailing reports to stakeholders.

Objectives#

In this tutorial, you will:

  • Build multiple agentic tools to perform different tasks.

  • Test tools to gauge performance.

  • Create a visual agent and add tools to the agent.

  • Interact with the agent in Prompt Studios.

Prerequisites#

To complete this tutorial, you will need:

  • Dataiku 13.4 or later, with the Advanced LLM Mesh add-on.

  • An Advanced Analytics Designer or Full Designer user profile.

  • A connection to at least one Generative AI model that supports tool calling and one that supports embedding. See supported models for tool calling and embedding (note that some models support both). Your administrator must configure the connection(s) in the Administration panel > Connections > New connection > LLM Mesh.

  • An internal code environment for retrieval augmented generation and agents. Your administrator must set this up in the Administration panel > Code Envs > Internal envs setup > Retrieval augmented generation code environment.

  • If you want the agent to send you a message, you’ll also need an administrator to grant access to an active messaging channel. This tutorial uses an email reporter, but other kinds of channels work similarly.

Create the project#

  1. From the Dataiku Design homepage, click + New Project.

  2. Select Learning projects.

  3. Search for and select Visual Agents.

  4. If needed, change the folder into which the project will be installed, and click Install.

  5. From the project homepage, click Go to Flow (or type g + f).

Note

You can also download the starter project from this website and import it as a zip file.

Embed the dataset#

The project includes a dataset of articles about the AI industry and technology published on the Medium website. We’ll work with a small sample to reduce computation cost.

The first step is to embed the articles in a knowledge bank. This will help the agent to understand the articles semantically and return better results than if we used the dataset alone.

See also

To learn more about embedding and Retrieval Augmented Generation, see Concept | Embed recipes and Retrieval Augmented Generation (RAG) and Tutorial | Retrieval Augmented Generation (RAG) with the Embed dataset recipe.

Create the Embed dataset recipe#

First, create the Embed dataset recipe.

  1. With the medium_articles dataset selected in the Flow, go to the Actions panel > LLM Recipes > Embed dataset.

  2. Choose the Embedding model your administrator has set up.

  3. Choose the Vector store type. ChromaDB is the default, though your administrator might have set up other options.

  4. Click Create Recipe.

Screenshot showing the steps to set up an Embed dataset recipe.

Configure the Embed dataset recipe#

Next, in the recipe settings, choose which columns to embed.

  1. For the Embedding column, choose text.

  2. For the Metadata columns, click + Add Column and choose title.

  3. Do the same for url and timestamp.

  4. Click Save in the top right.

  5. Run the recipe.

Screenshot showing the settings of the Embed dataset recipe.

When the recipe has finished running, the new knowledge bank medium_articles_embedded will appear in the Flow. We’ll use it to build our first agent tool.

Build agent tools#

We’ll create two managed tools — individual components that complete different tasks — that we will add later to an agent. Dataiku offers multiple built-in tools that you can configure in just a few clicks.

Add a knowledge bank search tool#

The first tool specializes in searching knowledge banks like the one we just built.

  1. Go to GenAI (GenAI icon.) > Agent Tools.

  2. Click + New Agent Tool.

  3. Choose Knowledge Bank Search from the available tools.

  4. Give the tool the Name search_articles.

  5. Click Create.

Steps to add a new knowledge base search tool.

Next, configure the tool to use the knowledge bank. We can choose from among the data columns we embedded for the agent to include as metadata and sources in its responses. We’ll also add some descriptions that give the agent some context about the knowledge bank and how the tool should be used.

  1. In the Design tab of search_articles, choose medium_articles_embedded as the Knowledge Bank.

  2. In the Knowledge Bank description, copy and paste the following text:

This knowledge bank contains articles posted on the Medium website about the AI and machine learning industry. Each article has a title, text, url, and date.

  1. In the Retrieval section, next to Metadata to retrieve, select title and url.

  2. Under Sources > With role, click + Add Role and select title, then for Select metadata, select title.

  3. Add another role for url.

  4. In Additional description, copy and paste the following text:

This tool is for the agent to retrieve records that correspond to a given topic.

  1. Click Save.

Steps to configure the article_search tool.

Test the search tool#

We can run a quick test of the tool to see how it performs. This gives a behind-the-scenes look at how the tool will run a query. Later, we’ll interact with the agent using a chat interface.

Let’s look for any articles that deal with deep learning.

  1. Still in the Design tab, move to the Test window.

  2. Replace the text Ask your question here with the following text:

What is the latest news on deep learning?

  1. Click Run Test.

Steps to test the articles search tool.

The tool returns several pieces of useful information under the test window, including:

Response

Includes the content and metadata of relevant articles, along with sources. You can change the number of documents, metadata, and sources retrieved in the Design tab.

Trace and Log

Can help audit and debug an agent’s actions.

Tool Descriptor

Provides information about the tool for agent use.

Add a messaging tool#

The next tool to build sends a message summarizing the articles returned using the knowledge bank search tool.

Note

You can send a message via Slack, Microsoft Teams, or email. An administrator must configure these settings. This tutorial uses email, though feel free to experiment with the message type of your choice and adapt the steps below accordingly.

  1. Return to the Agent Tools page and click + New Agent Tool.

  2. Choose the Send Message tool.

  3. Add the Name send_email and click Create.

  4. In the tool Design page, choose the Integration type of Mail.

  5. For the Channel, choose the channel configured by your administrator.

  6. Add the following text to the corresponding boxes:

    • To: Your email address

    • Subject: AI industry news report

    • Tool description > Additional description: This tool should be used to send a summary of the most relevant articles found by the search_articles tool, via email.

  7. Click Save.

Steps to add a new messaging tool.

Test the messaging tool#

Let’s run a quick test to make sure this tool is working correctly.

  1. In the Test window, optionally add your own message in the Test query box, replacing the text Message to be sent here.

  2. Click Run Test.

  3. Check that the Tool output window returns ok. You can also check your email or other messaging tool to find the test message.

Steps to test a new messaging tool.

Now that we have two tools that are working separately, let’s put them together to build an agent!

Create a new visual agent#

We’ll first create the agent, then add the tools.

  1. Go to GenAI (GenAI icon.) > Agents & GenAI Models.

  2. Click + New Agent.

  3. Choose Visual Agent.

  4. Give it the name Industry tracking agent.

  5. Click Create.

Steps to create a new agent from the Flow.

Configure the agent#

Next, configure the agent.

  1. In the agent Design tab, choose a model from available connections in the LLM dropdown.

  2. Copy and paste the following text into the Additional prompt window:

 You are a helpful assistant specializing in retrieving and summarizing industry and research news related to artificial intelligence and machine learning.

 The user will input a topic. If the input topic is related to the AI industry, you are tasked with the following actions:

1. Search the articles from the knowledge bank focusing on the input topic.
2. Generate a concise summary of the relevant articles.
3. Send an email containing the summarized findings to stakeholders, using the timestamp to put the most recent articles first.

 In all other cases, respond to the best of your knowledge and explicitly mention that this answer does not come from the knowledge bank.

  1. Click Add a Tool and choose the search_articles tool from the dropdown.

  2. Add a Tool again and choose the send_email tool.

  3. Click Save.

Steps to configure the agent.

The prompt instructs the agent to be helpful, so it should give polite and cheerful responses. It also tells the agent to give responses only about topics it can see in the knowledge bank. We’ll test this later!

Tip

The agent is now an object in the Flow and is version controlled. You can double-click on the visual agent icon to edit this version, v1, or create new versions and keep a history of your changes.

Test the agent#

Just as you tested individual tools, you can also test the agent to see how it responds and make sure there are no errors.

  1. In the Test window, replace the text Put your query here with this text:

What is the latest news on deep learning?

  1. Click Run Test.

Steps to test the agent.

The agent has responded with several relevant articles and sent an email with the summaries.

The message sent from the agent quick test.

Now let’s interact with the agent with a more user-friendly interface.

Note

The responses from your agent throughout this tutorial might look different depending on the models used and can differ from run to run.

Chat with the agent in Prompt Studio#

Agents automatically become objects available in the LLM Mesh, which means you can connect to them via a Prompt Studio or LLM visual recipes.

The chat interface in Prompt Studios provides a good platform to chat with agents and see how they perform before deploying them to stakeholders.

Let’s chat with the industry tracker agent!

  1. Go to GenAI (GenAI icon.) > Prompt Studios.

  2. Click + New Prompt Studio.

  3. Give the studio a name, such as Agent chat and click Create.

  4. Select the Chat option and click Create again.

    Steps to add a new Prompt Studio.

  5. In the Prompt design page, for the LLM, select Agent - Industry tracking agent.

The new Prompt Studio is connected to the agent, and you can chat with it here. We’ll start with the same question we tested earlier to compare the experience.

  1. In the chat message box, copy and paste this question, and send it to the chat:

What is the latest news on deep learning?

  1. If the agent responds that it has sent an email, you can ask it to post the summaries in chat too:

Please send the summaries here too.
Screenshot of chat with the agent in Prompt Studios.

The agent should respond with the summaries within the chat.

It’s also a good idea to test an agent with some queries outside the bounds of its knowledge. This will test whether the agent is behaving according to the prompt we gave it.

In the chat message box, copy and paste this message:

What is the weather in Paris today?
Screenshot of chat about the weather.

In this case, the agent acted as expected based on the prompt we gave it, stating that it does not have access to real-time weather data. And it helpfully offered a suggestion on how to find the correct answer to our question.

Feel free to keep chatting with the agent and test other questions!

Next steps#

Congratulations on building your first agent! If you’d like to keep experimenting, here are a few ideas:

  • Search articles in real time. For simplicity, this tutorial works with a static sample of articles. To build an agent that finds the latest news and research in the real world, you could add tools to:

    • Search the web for a topic using the Search the Web tool, which requires a Google account and API key.

    • Store results into a dataset using the Dataset Append tool.

    • Search the dataset using the Dataset Lookup tool.

    • Send reports using the Send Message tool.

  • Change the agent’s behavior. You might modify the Additional prompt in the agent and see how it behaves differently.

    • Take out instructions to be helpful and see how its tone changes.

    • Remove instructions about how it should respond to out-of-bounds topics and test its responses.

    • Add instructions on how to format the emails for better readability.

See also

See the reference documentation to learn more on Agentic AI.