How would we add progress to a long-running tool call

[antonioalegria]

With the new Background Task support, where you can have a long-running task updating progress, I am wondering what's the right way to do the same with a long-running Tool call with multiple steps?

[JoshuaC215]

I think it’s still designed to run within a single agent step but I could be wrong, not totally sure what’s possible.

Want to give a little more detail of the control flow you have in mind or use case? 🤔 cc @peterkeppert

[peterkeppert]

This is correct, present intended purpose of background task is to display progress of activities between user's request and the following AI response. I assumed that all such background tasks have to complete before the AI produces an answer, because their result is a required input for answering. However, I am also curious about the use case.

[antonioalegria]

Use case is I have tool calls that can take a few minutes and have a few large steps - think for example scraping a web site with multiple pages with some post-processing afterwards (e.g. extracting information). It would be good to let the user know we're at step 2/5, etc...

[peterkeppert]

My understanding is that this still happens before user sees any AI response, because all 5 steps are required for the agent to answer user’s query, correct?

In that case, it would be my approach to treat each stage of the tool call as a separate Task, for example:

1. Task: Scraping homepage
2. Task: Scraping subpage
3. Task: Processing information
4. …

They would still be displayed in single Task container and user would see the progress clearly. Does this work for you?

[antonioalegria]

And how would I trigger that -- create tasks inside the tool call? -- and that would then show the right things in the app?

[peterkeppert]

Background Tasks don't work well along the default Tool Calls status info solution provided in the streamlit_app.py. That's because Background Tasks use the custom message to transmit status, whereas the default solution relies on the sequence of AI messages with tool calls and Tool messages to provide status updates. The Background Tasks' custom messages interfere with the order of messages assumed in the default solution. So these two generally shouldn't be used together (also, even if they worked together, there would be two status containers for the same thing, which would be confusing).

The default solution can provide information when tool call starts and when it ends. Background tasks are much more versatile - you can break down a tool call to a series of tasks or providing continuous status updates (using the .write_data() method for a running tool call. Downside is, that you have to emit the Task events yourself for all the tool calls, if you decide to switch to this implementation.

I dont' know the details of the tool call, but I assume you are able to write code within the function that executes the tool actions (scraping, processing). So then you can create tasks inside the tool call, and since their status updates dispatch events, they are immediately displayed in the UI. Crude example here: peterkeppert@ddd1d87

[antonioalegria]

I think this hasn't been added yet, so I'll create an issue to support Tool Calls with steps or sub-tasks.