How to pass files to LLMs

[thierry-rietsch]

Hi
Thanks for crafting ControlFlow. I've just read through the documentation and I like the idea and approach of it very much.

Perhaps I have missed the relevant part but how can I pass files (e.g. images) to a task?

Thanks for helping me out, Thierry

[leonkozlowski]

Hi @thierry-rietsch

It looks like you can do this with tools - check out this example: https://controlflow.ai/examples/features/tools#set-up-example-data

The example from the docs:

import controlflow as cf

agent = cf.Agent(name="FileSearcher", tools=[list_files])


@cf.flow(tools=[update_user], default_agent=agent)  
def file_search_flow(query: str, directory: str):

    # Task 1: Find files containing the search term
    found_files = cf.Task(
        f"Identify files in the directory '{directory}' that might "
        f"relate to'{query}' and return a list of paths.",
        result_type=list[str]
    )

    # Task 2: Analyze file contents and report findings
    cf.run(
        f"Analyze the contents of the files for information related "
        "to the search term.",
        instructions='You must update the user on all your findings to complete the task.',
        tools=[read_file],
        depends_on=[found_files],
        result_type=None  # We don't need a return value as we're printing directly to the user
    )

# Run the flow within our test environment
with setup_test_environment() as temp_dir:
    file_search_flow(query="launch meeting", directory=temp_dir)

You would define what read_file does as a tool

def read_file(filepath: str) -> str:
    """Read the contents of a file from an absolute filepath."""
    with open(filepath, 'r') as file:
        return file.read()