the [get_code] button in aistudio.google.com provide a code that doesn't run when grounding is switched on

[hitchhiker3010]

Description of the bug:

When I switch on the grounding

"""
Install an additional SDK for JSON schema support Google AI Python SDK

$ pip install google.ai.generativelanguage
"""

import os
import time
import google.generativeai as genai
from google.ai.generativelanguage_v1beta.types import content

genai.configure(api_key=os.environ["GEMINI_API_KEY"])

# Create the model
generation_config = {
  "temperature": 1,
  "top_p": 0.95,
  "top_k": 40,
  "max_output_tokens": 8192,
  "response_mime_type": "text/plain",
}

model = genai.GenerativeModel(
  model_name="gemini-2.0-flash-exp",
  generation_config=generation_config,
  system_instruction="You're an expert video parser and extracts the places mentioned in the given videos. You're very helpful and harmless.",
  tools = [
    genai.protos.Tool(
      google_search = genai.protos.Tool.GoogleSearch(),
    ),
  ],
)

Below is the error message I'm receiving

I've installed google-generativeai==0.8.3

I'm using grounding with video input, please help.

Actual vs expected behavior:

I'm using grounding with video input.
While that's running on the ai studio website, the code given in "Get Code" section doesn't work.

Any other information you'd like to share?

No response

[Gunand3043]

Hi @hitchhiker3010, We already have similar issue on this #650

For time being, you can try replacing the code below to make grounding work.

model = genai.GenerativeModel(
  model_name="models/gemini-1.5-pro",
  generation_config=generation_config,
  tools="google_search_retrieval"
)

Thanks

[hitchhiker3010]

Thank you @Gunand3043 for the reply. Can you please help me with the correct code to get the desired output.
Have gone through the documentation and tried implementing, but I'm getting the following error

InvalidArgument: 400 Search Grounding is supported only for text-only requests

[Gunand3043]

Alright, to use grounding with multimedia input, you need to use the new Google Gen AI SDK. You can also go through our quick-start cookbook.

I have prepared a sample notebook on how to use grounding with multimedia input. Please have a look.

Thanks.

[hitchhiker3010]

Thanks a lot @Gunand3043. The attached notebook was helpful but I was looking for search grounding [which was possible with video in ai studio] as well, I tried using google_search_retrieval tool which gave me the following error ClientError: 400 INVALID_ARGUMENT. {'error': {'code': 400, 'message': 'Search Grounding is supported only for text-only requests.', 'status': 'INVALID_ARGUMENT'}}.

[Gunand3043]

Hi @hitchhiker3010 ,
The code returned by AI Studio uses the Gemini API SDK, which, according to the documentation, does not support multimodal (text-and-image, text-and-audio, etc.) prompts with grounding enabled. The team is already working on fixing the issue.