Function params description

[physimo]

I know it's kind of late to ask this, but where's the function parameter description went?
The beta version has this attribute in the GbnfJsonBasicSchema type, where you can make a description about the parameter like this (beta 12):

{
        description: "Will return current date and time",
        params: {
            type: "object",
            properties: {
                gmt: {
                    type: "string",
                    description: "Greenwich Mean Time in minus or positive value (e.g. \"+2\")"
                }
            }
        }
}

Now I see that it's no longer the case:
Object literal may only specify known properties, and 'description' does not exist in type 'GbnfJsonBasicSchema'.

May I know the reason why description attribute is removed?
Is it possible to just ignore the type and just place the description there so the model could read it?
Or is there other way to achieve this (giving parameter explanation or description to the model)?

[giladgd]

The description parameter was never officially supported before; you haven't gotten a type error for this previously because the type validation for the JSON schema wasn't complete in the beta version.
I'm working on enhancing the JSON schema support with additional features, so I'll introduce official support for description soon.

Since there is currently no official support for this property, if you ignore the type, it won't get fed to the model in some places.
It depends on the kind of model you use it with, since some chat wrappers include the original JSON schema, and some have their own layer of translation between it and the chat template's desired format.

The easiest way to check it is by putting something in the description and asking the model to tell you what it is to see whether it currently works with a given model.

[physimo]

The easiest way to check it is by putting something in the description and asking the model to tell you what it is to see whether it currently works with a given model.

I'll do that then.
But to be clear (correct me if I'm wrong), even if I ignore the type it will be passed to the model right? The only question here is whether the model can accept the description or not right?

I see some models like Meta llama 3.2 and Functionary 3.2 (which is based on llama 3.2 llama 3.1) has description in the function parameters like this and this, so I'll start with them.

[giladgd]

It depends on the model.
With some models it will, and with some it won't, hence I recommend you to test it.
If you find that it passes to the model, you can rest assured that it'll be passed every time you use that exact same model.

As of node-llama-cpp version 3.2.0, it won't be passed to Functionary models (Functionary 3.2 is based on Llama 3.1), but it will be passed to Llama 3.1 models.
The function description itself is always passed to the model, though.

[physimo]

Ah right, the Functionary 3.2 is based on Llama 3.1. That was my oversight.

So it turns out the params description is not passed with Functionary 3.2 (as you said), but why it's passed to Llama 3.1, even though Functionary 3.2 is based on Llama 3.1?
Is there any way to pass it? Since in their github it seems like it supports params description, it's only weird that it can't be passed but you can with Llama 3.1 right?
I'm using node-llama-cpp v3.0.1 at the moment.

[giladgd]

Functionary has its own kind of chat template that expects to get TypeScript type definitions for the available functions.
node-llama-cpp automatically converts the given functions and their schema to the model's desired chat format, which in this case is to rewrite it as TypeScript types.

In node-llama-cpp version 3.2.0 that implementation didn't take care of description in the transformation to TypeScript types.
Some other chat formats expect a JSON that's similar to the given JSON schema (like Llama 3.1), so the transformations done to the original JSON schema happened to pass the description parameter as-is in some cases.

I've just released a new version of node-llama-cpp (3.3.0) that includes many enhancements to the JSON schema grammar support, including support for the description parameter in function calling.
You can see an example of the function syntax in the chat template that functionary expects in the tests of node-llama-cpp if you're interested:

node-llama-cpp/test/standalone/chatWrappers/FunctionaryChatWrapper.test.ts

Lines 595 to 620 in 4d387de

	"// Supported function definitions that should be called when necessary.
	namespace functions {
	// Get a random number
	type getRandomNumber = (_: {min: number, max: number}) => any;
	// Some function
	type func1 = (_: {
	// Some message
	// minimum length: 3, maximum length: 10
	message: string,
	// Some words
	// maximum items: 5
	words: [string, string, ...string[]],
	// Some headers
	// minimum number of properties: 4, maximum number of properties: 12
	headers: {[key: string]: string},
	// Some mappings
	// minimum number of properties: 4, maximum number of properties: 12
	mappings: {a: boolean, b: number, c: string | null} & {[key: string]: string}
	}) => any;
	} // namespace functions",

[physimo]

Amazing!
Thanks for the update. It worked flawlessly!