Retool Agent dataset test case fails with error 400

[avr]

Hi

I ran into something odd when working with Retool Agents Evals and wanted to see if anyone else has hit this.

I’m creating a Dataset with a Test case that checks if a certain tool is used with a given input. The tool itself works as expected, but the test case still fails with this error:

400 Invalid 'tools[0].function.name': string does not match pattern.
Expected a string that matches the pattern '^[a-zA-Z0-9_-]+$'.

Here’s what I’m doing:

1. Create a new dataset
2. Add a test case of type Tool choice.
3. Select my tool from the dropdown
4. Set the input: “Find value for product with ID 39388.”
5. Set the expected argument productId = 39388.
6. Run the test cases in the eval

Expected: The test passes because the tool is called with the exact argument.
Actual: The UI shows the tool did run with the correct input, but the test still errors.

Would love to hear if this is a known issue or if I’m missing a setting somewhere.

Thanks!

[kent]

Hey @avr - looks like a bug; we're on it.

[Robin_Pham]

hey @avr, as a temporary solution before we fully push out a fix, you can rename your tool names to no longer have a . in them. As long as it's alphanumeric, OpenAI can interpret the tool name.