Forum Post: Databricks AI Gateway as Custom OpenAI Provider — Simple Text/Chat Calls Mostly Work, Agents Always Fail

💬 Agents
,
1

Hi everyone,

I’m trying to connect Databricks AI Gateway to Retool using a Custom AI Provider (OpenAI schema), and I’m seeing inconsistent behavior:

  • The basic Retool AI functions “Generate Text” and “Generate Chat Response” mostly work (about 3 out of 4 requests succeed).

  • Retool Agents never work — they fail on every attempt.

  • I’m trying to understand whether this is a configuration issue, a limitation in Databricks’ OpenAI-compatibility, or something on Retool’s side.

Current Setup

Here’s how I’ve configured my Custom AI Provider:

  • Provider Schema: OpenAI

  • Base URL:

    https://******.cloud.databricks.com/serving-endpoints/ai-gateway-prod/served-models/gpt-4o-mini/invocations

  • Custom Headers:

    {
  "Content-Type": "application/json",
  "Authorization": "Bearer {{ secrets['stage/skills-retool/ai-gateway/token'].token }}"
}

So essentially, I’m pointing Retool directly at the Databricks AI Gateway model’s /invocations endpoint while sending OpenAI-style request bodies.

Using this provider Retool AI actions (specifically Generate Text and Generate Chat Response) succeed most of the time.
But I’d estimate that roughly one out of four calls fail with:

image
image780×296 9.02 KB

In addition Retool Agents using this provider fail on every call, even with trivial prompts.

Do you have any idea why this is happening or how I can fix it? When I call the Databricks endpoint directly outside of Retool it works 100% of the time, but Retool’s AI actions sometimes fail - does Retool occasionally sends additional fields or parameters in the request body that might cause the Databricks endpoint to reject certain calls?

1 Like
4

Hi @TobiasOhlsson,

Is the Databricks tool you are trying to interact with their Mosaic AI Gateway?

Can you share your configuration set up in your Custom AI Provider page in Retool?