Help debug REST API behaving differently than image component

Queries and Resources
1

I have an image component which displays an image URL correctly. I would like to use the base64 encoded image for another query, so per advice here I addd a REST API query for the exact same url as the image component

image
image1308×1004 48.7 KB

This has worked in the past, but since sometimes last week, the REST API started failing (with some kind of signature error from the remote server) even though the image component still displays the image correctly. The image url also loads successful on its own in the browser directly (including in incognito mode, so no caching, cookies etc. issue).

Could you advice on how Retool is actually fetching the the REST API that could cause this difference in behavior?

The URL looks like thishttps://something.imgix.net/some-image.jpeg?auto=format%2Ccompress&ixlib=js-2.3.2&s=some-key. I have tried tweaking headers options ("Exclude default headers" etc.) to no avail.

2

If you go to the Output below the API query and select "API Request" if will show you the details of the request (there's a lot more there than I am showing in the screenshot; yours will have all the headers and body you are sending in the request).

image
image860×887 53.9 KB

I'd suggest starting your troubleshooting there, along with the error codes in the response.

3

I did check the API output, but it isn't informative. It just seems like somehow the way that Retool does the API call caused this error - as I mentioned before, this URL works both in the image component and in the browser directly. I also tried changing the user agent to Chrome to no avail.

How does the REST API call in Retool works? Does it go through some proxy? Was there any changes there in the last week that could caused this problem (since this worked before last week)?

{
  "request": {
    "url": "...",
    "method": "GET",
    "body": null,
    "headers": {
      "ot-baggage-requestId": "undefined",
      "x-datadog-trace-id": "8821928040033427533",
      "x-datadog-parent-id": "917006553634950919",
      "x-datadog-sampling-priority": "-1",
      "traceparent": "00-00000000000000007a6dc8cfd440204d-0cb9dc9a1e1dcf07-00",
      "tracestate": "dd=s:-1"
    }
  },
  "response": {
    "data": {
      "message": "sig_invalid"
    },
    "headers": {
      "x-imgix-proxy-status": [
        "403"
      ],...
    },
    "status": 403,
    "statusText": "Forbidden"
  }
}
4

@x.h Yes, RestAPI requests will be performed from our cloud backend using the fetch API. It looks like your request is being blocked by some authentication logic on the URL. It's returning 403 Forbidden status. I'd check if you need any access token or firewall exception to allow our server to make the request successfully.

5

@Harry_Doan the image loads fine in the browser (including in incognito mode) without any additional authentication. It still seem to me that the culprit is how Retool handles the fetching.

Is there anything in the Retool backend that would potential modify the url? The image is signed for its specific url and query parameters.

6

@x.h would you mind sharing or DM-ing me the link of the image so that I can verify? We don't do anything special in our backend. It looks like the message is blocked at your proxy level so I'd guess that's where you want to dig into.

7

@Harry_Doan hmm how to DM in this forum?

8

You can click on my name and click "message". Maybe you're a new user and can't message. Let me start a thread with you instead.

9

In our private thread, @x.h has resolved the issue by adding some missing headers to the API query

I did more digging and I think the salient difference between the Retool backend REST API fetch and the image component is having the referer and user-agent header. Once I imitated those header in the API call, it now works. Thanks for you help!

10

Unfortunately the issue wasn't fully resolved via adding the new headers; the server was very picky about non-browsers. I ended up "resolving" the issue by switching to a js script that calls fetch directly which bypasses the need to proxy the API call through Retool's server.