Troubleshooting Resource Connections
Introduction
Resources come in many shapes and sizes. In this guide I hope to go over troubleshooting steps you can take to help troubleshoot any resource which isnât working.
General Troubleshooting Tips
General Troubleshooting Tips
âError handling
The errors returned by different API services (e.g Salesforce, Jira etc) are not standardised. You may see different errors/error codes for the same underlying issue, depending on the API/resource in question and how those requests are handled. In some cases, Retool may abstract away verbosity from the error, due to being unable to handle the response returned by the API service in the backend before forwarding the error to you in the frontend. Which is why it can be a good troubleshooting step to test issues in Postman or another local client (to remove the Retool backend handling out of the troubleshooting process)
â Different Errors
Different errors being returned after a change to a resource/query is usually a good sign that the edited field is/was the cause. If unsure where the issue lies, make sure to keep track of all changes which cause new errors, and try different combinations of values for these fields (within reason)
âTry the request elsewhere
Try reproducing the request elsewhere, ideally in Postman or via another SaaS tool. If itâs working in another tool, then the issue likely lies with Retool. Please note, this isnât always the case. e.g local requests in Postman might work for example, if youâre making requests in Postman from your company wifi, with IPs which are already allowlisted.
Common Errors
Common Errors
âConnection Timeout
This occurs when a connection takes too long to be established between the client(Retool) and server.
Potential causes: network issues, server overload (too many active connections), network firewall restrictions, query-level timeout settings, environment variable (self hosted) timeout settings, api/service-level timeout settings or limitations, load balancer/firewall or other infrastructure timeout settings.
Solution: Check the Timeouts section below for a detailed breakdown.
âECONNRESET
This indicates the connection was unexpectedly closed by the server (the API/service you are using).
Potential causes: Server restarts/maintenance, server overload (too many active connections), application crashes/bugs, packet loss or latency.
Solution: Troubleshoot by checking server logs. In this case, the logs for the api/service you are using.
For self hosted customers, check your container logs to see if thereâs a more verbose error.
âENOTFOUND
A network error that occurs when Retool cannot find the host it is trying to connect to.
Potential causes: DNS issues, incorrect URL/host, API server/infra is down, network connectivity issues.
Solution: Check DNS records are correct (if you own the service). Check the specified URL is correct. Try pinging the URL in your terminal/infra to see if there are any infra issues or if the URL in question is problematic.
âECONNREFUSED
The connection was refused by the server. Usually seen when connecting to a host that is not listening on the specified port, or the requester is not permitted to access that port/IPPotential causes: Permission or allowlisting issues with API. Using the wrong port/host URL/endpoint. Server not running on specified port/Server issues.
Solution: Check allowlisting/API permission settings for the service. (if you own the service): Verify the server is running and accepting connections (on the correct port/IP/domain). Check firewall rules.
â200 errors
Some APIs (e.g Slack) confusingly return a 200 with an error message attached, instead of returning a 4XX error (which is best practice). This can cause issues if you donât expect and manually handle these responses, and will be confusing when debugging your app/workflow, as it will look like the request was successful at first glance.
â5XX errors
These indicate server-side issues. Specifically:500 Internal Server Error - general error on server-side code.
503 Service Unavailable - Server overloaded or down for maintenance.
Potential causes: Invalid JSON returned by API, Incorrect API endpoint, API overloaded or down, server side networking or infrastructure issues.
Solution: Try making the same request in Postman with the same inputs, if it works, validate the returned JSON. Make sure the URL/endpoint is correct. Check the status page for the service or contact their support to confirm if itâs an issue on their side (especially if the request is also not working from Postman). Try adding retries into your request if it is working intermittently. Check server logs (if owned)
â400 Bad Request
Malformed/bad request sent to server.Potential causes: Invalid JSON/parameters or JSON format/structure in request. Incorrect endpoint. Missing required parameters. Wrong âcontent-typeâ header value. Wrong request method (GET/POST etc). Payload too large. API server error
Solution: Check API docs for service to verify request is accurate and for any missing params or âgotchasâ. Validate payload, query params and headers. Make sure content-type is specified and is the correct value (if using a method with a payload). Make sure the request method is correct (GET vs POST). Make sure the URL is correct and if dynamic, that all variables are being calculated correct at runtime. If the request is working intermittently, add some retries to account for issues on the API server side. (Consider moving requests to Retool workflows to make retries and time between retries more configurable)
â401 Unauthorized
Authentication required to access the resource.Potential causes: Invalid authentication credentials provided, missing authentication headers, OAuth2 flow not finalised, incorrect resource credentials (user name/password/client id/secret/scopes etc), incorrect authentication flow.
Solution: Double check your credentials are correct, using the correct API key etc. Use a Request Bin tool/Retool workflow as your endpoint to see if the token values are being sent properly in the request. Make sure your Oauth2 flow is working and authentication was successful (check the token status in the resource). Make sure you are providing the correct scopes in the OAuth2 setup for the endpoint you are accessing, and that the callback url is correct. Check the api docs, and make sure you are passing in your headers/query/payload params correctly for the api, and that the endpoint is correct.
â403 Forbidden
Client does not have permission to access the resource. Verify authentication and authorization.Potential causes: Invalid scopes, IP restrictions, ACL restrictions/user permission restrictions, API usage restrictions, resource ownership,
Solution: Check scopes, make sure Retoolâs IPs are allowlisted (cloud), check api permissions. Check you have access to the api resource in question (e.g maybe youâre accessing something private or owned by another user). Check user permissions for the API.
â404 Not Found
The requested resource doesn't exist on the server. Check the URL for errors. Verify the resource exists.Potential causes: Invalid URL/endpoint, invalid resource identifier (in query parameters/payload/url), incorrect permissions, resource was deleted or moved,
Solution: Make sure any dynamic URLs are correctly resolving. If using an ID in a query parameter, JSON payload or directly in the URL; independently verify that the ID exists in your API serviceâs frontend and that the request format is correct as per their API docs. Make sure you have permission to view the resource (as depending on error handling, the API may return a 404 instead of a 401 or 403)
â429 Too Many Requests
Youâve hit a rate limit for the api, or the server is under heavy load.Potential causes: Using the API too frequently with your credentials, either too much burst, or sustained volume of usage. Sometimes the server may return a 429 during heavy load (global rate limit), even if your own rate limits are not exceeded.
Solution: Add retries with an exponential back-off to help guarantee delivery, make your api usage more efficient to minimise the amount of calls needed. Check for any unnecessary apps/workflows eating up usage. Consider upgrading plans to increase usage limits (for the API service, assuming they offer more usage on higher plans)
Networking
Networking
âNetwork restrictions
If allowlisting is configured on the API server side of your resource, then you may get an error message back depending on where you are querying the API from.
When testing things locally on a local client, (e.g. Postman) remember that requests will come from your own IP address, which may already be added to the APIâs allowlist if working from the office.
When testing on cloud, requests are proxied through our backend and come from our own IPs (see docs below)
For self hosted instances, requests will come from your egress/public IPs.
âCloud
ââAllowlisting IPs
If your resources have restrictions on which IPs may be used for access. Make sure that youâve added the respective retool IPs to the respective resourceâs config. (e.g in the Salesforce OAuth app settings).
Symptoms of misconfigured IPs allowlists may include: ECONNREFUSED errors, ECONNRESET errors, or connection timeouts.
For us-west-2 (default IPs) see these docs: https://docs.retool.com/data-sources/concepts/ip-allowlist-cloud-orgs
Similarly for eu-central-1, see our outbound region docs:
Enable Retool outbound regions | Retool Docs
ââEU/US Outbound Regions
For cloud Retool organizations, resources will default to using us-west-2 as the outbound region. You can overwrite this default in the advanced org settings page:
Otherwise, for more granular control, you can further overwrite the region again inside the resource itself:
Remember: Overriding the default outbound region will change which IPs requests are sent from, so if you have allowlisting on your resource side, make sure to add the respective IPs for each region as per the docs above in the Allowlisting IPs section.
âSelf-hosted
ââAllowlisting
For self hosted instances, the IP(s) that need to be allowlisted will be the public IP(s)/egress IP(s) for your server. For cloud providers such as AWS, this is usually found on the serverâs information page. Typically this is just one IP, but depending on your infrastructure setup there may be more. Make sure to add your public IP to the APIâs allowlist if you restrict access by IP.
Timeouts
Timeouts
There are 4 main causes of timeouts in Retool. For Cloud customers, this is simplified somewhat, and is limited to causes 1 and 2 below. If a request should only take a few seconds, but is timing out, this is usually an indicator of a networking issue.
1. Query-level timeout settings (inside Retool)
New queries have a default timeout of 10,000ms (10 seconds). This can be raised to a maximum of 600,000ms (10 minutes) on cloud as of writing.
For self hosted instances, this timeout is reinforced by the DBCONNECTOR_QUERY_TIMEOUT_MS
environment variable. If this env var is set to lower than your query-level timeout, then the env var value will take precedence.
Sometimes, this setting will be greyed out (e.g the snowflake resource). In these cases, a query-level timeout cannot be configured, and it will instead default to the DBCONNECTOR_QUERY_TIMEOUT_MS (10 minutes on cloud) or the highest possible timeout for that resource type.
As you can see, my query with a timeout of 11 seconds still works even though the timeout field is greyed out at 10 seconds.
2. Resource timeout settings (outside of Retool)
Some APIs/DBs may allow you to configure connection timeout limits on their side. Please note that some APIs may have a default, unconfigurable timeout limit which will take precedence over any Retool-specific values. Try to see if these values can be configured for your API if youâre seeing the queries return a timeout error before the retool-imposed value is reached.
Timeouts can also occur for a resource, if there are network restrictions in place. For example, in the image below, I have configured an AWS RDS DB with no permission to access the DB on port 5432. Instead of returning an ECONNREFUSED error, we get back a timeout instead.
3. DB_Connector level timeout settings (retool deployment environment variables)
For self hosted customers, if your queries are timing out before hitting the query-level timeout, make sure that the DBCONNECTOR_QUERY_TIMEOUT_MS environment variable is set to the correct value, and that youâve restarted your containers to apply these changes (depending on deployment type)
Queries environment variables | Retool Docs
4. Infrastructure level timeout settings (where Retool is hosted)
Load balancers are usually the biggest cause of infra-level timeouts, as they usually have a default configured timeout of around 2 minutes. Make sure that these values have been increased, along with any other potential infrastructure components which may enforce a timeout (such as firewalls).
Intermittent Connection Issues
Intermittent Connection Issues
âAllowlisting Issues
You may see intermittent issues if not all of the IPs are allowed properly. Often, when allowlisting a CIDR range (e.g 35.90.103.132/30) the /30 isnât supported by the third party allow list, which means that the 3 other ip addresses which it should represent are not being allowed, and if requests happen to originate from those IPs (75% chance) weâll see it fail 75% of the time.Solution: Try adding the individual IPs instead of the CIDR range.
âService Issues
If a resource was previously working, and thereâs been no change to your resource, or retool version upgrades/self hosted changes, then the service itself is probably having issues. This can happen relatively often depending on the API you are using. For example, OpenAI had an API incident recently, which stopped OpenAI resources from working in Retool. This may break other things, for example, Retool currently uses OpenAI embeddings in vectors, so vectors would stop working entirely if OpenAI is having an incident.
Solution: Checking the status page for the respective service is recommended, but is not always a source of truth. Try contacting the support for the respective service if you believe it is an intermittent issue which doesnât line up with any changes to Retool.
âRate Limits
Rate limiting is very common, and the impact/symptoms is often the same as if the service were down, depending on how good the error handling is for the service. Ideally if the service returns a 429, or a descriptive error message, weâll know why the issue is occurring. Sometimes APIs will return a 4XX or 5XX error, or will even return an HTML webpage instead of a JSON response, which can be quite confusing.
Solution: Try calculating your total rough API usage or check if your overall app/workflow usage has increased recently. Perhaps someone is working on a new app or workflow using this resource, and itâs misconfigured or eating up a lot of requests. Add retries with an exponential back-off.
âInput Changes
Sometimes queries can fail depending on the input for the query. Usually this is because the JSON response has some invalid characters/format which is causing a 500 error in retoolâs backend when we try to parse the response.
Solution: Confirm if the JSON is valid by recreating the request in Postman, and compare the response of a successful request vs an unsuccessful request.
Test Connection Button Reliability
Test Connection Button Reliability
The âtest connectionâ button is not always a source of truth for gauging if a resource is set up correctly. This button triggers a preset âtestâ request for the resource type you are connecting with, using the values you have supplied in the resource setup. It is recommended that you try creating a query in an app, and confirm the error to make sure. This may return a separate error which is easier to diagnose.
SSH Tunnels in Resources
SSH Tunnels in Resources
If youâre using an SSH tunnel to connect to your resource in Retool, this adds an extra layer of complexity for when things go wrong, as it could be an issue with the tunnel connection itself, or an issue with the SSH server -> api server connection. Generally itâs best to access the SSH server logs to understand where the issue lies.
If youâre seeing intermittent issues with a resource using an SSH tunnel, you may have too many open concurrent SSH tunnel requests which are hitting a Retool backend or infrastructure limitation (self hosted). Try spreading out your requests to see if this reduces these errors.
We have some docs on debugging SSH connection issues here: Configure SSH tunneling for resources | Retool Docs
TLS/SSL
TLS/SSL
Make sure your certificates have a valid expiry date, and were issued by a recognised CA. Some resourcesâ drivers have limited functionality, so if full verification is not working, try one of the other verification methods.
Check our documentation on this here: Configure SSL connections for data sources | Retool Docs
Required/Non-Required Fields
Required/Non-Required Fields
Remember to check whether or not all fields are actually required. If thereâs no * symbol for a field, it should not be required. For example, base URL is not a required field on a REST API resource, not specifying this field allows you to dynamically calculate the entire URL in an app, or specifying it allows you to restrict where the underlying authentication can be used (that base URL only). Remember to include http:// or https:// in your URLs, or the request wonât work.
Some fields, although not required, add extra functionality if specified. E.g, while the database name is not required to run a DB query, it may be used for introspection (to generate the schema) depending on the resource/retool version.
Always check whether or not fields in your resource are actually required or not, as you may be adding unnecessary, incorrect values which are only required for certain use cases for the given resource/api. E.g If the 3rd party api offers both a self hosted and cloud solution, different fields may be required to set up the connection depending on which type of customer you are.
Debugging Sent/Token Values
Debugging Sent/Token Values
If youâre not sure whether or not your authentication tokens or other values are being properly sent to your resource, you can configure a Retool workflow or request bin to be the base URL of a REST API resource. Add your authentication to the resource, and when you trigger a query using this resource, it will pass the token to the requestbin or Retool workflow, where you can check the logs for the raw value.
If you add a return block to your workflow, you can use this to return the value back as your query response, which allows you to use it elsewhere in an app if needed.
This method is also useful to check whether dynamic values for query parameters/headers/post body are being calculated and sent properly in the request, but especially useful for authentication parameters which are calculated inside the resource, as we wonât otherwise have access to preview them inside an app.
REST APIs
REST APIs
âREST APIs as a workaround
If an API with a preconfigured resource in Retool (e.g Slack/Google Sheets) is not working, or has limited functionality, you can try connecting to it using a generic REST API resource in Retool instead. Although this is more complicated to set up and manage, it provides much more flexibility, especially if there are any bugs or regressions affecting those prebuilt resources.
Prebuilt resources usually have more specialised connection options (e.g service accounts for google), so you may have to use more generic authentication methods (e.g OAuth2 ) when configuring a generic REST API resource.
âPassing in parameters
API docs for some APIs are not always clear. Sometimes when referencing âparametersâ this may mean query parameters or body parameters, though for some methods e.g âGET / DELETEâ no body is required.
Remember when supplying a raw post body, to include the content type header for your payload, or the API server will likely be unable to receive your request properly and will usually return a 4XX error.
Key:Value
content-type:application/json
Incorrect Credentials
Incorrect Credentials
Although obvious, you should always double check the credentials entered are valid, as there can be some weird edge cases which Iâll touch on here:
If youâve copied the values over from an external source, make sure there are no special characters being added. Sometimes what looks like a space, can actually be an invisible line break character, which may cause issues with your auth.
Check each field for extra spaces, before and after the text.
Check each field for new lines and to make sure that no extra characters were accidentally entered in the string.
When dealing with environment variables (self hosted customers), make sure to wrap values with quotes as needed (depending on deployment type)
Case Sensitive Values
Case Sensitive Values
Some APIs have case sensitive requirements for query parameters, headers or post body params (the parameter key names, not the key values). This is usually considered bad practice, and is definitely something to be mindful of. Requests proxied through the backend may standardise key names by making them lower case, depending on the resource and which library we are using behind the scenes to process the query data. Using Postman is a good way to find these edge cases.
Parameter order
Parameter order
Similar to the case sensitive values section, some APIs require parameters to be received in a certain order, this is considered bad practice, and the library Retool uses to make requests may change the order of query parameters at runtime.
Authentication Types
Authentication Types
Itâs worth noting that not all APIs follow the same authentication flow for certain auth types. E.g they may require an extra variable or require params in a certain order.
For these cases, the authentication may not work, and you may have to try to manually create the flow that they use, by using a custom auth in Retool.
âBasic
If a basic authentication is not working, itâs most likely an invalid credentials issue. If the creds are fine, perhaps the API service is expecting the token in a different way, or has a non-standard flow that it uses. Usually you can manually re-create the basic authentication by base-64 encoding your user and password with a colon in-between i.e:
user:password
This should get passed in as an authentication header i.e:
Authorization: Basic ENCODEDVALUE
You may need to tweak the above to suit your API if it has a non-standard setup.
âOAuth2
ââCallback URL
One of the most common things that goes wrong with OAuth2 resources, is that the callback URL changes depending on whether the auth is shared amongst all users or not.
Here are some screenshots for both checkbox values. You can see the callback URL changes, which means youâll need to update this value in your OAuth app, or the tokens will not save properly.
ââScopes
Scopes are not required for the OAuth2 flow, but if your OAuth2 app/API service has scopes defined, make sure to select the correct, matching scopes in Retool or youâll probably see 401/403 errors when trying to make requests to the resource.
âCustom Auth
Custom auth is useful for non-standard auth flows, and can also be used for workarounds when it comes to saving and re-using tokens. For example, you could add a step to your custom auth flow which sends the token to a Retool Workflow to save the value to a DB, effectively allowing you to use the token elsewhere.
If youâre running into issues refreshing the token in your custom auth (non-oauth2), try setting the refresh auth flow to the same as your authentication flow, and use a time based expiration set to a value lower than the length of your access tokenâs life span.
Remember to test the auth workflow to make sure the JSON paths are correct.
Create a New Resource
Create a New Resource
With newer versions of Retool, come updates to the resource configuration page. So as not to break existing behaviour for existing resources, some new features will not automatically apply to existing resources. If youâre running into issues with an existing resource, consider creating a new resource from scratch (youâll need to update references to it in your apps, which you can do fairly quickly with the âusageâ tab). Creating a new resource from scratch may unlock new features to fix the issue, or may fix any corrupted/underlying configuration issues with the existing resource.
AI Actions / Vectors
AI Actions / Vectors
Retool AI Actions with vectors use OpenAI embeddings to generate better answers. If AI actions have stopped working (even for non openAI keys) itâs most likely failing to get the embeddings due to an outage with OpenAI, you can check their status page here: https://status.openai.com/
Without supplying your own key, you will be subject to an org-wide rate limit, so bear this in mind in case you run into rate limiting errors.
RetoolDB and self hosted DB
RetoolDB and self hosted DB
RetoolDB acts like a normal Postgres database, if you have the connection string, you can still connect to it in retool via a Postgres resource if you need any custom functionality which is restricted in the pre-built RetoolDB resource, or if there are any issues accessing your DB via the normal RetoolDB resource. This also goes for the (self hosted) Retool Storage DB which contains all of the instanceâs data. Connecting this Storage DB as a resource allows for a lot of workarounds by directly manipulating the DB values (use with caution).
Database Resource issues
Database Resource issues
âPrepared statements
The most common issue with Database resources is encountered when trying to dynamically select a table name (or some other dynamic properties) in an SQL query with javascript {{ }}. To dynamically select a table, you will need to disable prepared statements in the resource itself. This is often a good troubleshooting step when SQL queries with dynamic data are not behaving as expected.
âConcurrent Connection limits
For heavily-used database resources, you may see connections intermittently fail. In these cases, itâs usually the connection limit (enforced on the DB server side) being reached. This will kill new connections or cause them to time out depending on the DB setup.Token Refresh Issues
Token Refresh Issues
If your resources (especially OAuth2 resources) are working initially, but stop working after a short period of time (1-24 hours usually, depending on API) , then there is likely an issue with the refresh flow to generate a new access token. Sometimes this flow has separate scopes required ( usually called offline_access), so make sure to include those.
For OAuth2 resources, try setting the âAccess Token Lifespanâ in the advanced settings. You can also specify a refresh auth cadence for custom auth flows.
Please note, refresh tokens can be revoked by the API, which will cause the refresh flow to fail. Re-authenticating should generate a new refresh token to fix that. Some APIs do not allow for a permanent/auto-rotating refresh token, which means users will have to remember to re-authenticate every X days/months. You can try to extend the access token duration as long as possible in the settings to reduce the tediousness of rotating these tokens.
Consider adding an Auth trigger, to re-run the auth flow either on a time-based interval, or based on an API requestâs response code.
Manual Re-authentication Required
Manual Re-authentication Required
If you are using a custom auth, and are being prompted to re-authenticate your resource when you open your app, make sure your refresh flow is set up correctly. Make sure the auth trigger (example image above) is set, and set to run without prompting the user. If your refresh flow isnât working, you can try to add a refresh flow by copying the normal auth request flow. This isnât really feasible for standard OAuth2 as the user would have to manually re-authenticate each time (instead the refresh flow should use the refresh token to generate a new access token according to the API docs for the service)
If your OAuth2 resources are prompting you to re-authenticate on a regular basis, then there is likely an issue with the refresh token/flow. Make sure the refresh token is being saved properly after authenticating (you can check the token status in the resource), and that you have the required scopes to refresh the access token. You may need to manually create the re-authentication flow in a custom auth if it is non-standard.