Troubleshooting Resource Connections
Introduction
Resources come in many shapes and sizes. In this guide I hope to go over troubleshooting steps you can take to help troubleshoot any resource which isn’t working.
General Troubleshooting Tips
General Troubleshooting Tips
Error handling
The errors returned by different API services (e.g Salesforce, Jira etc) are not standardised. You may see different errors/error codes for the same underlying issue, depending on the API/resource in question and how those requests are handled. In some cases, Retool may abstract away verbosity from the error, due to being unable to handle the response returned by the API service in the backend before forwarding the error to you in the frontend. Which is why it can be a good troubleshooting step to test issues in Postman or another local client (to remove the Retool backend handling out of the troubleshooting process)
Different Errors
Different errors being returned after a change to a resource/query is usually a good sign that the edited field is/was the cause. If unsure where the issue lies, make sure to keep track of all changes which cause new errors, and try different combinations of values for these fields (within reason)
Try the request elsewhere
Try reproducing the request elsewhere, ideally in Postman or via another SaaS tool. If it’s working in another tool, then the issue likely lies with Retool. Please note, this isn’t always the case. e.g local requests in Postman might work for example, if you’re making requests in Postman from your company wifi, with IPs which are already allowlisted.
Common Errors
Common Errors
Connection Timeout
This occurs when a connection takes too long to be established between the client(Retool) and server.
Potential causes: network issues, server overload (too many active connections), network firewall restrictions, query-level timeout settings, environment variable (self hosted) timeout settings, api/service-level timeout settings or limitations, load balancer/firewall or other infrastructure timeout settings.
Solution: Check the Timeouts section below for a detailed breakdown.
ECONNRESET
This indicates the connection was unexpectedly closed by the server (the API/service you are using).
Potential causes: Server restarts/maintenance, server overload (too many active connections), application crashes/bugs, packet loss or latency.
Solution: Troubleshoot by checking server logs. In this case, the logs for the api/service you are using.
For self hosted customers, check your container logs to see if there’s a more verbose error.
ENOTFOUND
A network error that occurs when Retool cannot find the host it is trying to connect to.
Potential causes: DNS issues, incorrect URL/host, API server/infra is down, network connectivity issues.
Solution: Check DNS records are correct (if you own the service). Check the specified URL is correct. Try pinging the URL in your terminal/infra to see if there are any infra issues or if the URL in question is problematic.
ECONNREFUSED
The connection was refused by the server. Usually seen when connecting to a host that is not listening on the specified port, or the requester is not permitted to access that port/IPPotential causes: Permission or allowlisting issues with API. Using the wrong port/host URL/endpoint. Server not running on specified port/Server issues.
Solution: Check allowlisting/API permission settings for the service. (if you own the service): Verify the server is running and accepting connections (on the correct port/IP/domain). Check firewall rules.
200 errors
Some APIs (e.g Slack) confusingly return a 200 with an error message attached, instead of returning a 4XX error (which is best practice). This can cause issues if you don’t expect and manually handle these responses, and will be confusing when debugging your app/workflow, as it will look like the request was successful at first glance.
5XX errors
These indicate server-side issues. Specifically:500 Internal Server Error - general error on server-side code.
503 Service Unavailable - Server overloaded or down for maintenance.
Potential causes: Invalid JSON returned by API, Incorrect API endpoint, API overloaded or down, server side networking or infrastructure issues.
Solution: Try making the same request in Postman with the same inputs, if it works, validate the returned JSON. Make sure the URL/endpoint is correct. Check the status page for the service or contact their support to confirm if it’s an issue on their side (especially if the request is also not working from Postman). Try adding retries into your request if it is working intermittently. Check server logs (if owned)
400 Bad Request
Malformed/bad request sent to server.Potential causes: Invalid JSON/parameters or JSON format/structure in request. Incorrect endpoint. Missing required parameters. Wrong ‘content-type’ header value. Wrong request method (GET/POST etc). Payload too large. API server error
Solution: Check API docs for service to verify request is accurate and for any missing params or ‘gotchas’. Validate payload, query params and headers. Make sure content-type is specified and is the correct value (if using a method with a payload). Make sure the request method is correct (GET vs POST). Make sure the URL is correct and if dynamic, that all variables are being calculated correct at runtime. If the request is working intermittently, add some retries to account for issues on the API server side. (Consider moving requests to Retool workflows to make retries and time between retries more configurable)
401 Unauthorized
Authentication required to access the resource.Potential causes: Invalid authentication credentials provided, missing authentication headers, OAuth2 flow not finalised, incorrect resource credentials (user name/password/client id/secret/scopes etc), incorrect authentication flow.
Solution: Double check your credentials are correct, using the correct API key etc. Use a Request Bin tool/Retool workflow as your endpoint to see if the token values are being sent properly in the request. Make sure your Oauth2 flow is working and authentication was successful (check the token status in the resource). Make sure you are providing the correct scopes in the OAuth2 setup for the endpoint you are accessing, and that the callback url is correct. Check the api docs, and make sure you are passing in your headers/query/payload params correctly for the api, and that the endpoint is correct.
403 Forbidden
Client does not have permission to access the resource. Verify authentication and authorization.Potential causes: Invalid scopes, IP restrictions, ACL restrictions/user permission restrictions, API usage restrictions, resource ownership,
Solution: Check scopes, make sure Retool’s IPs are allowlisted (cloud), check api permissions. Check you have access to the api resource in question (e.g maybe you’re accessing something private or owned by another user). Check user permissions for the API.
404 Not Found
The requested resource doesn't exist on the server. Check the URL for errors. Verify the resource exists.Potential causes: Invalid URL/endpoint, invalid resource identifier (in query parameters/payload/url), incorrect permissions, resource was deleted or moved,
Solution: Make sure any dynamic URLs are correctly resolving. If using an ID in a query parameter, JSON payload or directly in the URL; independently verify that the ID exists in your API service’s frontend and that the request format is correct as per their API docs. Make sure you have permission to view the resource (as depending on error handling, the API may return a 404 instead of a 401 or 403)
429 Too Many Requests
You’ve hit a rate limit for the api, or the server is under heavy load.Potential causes: Using the API too frequently with your credentials, either too much burst, or sustained volume of usage. Sometimes the server may return a 429 during heavy load (global rate limit), even if your own rate limits are not exceeded.
Solution: Add retries with an exponential back-off to help guarantee delivery, make your api usage more efficient to minimise the amount of calls needed. Check for any unnecessary apps/workflows eating up usage. Consider upgrading plans to increase usage limits (for the API service, assuming they offer more usage on higher plans)
Networking
Networking
Network restrictions
If allowlisting is configured on the API server side of your resource, then you may get an error message back depending on where you are querying the API from.
When testing things locally on a local client, (e.g. Postman) remember that requests will come from your own IP address, which may already be added to the API’s allowlist if working from the office.
When testing on cloud, requests are proxied through our backend and come from our own IPs (see docs below)
For self hosted instances, requests will come from your egress/public IPs.
Cloud
Allowlisting IPs
If your resources have restrictions on which IPs may be used for access. Make sure that you’ve added the respective retool IPs to the respective resource’s config. (e.g in the Salesforce OAuth app settings).
Symptoms of misconfigured IPs allowlists may include: ECONNREFUSED errors, ECONNRESET errors, or connection timeouts.
For us-west-2 (default IPs) see these docs: https://docs.retool.com/data-sources/concepts/ip-allowlist-cloud-orgs
Similarly for eu-central-1, see our outbound region docs:
Enable Retool outbound regions | Retool Docs
EU/US Outbound Regions
For cloud Retool organizations, resources will default to using us-west-2 as the outbound region. You can overwrite this default in the advanced org settings page:
Otherwise, for more granular control, you can further overwrite the region again inside the resource itself:
Remember: Overriding the default outbound region will change which IPs requests are sent from, so if you have allowlisting on your resource side, make sure to add the respective IPs for each region as per the docs above in the Allowlisting IPs section.
Self-hosted
Allowlisting
For self hosted instances, the IP(s) that need to be allowlisted will be the public IP(s)/egress IP(s) for your server. For cloud providers such as AWS, this is usually found on the server’s information page. Typically this is just one IP, but depending on your infrastructure setup there may be more. Make sure to add your public IP to the API’s allowlist if you restrict access by IP.
Timeouts
Timeouts
There are 4 main causes of timeouts in Retool. For Cloud customers, this is simplified somewhat, and is limited to causes 1 and 2 below. If a request should only take a few seconds, but is timing out, this is usually an indicator of a networking issue.
1. Query-level timeout settings (inside Retool)
New queries have a default timeout of 10,000ms (10 seconds). This can be raised to a maximum of 600,000ms (10 minutes) on cloud as of writing.
For self hosted instances, this timeout is reinforced by the DBCONNECTOR_QUERY_TIMEOUT_MS
environment variable. If this env var is set to lower than your query-level timeout, then the env var value will take precedence.
Sometimes, this setting will be greyed out (e.g the snowflake resource). In these cases, a query-level timeout cannot be configured, and it will instead default to the DBCONNECTOR_QUERY_TIMEOUT_MS (10 minutes on cloud) or the highest possible timeout for that resource type.
As you can see, my query with a timeout of 11 seconds still works even though the timeout field is greyed out at 10 seconds.
2. Resource timeout settings (outside of Retool)
Some APIs/DBs may allow you to configure connection timeout limits on their side. Please note that some APIs may have a default, unconfigurable timeout limit which will take precedence over any Retool-specific values. Try to see if these values can be configured for your API if you’re seeing the queries return a timeout error before the retool-imposed value is reached.
Timeouts can also occur for a resource, if there are network restrictions in place. For example, in the image below, I have configured an AWS RDS DB with no permission to access the DB on port 5432. Instead of returning an ECONNREFUSED error, we get back a timeout instead.
3. DB_Connector level timeout settings (retool deployment environment variables)
For self hosted customers, if your queries are timing out before hitting the query-level timeout, make sure that the DBCONNECTOR_QUERY_TIMEOUT_MS environment variable is set to the correct value, and that you’ve restarted your containers to apply these changes (depending on deployment type)
Queries environment variables | Retool Docs
4. Infrastructure level timeout settings (where Retool is hosted)
Load balancers are usually the biggest cause of infra-level timeouts, as they usually have a default configured timeout of around 2 minutes. Make sure that these values have been increased, along with any other potential infrastructure components which may enforce a timeout (such as firewalls).
Intermittent Connection Issues
Intermittent Connection Issues
Allowlisting Issues
You may see intermittent issues if not all of the IPs are allowed properly. Often, when allowlisting a CIDR range (e.g 35.90.103.132/30) the /30 isn’t supported by the third party allow list, which means that the 3 other ip addresses which it should represent are not being allowed, and if requests happen to originate from those IPs (75% chance) we’ll see it fail 75% of the time.Solution: Try adding the individual IPs instead of the CIDR range.
Service Issues
If a resource was previously working, and there’s been no change to your resource, or retool version upgrades/self hosted changes, then the service itself is probably having issues. This can happen relatively often depending on the API you are using. For example, OpenAI had an API incident recently, which stopped OpenAI resources from working in Retool. This may break other things, for example, Retool currently uses OpenAI embeddings in vectors, so vectors would stop working entirely if OpenAI is having an incident.
Solution: Checking the status page for the respective service is recommended, but is not always a source of truth. Try contacting the support for the respective service if you believe it is an intermittent issue which doesn’t line up with any changes to Retool.
Rate Limits
Rate limiting is very common, and the impact/symptoms is often the same as if the service were down, depending on how good the error handling is for the service. Ideally if the service returns a 429, or a descriptive error message, we’ll know why the issue is occurring. Sometimes APIs will return a 4XX or 5XX error, or will even return an HTML webpage instead of a JSON response, which can be quite confusing.
Solution: Try calculating your total rough API usage or check if your overall app/workflow usage has increased recently. Perhaps someone is working on a new app or workflow using this resource, and it’s misconfigured or eating up a lot of requests. Add retries with an exponential back-off.
Input Changes
Sometimes queries can fail depending on the input for the query. Usually this is because the JSON response has some invalid characters/format which is causing a 500 error in retool’s backend when we try to parse the response.
Solution: Confirm if the JSON is valid by recreating the request in Postman, and compare the response of a successful request vs an unsuccessful request.
Test Connection Button Reliability
Test Connection Button Reliability
The ‘test connection’ button is not always a source of truth for gauging if a resource is set up correctly. This button triggers a preset ‘test’ request for the resource type you are connecting with, using the values you have supplied in the resource setup. It is recommended that you try creating a query in an app, and confirm the error to make sure. This may return a separate error which is easier to diagnose.
SSH Tunnels in Resources
SSH Tunnels in Resources
If you’re using an SSH tunnel to connect to your resource in Retool, this adds an extra layer of complexity for when things go wrong, as it could be an issue with the tunnel connection itself, or an issue with the SSH server -> api server connection. Generally it’s best to access the SSH server logs to understand where the issue lies.
If you’re seeing intermittent issues with a resource using an SSH tunnel, you may have too many open concurrent SSH tunnel requests which are hitting a Retool backend or infrastructure limitation (self hosted). Try spreading out your requests to see if this reduces these errors.
We have some docs on debugging SSH connection issues here: Configure SSH tunneling for resources | Retool Docs
TLS/SSL
TLS/SSL
Make sure your certificates have a valid expiry date, and were issued by a recognised CA. Some resources’ drivers have limited functionality, so if full verification is not working, try one of the other verification methods.
Check our documentation on this here: Configure SSL connections for data sources | Retool Docs
Required/Non-Required Fields
Required/Non-Required Fields
Remember to check whether or not all fields are actually required. If there’s no * symbol for a field, it should not be required. For example, base URL is not a required field on a REST API resource, not specifying this field allows you to dynamically calculate the entire URL in an app, or specifying it allows you to restrict where the underlying authentication can be used (that base URL only). Remember to include http:// or https:// in your URLs, or the request won’t work.
Some fields, although not required, add extra functionality if specified. E.g, while the database name is not required to run a DB query, it may be used for introspection (to generate the schema) depending on the resource/retool version.
Always check whether or not fields in your resource are actually required or not, as you may be adding unnecessary, incorrect values which are only required for certain use cases for the given resource/api. E.g If the 3rd party api offers both a self hosted and cloud solution, different fields may be required to set up the connection depending on which type of customer you are.
Debugging Sent/Token Values
Debugging Sent/Token Values
If you’re not sure whether or not your authentication tokens or other values are being properly sent to your resource, you can configure a Retool workflow or request bin to be the base URL of a REST API resource. Add your authentication to the resource, and when you trigger a query using this resource, it will pass the token to the requestbin or Retool workflow, where you can check the logs for the raw value.
If you add a return block to your workflow, you can use this to return the value back as your query response, which allows you to use it elsewhere in an app if needed.
This method is also useful to check whether dynamic values for query parameters/headers/post body are being calculated and sent properly in the request, but especially useful for authentication parameters which are calculated inside the resource, as we won’t otherwise have access to preview them inside an app.
REST APIs
REST APIs
REST APIs as a workaround
If an API with a preconfigured resource in Retool (e.g Slack/Google Sheets) is not working, or has limited functionality, you can try connecting to it using a generic REST API resource in Retool instead. Although this is more complicated to set up and manage, it provides much more flexibility, especially if there are any bugs or regressions affecting those prebuilt resources.
Prebuilt resources usually have more specialised connection options (e.g service accounts for google), so you may have to use more generic authentication methods (e.g OAuth2 ) when configuring a generic REST API resource.
Passing in parameters
API docs for some APIs are not always clear. Sometimes when referencing ‘parameters’ this may mean query parameters or body parameters, though for some methods e.g ‘GET / DELETE’ no body is required.
Remember when supplying a raw post body, to include the content type header for your payload, or the API server will likely be unable to receive your request properly and will usually return a 4XX error.
Key:Value
content-type:application/json
Incorrect Credentials
Incorrect Credentials
Although obvious, you should always double check the credentials entered are valid, as there can be some weird edge cases which I’ll touch on here:
If you’ve copied the values over from an external source, make sure there are no special characters being added. Sometimes what looks like a space, can actually be an invisible line break character, which may cause issues with your auth.
Check each field for extra spaces, before and after the text.
Check each field for new lines and to make sure that no extra characters were accidentally entered in the string.
When dealing with environment variables (self hosted customers), make sure to wrap values with quotes as needed (depending on deployment type)
Case Sensitive Values
Case Sensitive Values
Some APIs have case sensitive requirements for query parameters, headers or post body params (the parameter key names, not the key values). This is usually considered bad practice, and is definitely something to be mindful of. Requests proxied through the backend may standardise key names by making them lower case, depending on the resource and which library we are using behind the scenes to process the query data. Using Postman is a good way to find these edge cases.
Parameter order
Parameter order
Similar to the case sensitive values section, some APIs require parameters to be received in a certain order, this is considered bad practice, and the library Retool uses to make requests may change the order of query parameters at runtime.
Authentication Types
Authentication Types
It’s worth noting that not all APIs follow the same authentication flow for certain auth types. E.g they may require an extra variable or require params in a certain order.
For these cases, the authentication may not work, and you may have to try to manually create the flow that they use, by using a custom auth in Retool.
Basic
If a basic authentication is not working, it’s most likely an invalid credentials issue. If the creds are fine, perhaps the API service is expecting the token in a different way, or has a non-standard flow that it uses. Usually you can manually re-create the basic authentication by base-64 encoding your user and password with a colon in-between i.e:
user:password
This should get passed in as an authentication header i.e:
Authorization: Basic ENCODEDVALUE
You may need to tweak the above to suit your API if it has a non-standard setup.
OAuth2
Callback URL
One of the most common things that goes wrong with OAuth2 resources, is that the callback URL changes depending on whether the auth is shared amongst all users or not.
Here are some screenshots for both checkbox values. You can see the callback URL changes, which means you’ll need to update this value in your OAuth app, or the tokens will not save properly.
Scopes
Scopes are not required for the OAuth2 flow, but if your OAuth2 app/API service has scopes defined, make sure to select the correct, matching scopes in Retool or you’ll probably see 401/403 errors when trying to make requests to the resource.
Custom Auth
Custom auth is useful for non-standard auth flows, and can also be used for workarounds when it comes to saving and re-using tokens. For example, you could add a step to your custom auth flow which sends the token to a Retool Workflow to save the value to a DB, effectively allowing you to use the token elsewhere.
If you’re running into issues refreshing the token in your custom auth (non-oauth2), try setting the refresh auth flow to the same as your authentication flow, and use a time based expiration set to a value lower than the length of your access token’s life span.
Remember to test the auth workflow to make sure the JSON paths are correct.
Create a New Resource
Create a New Resource
With newer versions of Retool, come updates to the resource configuration page. So as not to break existing behaviour for existing resources, some new features will not automatically apply to existing resources. If you’re running into issues with an existing resource, consider creating a new resource from scratch (you’ll need to update references to it in your apps, which you can do fairly quickly with the ‘usage’ tab). Creating a new resource from scratch may unlock new features to fix the issue, or may fix any corrupted/underlying configuration issues with the existing resource.
AI Actions / Vectors
AI Actions / Vectors
Retool AI Actions with vectors use OpenAI embeddings to generate better answers. If AI actions have stopped working (even for non openAI keys) it’s most likely failing to get the embeddings due to an outage with OpenAI, you can check their status page here: https://status.openai.com/
Without supplying your own key, you will be subject to an org-wide rate limit, so bear this in mind in case you run into rate limiting errors.
RetoolDB and self hosted DB
RetoolDB and self hosted DB
RetoolDB acts like a normal Postgres database, if you have the connection string, you can still connect to it in retool via a Postgres resource if you need any custom functionality which is restricted in the pre-built RetoolDB resource, or if there are any issues accessing your DB via the normal RetoolDB resource. This also goes for the (self hosted) Retool Storage DB which contains all of the instance’s data. Connecting this Storage DB as a resource allows for a lot of workarounds by directly manipulating the DB values (use with caution).
Database Resource issues
Database Resource issues
Prepared statements
The most common issue with Database resources is encountered when trying to dynamically select a table name (or some other dynamic properties) in an SQL query with javascript {{ }}. To dynamically select a table, you will need to disable prepared statements in the resource itself. This is often a good troubleshooting step when SQL queries with dynamic data are not behaving as expected.
Concurrent Connection limits
For heavily-used database resources, you may see connections intermittently fail. In these cases, it’s usually the connection limit (enforced on the DB server side) being reached. This will kill new connections or cause them to time out depending on the DB setup.Token Refresh Issues
Token Refresh Issues
If your resources (especially OAuth2 resources) are working initially, but stop working after a short period of time (1-24 hours usually, depending on API) , then there is likely an issue with the refresh flow to generate a new access token. Sometimes this flow has separate scopes required ( usually called offline_access), so make sure to include those.
For OAuth2 resources, try setting the ‘Access Token Lifespan’ in the advanced settings. You can also specify a refresh auth cadence for custom auth flows.
Please note, refresh tokens can be revoked by the API, which will cause the refresh flow to fail. Re-authenticating should generate a new refresh token to fix that. Some APIs do not allow for a permanent/auto-rotating refresh token, which means users will have to remember to re-authenticate every X days/months. You can try to extend the access token duration as long as possible in the settings to reduce the tediousness of rotating these tokens.
Consider adding an Auth trigger, to re-run the auth flow either on a time-based interval, or based on an API request’s response code.
Manual Re-authentication Required
Manual Re-authentication Required
If you are using a custom auth, and are being prompted to re-authenticate your resource when you open your app, make sure your refresh flow is set up correctly. Make sure the auth trigger (example image above) is set, and set to run without prompting the user. If your refresh flow isn’t working, you can try to add a refresh flow by copying the normal auth request flow. This isn’t really feasible for standard OAuth2 as the user would have to manually re-authenticate each time (instead the refresh flow should use the refresh token to generate a new access token according to the API docs for the service)
If your OAuth2 resources are prompting you to re-authenticate on a regular basis, then there is likely an issue with the refresh token/flow. Make sure the refresh token is being saved properly after authenticating (you can check the token status in the resource), and that you have the required scopes to refresh the access token. You may need to manually create the re-authentication flow in a custom auth if it is non-standard.