CORS issue: fetch from APIs in library custom components

My goal here is two fold:

  1. emmulate the functionality seen here:
    Small business loans - SME loans - Starling Bank
    about halfway down the page there's a "What is the name of your company?" box. When you start to type in you company house, it queries an API (UK govt companies house) and returns the potential matches for what you're typing

  2. more broadly, by seeing if we can do this, I want to see if we can use retool more broadly for relatively sophisticated data intensive applications (e.g. accessing xero). Don't worry, we're not trying to do everything in the front end, but we need to make sure we can hit several APIs relatively painlessly.

So I created the React function which (after some messing about with CORS issue as discuss below) works on my local server does this perfectly. I won't paste it all here, but here's the salient bit:

try {
  const response = await fetch(
    `https://api.company-information.service.gov.uk/search/companies?q=${encodeURIComponent(
      companyName
    )}&size=7&status=active`,
    {
      headers: {
        Authorization: `Basic ${btoa(apiKey + ':')}`, // Basic auth with API key - the auth is simple and just to stop spam
      },
    }
  );
  const result = await response.json();

  if (response.ok) {
    setResults(result.items || []);
  } else {
    setError('Error fetching companies: ' + result.error);
  }
} catch (err) {
  setError('Error fetching companies: ' + err.message);
} finally {
  setLoading(false);
}

};

Now this doesn't work because of CORS issue (I can't use the javascript to access a third party API). On my local server, I have a simple way around it, which works:

Unfortunately this doesn't work for custom component library (which I can create just fine as the steps described here Build custom component libraries | Retool Docs are excellent) as clearly the upload to retool doesn't accept any mods I make to package.json...

Here's what I've tried, all of which failed:

  • Use an actual proxy: const url = https://cors-anywhere.herokuapp.com/https://api.company-information.service.gov.uk/search/companies?q=${encodeURIComponent(query)};
  • Use mode:'no-cors'
  • Create the API in retool (tested, works) and then try to access it from the component using
    const fetchCompanies = async (companyName) => {
    setLoading(true);
    const result = await window.Retool.getQuery("companySearchQuery").trigger({
    additionalScope: { companyName },
    });
    setLoading(false);
    if (result && result.data) {
    setResults(result.data.items || );
    }
    };

but it appears I cannot access window.Retool from the custom component.

I'm almost certainly wrong, there's almost certainly a way, but I cannot work it out.

Thanks in advance!

I think you could do this from the preloaded js for the app by assigning a getter function to a window property which retrieves the global window.Retool object:

window.getRetool = (() => {return window.Retool})
//or verbosely with
window.getRetool = function(){ return window.Retool }

careful if you don't use a function here as it could be possible to make a 'shallow copy' of window.Retool as a new variable instead, then when you used the var later on it would almost def be holding out-of-date or stale data.

Thanks for the reply, much appreciated. It seems I can't even access retool from the custom component. I tried this little script:

    return new Promise((resolve, reject) => {
      const checkRetool = () => {
        try {
          const Retool = window.getRetool();
          if (Retool && Retool.queries) {
            resolve(Retool);
          } else {
            throw new Error('Retool not ready');
          }
        } catch (error) {
          if (retries === 0) {
            reject('Retool environment not detected after multiple attempts.');
          } else {
            setTimeout(() => {
              checkRetool(--retries);
            }, delay);
          }
        }
      };

And it throws the error "Retool environment not detected after multiple attempts." - i.e. I can't get hold of the retool environment after multiple tries. I must be doing something wrong, but I've gone through all the steps in the custom component library, and I manage to get the components to display just fine in retool app, but getting hold of the environment inside the component is where it fails.

If you or anyone has made a custom component library that accesses the retool environment and they don't mind posting it here so I can see where I'm going wrong, I'd be immensely grateful.

if you have access to Retool RPC (I forget what the requirements are for it, if there are any) you could spin up a quick server app somewhere and call the API from the other end of the RPC call outside of Retool where you can run good ol 'npm run start' and install whatever library you normally would to accomplish your task. it's a breath of fresh air :wink:

Thanks - that's what we're going to have to do it seems... Good thing it allows all this. Thanks again!

1 Like
//import { RetoolRPC } from "retoolrpc"
const RetoolRPC = require("retoolrpc").RetoolRPC;

const rpc = new RetoolRPC({
    apiToken: process.env.RETOOL_API_TOKEN,
    host: 'https://' + process.env.RETOOL_ORG + '.retool.com',
    resourceId: process.env.RETOOL_RESOURCE_ID,
    environmentName: process.env.RETOOL_ENVIRONMENT?? 'production',
    pollingIntervalMs: 1000,
    version: '0.0.1', // optional version number for functions schemas
    logLevel: 'info', // use 'debug' for more verbose logging
  })

rpc.register({
  name: 'getCurrentTelegramData',
  arguments: {
    name: { type: 'string', description: 'Your name', required: false },
  },
  implementation: async (args, context) => {
    return {
      message: JSON.stringify(retool_current),
      context,
    }
  },
})

rpc.listen()

that worked for me. you can probly do something a bit more useful like below but :man_shrugging: maybe it'll magically work and you don't have to do anything w it, :face_with_open_eyes_and_hand_over_mouth:
code does always work the 1st time.

// rpc_init.js

const RetoolRPC = require("retoolrpc").RetoolRPC;

function initializeRPC(retool_current) {
  const rpc = new RetoolRPC({
  apiToken: process.env.RETOOL_API_TOKEN,
  host: 'https://' + process.env.RETOOL_ORG + '.retool.com',
  resourceId: process.env.RETOOL_RESOURCE_ID,
  environmentName: process.env.RETOOL_ENVIRONMENT?? 'production',
  pollingIntervalMs: 1000,
  version: '0.0.1', // optional version number for functions schemas
  logLevel: 'info', // use 'debug' for more verbose logging
})
  RetoolRPC.register({
    name: 'getCurrentTelegramData',
    arguments: {
      name: { type: 'string', description: 'Your name', required: false },
    },
    implementation: async (args, context) => {
      try {
        return {
          message: JSON.stringify(retool_current),
          context,
        };
      } catch (err) {
        console.log('RPC Error:', err);
      }
    },
  });

  RetoolRPC.listen();
}
module.exports = { initializeRPC };

@bobthebear do you experience very high latency when using RetoolRPC?

If so, is that simply acceptable for your usecase? Or do you have a workaround?