Workflow vs lambda function - installing libraries and other limitations

I am looking to create a script that exports data from a postgreSQl data base into google sheets and then formats that data so that I can send clients a PDF or editable google sheet / excel file. I wanted to use the node.js wrapper googleapis to make this process a bit easier rather than having to rely on a number of REST API calls. It seems that it is not possible to import npm packages to a workflow. I'm also worried about potential timeouts or other limitations. Would it be better to create lambda functions for the various export scripts with layers for the googleapis module or is there an easy way to do this with workflows? Its a bit of a pain to set up lambda functions, but once its set up I don't have a lot of limitations. My concern is that I'll hit a wall with the workflows.

I had the same question for the medusa client library. Medusa JS Client | Medusa.

Hello @benjaminfortunato!

Very interesting use case. You are correct that JS libraries cannot be imported into workflows.

But I believe that you should be able to accomplish most of the actions you need and utilize the JS libraries in a Retool App. Then once the data payloads are set properly, use workflows to loop through and dispatch the data.

The only functionality you might need a more creative solution for would be running this via a script. You could definitely trigger the events manually in a Retool app, but currently you would not be able to use a workflow to trigger the app to run :melting_face:

In the app you can:

  1. Query Postgres for table data
  2. Use JS libraries to transform the data
  3. Gsheet -> format->PDF-> gSheet/Excel
  4. Send the finalized data payload and the emails to send them over to Workflow (or loop through in app)

I found this doc about limitations on Workflows also for future reference if you are curious.

I would test out the libraries you want to use first to make sure there are no hiccups first!

It sounds like you can’t creat a rest api for a workflow so if I wanted to create a workflow that is accessible from anouther client this would not be possible.

It sounds like using a lambda function would be the best bet and then just calling that function from retool. That way I don’t have the limit of not being able to install libraries and I also have the ability to utilize this function from multiple clients

Hey @benjaminfortunato I actually justt confirmed that you can add JS libraries to Workflows!

Apologies for the confusion. using lambda functions is also not a bad option for greater levels of control.

If you go into the panel on the left side of workflows you can add in libraries that will load in when the runtime environment is created.

GoogleAPIs are fairly standard and should definitely work, for Medusa JS I am not super familiar with the library so I would recommend testing it out before building things out.

As you said using AWS Lambdas give even more control over setup.