A simpler way to do this? Feedback on Workflows

My latest try at Workflows. I have multistep, but fairly straightforward JS query that I transitioned to a workflow. It get all records that have not had notification emails sent out, send them and then updates the records saying they have been sent.

var url, rv, el; let ids = []; let fails = []

let data = await qryJobSolitsToSendCount.trigger()

if (data.length === 0) return

data = await qryJobSolitsToSend.trigger()

for (let i = 0; i < data.length; i++) {
  if (data[i].tasks) {
    el = data[i]
   
    data[i].tasks.forEach(task => {
      url = {
        "action": "accept", 
        "contractor_id": task.contractor_id, 
        "job_board_id": task.job_board_id, 
        "acceptance_status_id": task.acceptance_status_id, 
        "taskId": task.task_id, 
        "publickey": el.publickey 
      }
      task.accept_url = "https://captiontracker.retool.com/embedded/public/d4d83a21-4d82-8a74-6d80532a625c#i=" + btoa(JSON.stringify(url))
      url.action = "reject"
      task.reject_url = "https://captiontracker.retool.com/embedded/public/d4d83a21-4d82-8a74-6d80532a625c#i=" + btoa(JSON.stringify(url))
      task.private_post = task.acceptance_status_id === 2
    })
    //return
    rv = await restSendOneEmail.trigger({
          additionalScope: {
          to: el.email,
          from: "brad@mathews-lowcode.com",
          template: "solitsheet",
          hbData: el
        }})
    if (rv?.id) {
      data[i].tasks.forEach(task => {
              ids.push({job_board_id: task.job_board_id, job_solit_sent: 1})
      })
    } else {
      el.error = restSendOneEmail.data.message
      fails.push(el)
    }
  }
}

if (ids.length > 0) {
  qryUpdateSolitSendStatus.trigger({
        additionalScope: {
        data: ids
      }})
}

if (fails.length > 0) {
  emailFails1.setValue(fails)
  txtSolitSendStatus.setValue(`##### There were error sending emails:`)
} else {
  txtSolitSendStatus.setHidden(true)  
}

This is what is translates to in Workflows, and I had to customize some boilerplate code as well (How to do an action based on the results of each REST call in a loop - #3 by bradlymathews):

This seems way too complex! It took me twice as long to make this workflow as it took to do the original JS query. Now much of this time is still climbing the learning curve, but still!

For very simple things Workflows is probably pretty nice, but it fails to scale in complexity pretty quickly. It also fails to take much advantage of the core Retool dev environment.

What would have been far easier would be to take the original js and sql queries and wrap a cron or webhook around them. This would look more like a module: inputs, outputs, queries, temp vars and all that, just without the UI. That would be far more powerful, just as easy to use as the rest of Retool and more familiar.

I will still look for applicable use cases going forward as I know it does have some utility, but for now I still need to use Azure functions for most of what Workflows was supposed to handle.

I guess the counter argument is you've made your monolithic code block into discrete functions that can be more easily modified independently and potentially more readable (especially when someone comes to edit that code block in 6 months time!)
Agree that more integration with the retool environment, especially shared queries and some of the transformer tools would be a big improvement.

That does open a long running debates about code architecture, function sizes and such!

Everyone's mind reasons a bit differently, so others could correctly see this the other way, but I definitely find the way I have to make this work in Workflows is more difficult to create and thus more difficult to maintain. Possibly future updates would alleviate some of that.

I would have to politely dispute my code being monolithic. It is 40 lines of code including brackets and UI interface. It calls external functions (queries) to perform discrete actions and it encapsulates a rather simple logic flow.

To be fair, right now I'd be happy if I could rename a workflow query without the UI crashing every time and making me reload the page :sweat_smile:

I didn't mean monolith in a negative sense, by the way, just playing devils advocate on why these kind of workflow tools (Zapier & Make and the like) tend to follow these design patterns. I would agree that your single block of code is likely easier to follow than all the connections that the workflow version requires you to add.

1 Like

YES!

My favorite side of the argument is Devil's advocate so keep it coming!

Oh yeah, this block type UI workflow has been used successfully (and not so successfully) since the 90's. I apologize if I cast aspersions on the concept. Thinking aloud here: I have not tried your examples but they are typically aimed at non-programmers and each block has lots of options and switches and sliders to tell them what to do and how to do it. So for a no-code platform they are great.

But Retool is very much not a no-code platform, it is low-code and thank God for that! Trying too hard to fit no-code paradigms on it may be a mistake. That is why their current modules method would be a much better fit for the "Retool Way" than the connected-block one.

Hey @bradlymathews!

Workflows are actually designed for you to be able to take a JS query and wrap a cron or webhook around it using function blocks to make calls to your resources!

Instead of calling query.trigger() you can define a function in the left panel along with any parameters you would otherwise pass to it using additionalScope:

For instance, the following trigger call:

await restSendOneEmail.trigger({
  additionalScope: {
    to: el.email,
    from: "brad@mathews-lowcode.com",
    template: "solitsheet",
    hbData: el
  }
})

would become

await restSendOneEmail(el.email, "brad@mathews-lowcode.com", "solitsheet", el)

and the rest of your JS code can exist as is!

Maintaining the flexibility that comes with the code part of Retool as a platform is a high priority for the devs and feedback like this is very much appreciated. Curious to know how you find the above works out for you and if it solves the problem the way you'd like.

As to the crashing UI @dcartlidge that's certainly a tough bug to deal with :sweat: our engineers have been actively working on a fix and we hope to have one out by the end of today, thanks for surfacing it here as well!

I did try to trigger a query from the function could not see any references to the other query blocks I had created.

I tried this and I am getting an error:

:thinking: the code you're calling along with the parameters you're passing should be declared as a function - you can then call it from ordinary blocks within the workflow:

In your case, the JS code you've written would be a regular JS block in the workflow and you'd create function blocks for each of qryJobSolitsToSend, restSendOneEmail, and qryUpdateSolitSendStatus.

Does that work?

Ahh, ok I think I see the piece I wasn't grokking. In this pattern, functions are your queries! You have to reverse your thinking.

1 Like