Dynamically add table rows to a data table component

App Building
,
1

I have a long running job which process a bunch of rows from CSV one by one (going through a bunch of api calls).I followed the sample here and was able to build something where it processes each row of CSV and runs an operation.

The progress section updates as it runs but the result data table only updates once all of the items are processed. Is there a way to add the rows one by one as the complete ?

I know that i can use setData(oldData + newRow) while processing every row, but I was wondering if there is a more elegant way rather than repainting the whole table over and over again.

2

Hey Ashwin!

Great question! To run a query for each row in a CSV and display the data as it comes back, rather than waiting for all the data to be returned you can use a slice of temporary state set to an array and reference this array in a custom column in your table.
For this setup, starting from where you left off in this example from our docs with a couple modifications. First I would setup the slice of state. You can create a slice of state in the left panel by clicking this button:


Then in your query, you could initialize this slice of state to be an array of the same length as your total rows:

Then as you trigger each of these queries, on success you can add each value to its index by using state1.setIn() which takes two arguments, the first is an array of keys/indexes (in this case "i" or our current index) and the second is the data returned from the query

715×109 14.1 KB


Copyable Code Here:

var rows = filepicker1.parsedValue;
var errors = '';
var total = rows.length;
state1.setValue(new Array(total))

function runQuery (i) {
  // Update the statusText with the current progress
  statusText.setValue(i.toString() + '/' + total.toString())
  
  if (i >= rows.length) {
    console.log('Finished running all queries');
    return;
  }

  console.log('Running query for row', i);

  query1.trigger({
    additionalScope: { name: rows[i].name }, // This is where we override the "i" variable from step 2!
    // You can use the argument to get the data with the onSuccess function
    onSuccess: function(data) {
      state1.setIn([i], data)
      runQuery(i + 1);
    },
    onFailure: function(error) {
      // Update the errorsText with all the errors encountered
      errors += 'Found error at line ' + i.toString() + ':  ' + error +  '\n\n';
      errorText.setValue(errors);
      runQuery(i + 1);
    }
  });
}

runQuery(0);

Now that we are collecting the results of this data and saving it to this existing array, we need to reference this in our table. We can create a custom column for this and map its "Calculation" field to our array. In a columns mapper you have access to the "i" variable as well so our mapper could look like this


Now when we trigger our JS query it should display the results in our new custom column for each row as they come back from the API call:




Here's a link to a copy of this test app and a sample CSV for upload.

Hope this helps, let me know if you have any questions! :slight_smile:



3 Likes
3

Thanks @Chris-Thompson . Actually my use case was a little different, there is nothing from the input CSV that i put up on the table, so I was reading a CSV and then as and when the rows are processed i wanted them to get added to a table (lets just say this is literally an output screen). My I did try to use temporary state as described in this doc page and tried to point the data of the table to just be this state variable but was having some trouble with doing this. To be specific, when i was updating a temporary state variable in onSuccess or onError of and long running async loop, it wasn't updating the temporary state variable.

state1.setValue(state1.value.concat([<new object/row after processing>]))

This was not just the case with anything that i was using for the table but was happening to all the temporary state variable setting operation in the long running async process.

The hack that seemed to work for me was setting the data first in a local variable in script and then updating the temporary state with that local variable. I have no idea(yet) why this works and setting the temporary state directly doesn.t :frowning:

Will be happy to share more about the bug or the dashboard i created if required :slight_smile: