Value formatting in keyvaluemap components

[markbenepass]

Hi - I've written a transformer using JSON.stringify to nicely format a JSON object. But when it's passed to a Key Value Map component that formatting gets thrown away.

Would be really nice if that formatting was preserved, so complicated objects could be readable. Thanks!

Little code sample to show what I mean:

var benefits = {{get_user.data.benefits}};

Object.keys(benefits).forEach(function(key, index) {
benefits[key] = JSON.stringify(benefits[key], null, 2);
});
return benefits

[justin]

Hey @markbenepass, and welcome to the community! Just to make sure I understand correctly - you’re using a JS transformer to apply this function to your query’s data? If so, you should be able to reference that transformed data in the key value map component (https://docs.retool.com/docs/transformers). Is it the JSON.stringify formatting you want to preserve, or the null replacing? Or both?

[markbenepass]

Hi Justin, thanks for the reply. Here’s the use-case, will try and explain it well:

I have a query that I display in a table, and one column contains Key/Value Maps that I frequently need to explore. On the backend it’s python dicts that turn into JSONFields in the DB. I have a transformer that I’m using to pull out the column that from the query, and do some mild modifications to the data. The data structure is actually a nested dictionary, something like:
{“primary_key”: {
“key1” : “value1”,
“key2”: “value2”,
},}

I want to use the Key Value Map Component in Retool to nicely display that data (column -> transformer), but because of the nested dicts the “value” part of the Key Value Map is still really hard to read. When I use JSON.stringify in the transfomer it does a nice job of formatting (indents, replacer function), but that formatting isn’t passed along to the Key Value Component. So the value portion is just one big string dump, instead of maintaining indents for readability. Does that make sense?

Thanks for the help!

[justin]

Hey @markbenepass, sorry for the delay here, the team has been enjoying a few days off

Yes, your use case totally makes sense. Two approaches that might help here:

1. Create a transformer as query

There are two ways to create transformers in Retool: attached to a specific query (e.g. in that query's window) or as a separate query in of itself (click on the "transformers" tab in the query editor). If you choose the latter method, you can reference query in the transformer (e.g. {{ query.data }}) the transformer's returned data directly via {{ transformer.data }} - that may preserve the formatting.

2. Do the prettifying in the key value reference

You can skip the transformer step and just use your Object.keys loop in the "Data" field for your key value component. Forgive me if I mess up syntax, but it should look something like this:

{{
    Object.keys(get_user.data.benefits).forEach(key => 
        JSON.stringify(key, null, 2);
    )
}}

Let me know if either of these work!

[markbenepass]

Hi Justin,

#1 Doesn’t preserve formatting. That’s the first thing I tried - creating a transformer that returns with the formatting I want, and the Key Value Map throws it away.

#2 Can you clarify what’s allowed in inline JS in retool? Can I set and return variables? Your example is complicated in this case because forEach returns undefined, and what I need to do is return the object with each property modified by JSON.stringify

Edit: adding a question here - does Retool use the inline {{}} syntax as an implicit return? Is that the right way to think about it?

Thanks

[justin]

Gotcha. For #2 - yes, pretty much it's an implicit return, so you can't really set variables in there - which means the JS in {{ }} can get pretty hairy sometimes That's why for longer JS we recommend using a Run JS Code query, where you can work w/ variables / return statements. I'm not quite sure I understand the issue you're having with inline JS?

[markbenepass]

Hi Justin, thanks again for the help. This isn't a super pressing issue (just nice for usability), but I've spent too much time now so really want to get a working solution

Because of the implicit return it's challenging to get the output that I need. forEach returns undefined. Map returns an array. Do you happen to know some javascript I can use within {{}} to iterate through the properties of an object and modify them?

[justin]

Usability is important!!

Based on the data structure that you shared above, I think the forEach syntax should work for this. Something like:

{{
    Object.keys(get_user.data.benefits).forEach(key => 
        //Do something to each nested object
        //e.g. get_user.data.benefits.primary_key[key] 
    )
}}

I can't give you a specific solution without understanding what you're trying to do with your data here - if this doesn't help, I would recommend reaching out via Intercom to get chat support

[markbenepass]

Hi Justin,

Ok can you please do me one favor - can you explain how something like forEach (which returns undefined) works with the implicit return you mentioned before? I haven’t been able to get that to work in any of the many many permutations I’ve tried.

All I’ve seen is hundreds of examples of:
Error in template ‘template here’: An unexpected token has been found. Please make sure your JS syntax is correct.

As a separate issue, that message isn’t very helpful

Thanks again,
MF

[justin]

Yea for sure! .forEach iterates through an array, lets you apply a function to each element, and then returns the mutated array. So if I had an array of dollar values like [$45, $56, $67] and I wanted to remove the dollar signs, I could use:

[$45, $56, $67].forEach(element => element.replace("$",""))
# returns: [45,56,67]

When you’re working with an object, you’ll need to get it into an array format one way or another: you can do that by extracting keys (Object.keys), values (Object.values), or any other arbitrary field (some_object.map(thing => thing.some_field)).

In terms of your error: can you send over a screenshot?

[markbenepass]

Really appreciate your help.

Here is a transformer that does exactly what I want, need to replicate it in the Key Value Map.

var benefits = {{get_claim_user.data.benefits}};
Object.keys(benefits).forEach(function(key, index) {
	benefits[key] = JSON.stringify(benefits[key], null, 2);
})
return benefits

And an example of the issue I've with returning undefined is attached

[markbenepass]

Update, got this to work (meaning do the transformation I need in-line in a Key Value Map) ... BUT...

The Key Value Map still throws away the formatting!
(notice the \n's that aren't respected in the Key Value component)

Screenshot that shows the full thing below:

[markbenepass]

And another screenshot to show what the transformer outputs (the expected result, what I need):

[justin]

@markbenepass thanks for the screenshots. At this point I think the is just that the key value map is overriding whatever formatting you’ve applied via the transformer (and now we can safely say - also via the inline JS for the key value map). One last question for you - what happens if you put this data into a table instead of a key value map?

In terms of key value map formatting, I’ll file a ticket and share with the team!

[markbenepass]

Same thing on tables, screenshot attached

Yes please file a bug and let me know when those components will reflect formatting, especially newlines and tabs / indents. Thanks for the help Justin!

[justin]

You got it @markbenepass, sorry for all the back and forth and thanks for bearing with me

[markbenepass]

Hey Justin - wanted to check in on this. Please let me know if it gets added to the roadmap, thanks

[markbenepass]

Hey Justin - checking in again

[justin]

Hey @markbenepass, sorry for the delay here! Unfortunately we haven't gotten to fixing this yet I've got you down in the ticket to be notified when we do!

[alex-w]

Hey @markbenepass

I'm sorry to say I don't think this will be a bug we will be able to prioritize soon, but I do also come bearing good news. Both the table and key value map components have the option to "Render as HTML" which means we can use a .replace() function to convert this to html like so:

here is the regex you can add:

.replace(/(?:\r\n|\r|\n)/g, '<br>')