Firestore dates written as strings, not Timestamps

[justin]

Yep, agreed it's hacky - we're still trying to figure out the right way to pass these to Firebase without turning them into strings. We should definitely support this if it's coming via a JS query to - I'll talk to the team and see what we can do here.

Re: writing back - so you can write back w/ an object that has _seconds and _nanoseconds but it looks like Firebase is just making that an object. If you want it to be a timestamp field it needs to be converted to a date first.

Re: timestamp / email, you are right - we ended up reverting back to the old format (string) to maintain backwards compatibility. I know this is confusing (and we apologize ) - adding a note into our docs mentioning it today.

Sorry again about this @brettski - def not an ideal experience and we'll see what we can do to improve it.

[brettski]

It’s fine that you converted back, but please, please send an updated email that you did because everybody’s stuff was breaking!

Thank you for the help today and have a wonderful weekend.

[jeff_melon]

Will there be a fix/change to update firebase with a timestamp if you pass a timestamp from a js query to a firebase query?

For example, right now, I need to pass an object from a js query to firebase, which then updates/adds a document in firebase.

Code example:
jsQuery

jsObject: {
  items: {
    ...
    timeUpdated: new Date()
    ...
  }
  timeCreated: new Date()
}

firebaseQuery value:

{{jsQuery.data}}

So this previously worked before the changes and everything that was a timestamp updated as a timestamp in firebase. With the recent changes, I had to add a workaround that was mentioned here to convert the timestamp from the jsObject in the firebaseQuery:

firebaseQuery value updated:

{
  items: {{itemsObject.data.items}}
  timeCreated: {{new Date(itemsObject.data.timeCreated)}}
}

This workaround works for timeCreated, however, I cannot set the items object timestamps with a “new Date()” because for my application, the items object is dynamic and the size of object is unknown until the time of triggering the queries.

[justin]

Hey @jeff_melon! Yep, we’re working on fixing this this week

[mark.divitt]

Any updates on this one? I have an existing date that I don’t want changed in any way. I want FireStore to continue to see it as type Timestamp. Thanks!

[brettski]

Hello Retool Team!

Poking my head up here to see if there are solutions around this coming down the pike.

Thank you,
Brett

[Rauli]

Any news on this? This bit us hard on a week we were supposed to go live.

[ben]

The value will be stored as a Firebase timestamp if you send it a date object, here is an example using moment:

[brettski]

Hey @ben thanks for the information. Yes, I know about this workaround, the biggest issue is that is is really rare that the date is the only thing being updated, most often it is an entire object being sent to be added or updated, which gets stringified and the date is stored as a string. There is currently know way to get the date as a Date when it's part of an object.

I just thought of this (and not thought through at all), but how about an option to JSON.parse a value being used or the ability to pass it through a transformer first? I am sure there are caveats there too.

Thank you,

Brett

[ben]

Sorry for the delay!

The issue is that we stringify the items passed into the additionalScope, and that even when you JSON.parse() on the other side (in this case within the Firebase query), the value does not get converted back into a Moment object (it's just trying to parse a string). Instead, we have to use Moment within the Firebase query again, to convert the string into an object again.

So if we have our JS query:

Then within our Firebase query, we parse it back to an object:

Which then gets stored in FB as a Timestamp!

[brettski]

Plausible approach, yes, Thank You. Very manual though and error prone as you have to remember to update the firestore query when you update the javascript query. And there is no reuse of firestore queries.

If it could go through some type of transformer first it could remove the manual updates of any field, and only those of Date fields would need to be added/adjusted for. For example:

const lastUpdatedAt = data.recordUpdates.lastUpdatedAt;

return {
  ...data.recordUpdate,
  moment: moment(data.recordUpdates.lastUpdatedAt),
}

[compass]

This solution won't work for an array of dates from a transformer. eg:

I've tried using an external library to 'moment' the dates:

When i use the function in the value field of the, which appears correct in the preview box, but writes null to the DB.

[brettski]

Are you able to provide a clearer example, I cannot read what is there. Thank you.

[compass]

Thanks for looking into it, I updated the message above with more details.

[brettski]

Oh, this is a tough one. Based on what you have I would expect item to be an array of map field types which has two elements each named date, with the string value of a date. Since both maps use a key value of date, I am assuming the write fails and that is why it's null (or something like that).

You are trying to write:

item: [
  {
    date: '2022-03-10T17:35:39.165Z'
  },
  {
    date: 'e0ee-03-10T17:35:39.165Z
  }
]

This is invalid for Firestore to store as both map types have the same key value. Try adding a manual document in Firestore and you'll see.

How do you want the data to be in Firestore? We'll start with this and see how we can construct it in Retool so you can get the data written and with Firestore Timestamp types.

[compass]

Thanks for looking into it. I changed the array to:

and the function to

The preview and run works:

The DB is updated but with a null for the items.

[brettski]

Based on the json value in the preview that looks valid. We would need to see how all the pieces are put together to determine why Firestore is saving a null in this case.

[compass]

Thanks for checking. It appears Retool is converting dates into strings when passed between transformers / scripts, that should be classified as a bug right?

[brettski]

I am pretty sure it is listed already, it is the basis around this entire forum thread. There are some work-arounds listed here, though they are a bit of a pain to deal with.

[Oleksandr_Tserkovnyi]

Hi guys, still struggling with this problem on my side, tell me how dynamically prepare a field with moment? For instance I have a json where names of fields with timestamp could be different, so I iterate them and prepare json, but on my end it does not work throws this: Unexpected token 'F', "Failed to "... is not valid JSON, tell me how could I achieve that?

This is my example:

(function(){
  const initialValue = CollectionItemJSONEditorCreate.value;
  const fieldsToProcessAsTimeStamp = timestampFields.value.split(',');
  const result = Object.keys(initialValue).reduce((aggr, next) => {
    if (fieldsToProcessAsTimeStamp.includes(next)) {
      aggr[next] = moment(initialValue[next]);
    }
    return aggr;
  }, {});
  return {...initialValue, ...result};
})()
}}