Query unable to fetch all data from a Firebase Collection

Hello,

So whenever I run my query without a limit, so I can fetch all the documents from the drafts collection, it gives me a timeout error.

Just after 30.8 seconds
image

Even thought I set the timeout to 120 seconds (maximum allowed)

Thanks in advance!

Hey @Dias

It looks as though this is likely an issue on our end. I've filed a bug report and can let you know here if there's a resolution. In the meantime, have you experimented with batching your requests to Firestore to see if you can pull in your full dataset with multiple faster running queries?

Hello, thank you for the support and will be awaiting resolution.

Regarding the batching of requests, I'm not sure how I would go about it in Retool. I also don't think batching in firebase is available for reads, only writes.

Awaiting response,

Dias

Ah I see, I wasn't aware of that. There are the startAt and endAt properties in a query that can be used for cursor-based pagination as described here, with Retool you could build out a JS query that loops through all the pages you're looking to fetch in one go. Does that seem like it could work in your case?

Hi @Kabirdas , I'm having the same issue... Is there a way of removing that timeout? How can we then query a Collection with more that 7000 documents? That's very important feature for us. Thanks for your support!

Hey @Modo,

At the moment there is no way to remove the timeout though there may be a workaround for your particular use case. If you'd like to explore other potential approaches would you mind describing how the 7000 document query would fit in with the rest of your app?

Hi Kabirdas,

We're fetching all the documents from the users collection, and merging it with the records from the auth, so we can show a table with all the required information from each user (uuid, email, name, etc...)

What is the workaround you propose?

Thanks!

Hi @Kabirdas , I'm trying also with the "start At" and "end At" parameters but they doesn't seem to work... Any idea? Thanks for your support!

@Modo sorry for losing track of this post!

There's more information about how you can handle firebase pagination here!