Query with limit and search filter

So I have a state, a query and a transformer, a collection list and search field (text input). I’m making a mobile app.

The data of the query is stored in a state. The transformer filters the state. And the collection list uses the transformer as the source.

I started with using the text input value in the transformer to filter the collection list. But then I noticed I’m missing entries. This is caused by the “limit” I use in the query. There are a lot of entries/records that aren’t loaded because the limit is reached. And so not all matches are returned. They were not there in the first place.

How would you change this so that the search text input does retrieve those records that were not returned. (Also important that it works with “” (empty string) because initially it doesn’t have a value and should just return everything until limit is reached.

Kind regards for your insight in advance :sweat_smile:

Hey @BobandBill! So basically, you're interested in pulling in all your data so you can then filter it? :slight_smile:

Where is your data coming from? Are you interested in pulling all data in like this?

http://community.retool.com/t/how-to-run-a-rest-api-query-for-each-item-in-an-array-and-return-all-results-together/2352

Or are you interested in just filtering what you currently have?

Basically, I want to limit the results. But also want to query the database when the search input is used and not only filter the results using a transformer. Using a transformer is snappier. But then I miss results that were not in the first 100 records (LIMIT 100). I’m also concerned that if I use the change event on the search input that every time I add a key to the input that the query runs again if I don’t use a transformer. I’ve tried debounce, but that makes it feel less snappy.

This is definitely a tricky problem to solve! Generally, with APIs that have limits and limited ability to search, users have to employ a solution like this:

http://community.retool.com/t/build-a-workflow-to-return-all-results-for-a-cursor-based-paginated-api-hubspot/17259

That way, they can kind of have the best of both worlds.

Otherwise, have you looked into the Query JSON with SQL query type? I kind of explain it in this answer here, but let me know if you have any questions at all!

http://community.retool.com/t/search-textinput-from-api-get-response/17580