Support for GraphQL relay-based pagination could be better

[cstimber]

• Goal: Create a table that does server-side pagination based on GraphQL relay cursors.

• Steps:

  • I've studied the documentation on GQL relay-based pagination: Optimize queries with server-side pagination | Retool Docs.
  • To reproduce the problem, implement a GQL relay API that doesn't calculate hasNext page when paging backwards, and try to connect it to a Retool table.

• Details:
  I've defined the inputs to the GQL query based on the Retool table's pagination field like so:
  

  

I've configured the Retool table's pagination like so:

There are a few problems here:

• I have to write some logic involving beforeCursor in order to determine what parameters to pass to the GQL query because the Retool table doesn't separate last from first. (As stated in Retool's documentation linked above but without an example or specifics on what that logic should look like.)
• Because my API doesn't calculate hasNextPage when paging backwards--as the Connection specification allows--I have to add some logic to "has next page", using beforeCursor to know that there's a next page after I paginate backwards.
• AFAICT the semantics of beforeCursor are undocumented, which means that I can't be certain that my implementation of the first two bullet points is correct.

I think that Retool could make its support for GQL relay-based pagination a lot better by simply requiring the user to populate a pageInfo field for the Retool table to read and by giving the Retool table a pagination property to pass to the GQL query. The query and the Retool table could each assume that pageInfo and pagination are Javascript objects that match the Cursor Connection specification and act accordingly.

[Darren]

Hey @cstimber - thanks for reaching out.

Before we dig into your feedback, is the current implementation meeting your functional needs or are there still issues that you're trying to iron out? I want to make sure we focus there initially just in case you're currently being blocked.

In general, relay-based pagination is not great at bi-directional navigation. It's original use case was actually infinite scroll, I believe. As such, any implementations of bi-directional pagination tend to be a bit hacky. You can get a sense for how long this conversation has been going on by checking out some of the issues on GraphQL's GitHub repository. The specs you cited also make it clear that calculating hasPreviousPage when navigating forward (and vice versa) is completely at the discretion of the server.

As such, Retool's implementation is doing what it can with the information provided by pageInfo. I think we can and probably should tweak its behavior to "remember" that hasNextPage should be true when navigating backwards the same way that we currently do for hasPreviousPage when navigating forwards. I'll talk to our team about getting that implemented and provide an update here as soon as possible.

Last but not least, we ride a fine line between "hard-coding" data integrations and allowing for flexibility when designing components and data connections. I think we learn a little towards the latter, as well, in order to make the tools we build more resistant to changing standards and specs. That said, we definitely want those same tools to also be as user-friendly as possible, so I'll take that feedback to the team as well!

[cstimber]

@Darren

is the current implementation meeting your functional needs or are there still issues that you're trying to iron out?

Hey, sorry I just saw your response. It's meeting my functional needs, but so does the simpler "Cursor based" pagination, with less fuss, and that's what I'm now using instead of relay-based pagination.