I found this old thread which asked about exactly the feature that I'm looking for:
The feature was apparently added, but it doesn't seem to be present currently:
I'm wondering whether it was added to the legacy table component and not included in the current one?
Hello @Luke_Bryant,
Great catch, it looks like you are correct. We had added it to the legacy table but we need to add this to the new table component as well!
Let me file a feature request for this right now and I will keep you updated on any news I hear 
Hi @Jack_T, we use this kind of pagination in a few apps and would love to have this added to the new table component too! Any timeline on when this might be implemented?
Hi @edbordin-linktree Thanks for reaching out. This request is in our backlog, so, unfortunately, we don't have an eta that we can share. If that changes, we'll follow up on this topic
I will let our eng team know that you checked on this request 
