Handling Pagination with the MS Graph API in Logic Apps and Power Automate

This is a quick article to describe how to do pagination in Logic Apps and Power Automate when you use the HTTP connector to get data from the Microsoft Graph API, or indeed any other OData compliant API.

Despite the Graph API being OData, and therefore, I guess, adhering to a standard that would allow pagination to be easily included in the generic HTTP action, because it’s exactly the same for any OData compliant API, it seems you need to roll your own pagination.

When you run a call to the Graph API that has more results than your $top count is limited to (max 999) then you get a property in the body of the request called @odata.nextLink

If the number of results is less than or equal to your top count then that property simply doesn’t exist.

There are often many ways to achieve an outcome, and here’s how I’ve done it.

In my example I’m getting users from Azure AD.

Firstly, I initialise a few variables: A boolean that used in the looping logic, an array variable to store the items retrieved from the API, and a string variable for the next link.

Get the users in an HTTP request. The default top count is 100. I choose not to change that, but you can cut down on the time this processes by making it as high as it goes: 999. Just add $top as a query and set its value to your needs.

Now add all the items to the array variable you initialised earlier. The input for the For each is body(‘Get_Users’)?[‘value’]

I don’t bother with Parse JSON actions because it’s usually easier to manually select the properties in the expression editor than stuff around with schema definitions (in my opinion). Although the input in the screen shot below, value, looks like dynamic content it was actually typed into the expression editor. When you save and come back later, the Logic Apps designer sees it as a property of the Get users action and makes it look like that.

Now we see if there’s a next page by very crudely looking for the string ‘odata.nextLink’ in the entire response from the HTTP GET. Hopefully nobody creates a user account with that exact string in any of the attributes we’ve picked out with the $select query!

In the If false/No branch, do nothing. In the if true/Yes branch, decode the @odata.nextLink property in a Set variable action. The variable being the string that was initialised earlier. The expression is decodeUriComponent(body(‘Get_Users’)?[‘@odata.nextLink’])

Now add a Do until, or Until loop with the boolean variable is equal to true. Inside, that the first action is another HTTP GET with the same OAuth parameters but the URI is the string variable (the URI decoded @odata.nextLink property). All the original queries are included in the nextLink so no need to include them again:

Another loop to append the results to the array variable:

This is pointless: See the comments at the end.

Now another condition. We’re still within the Do Until here. The expression within the Set variable action (in the If true branch) called update nextLink is decodeUriComponent(body(‘Get_Users_2’)?[‘@odata.nextLink’])

Lastly you can put the array in a compose to see all the results from the first GET and any subsequent loops combined.

That’s it folks. Good luck. If you like my work, please feel free to buy me a coffee

6 Comments

  1. The built in pagination mentioned by CG does not work. I tried it and perhaps it works for returning sql queries but not for returning vm’s with the Azure API.

    I have to say kudos this is a great post and I like your way of coding it’s very similar to mine. Not many people would be able to figure this one out, I appreciate it!

    Liked by 1 person

    1. Thanks for the feedback Brian. If there’s anything I’d do differently now, instead of adding all the items into the array variable in a For each/Apply to each, I’d use a compose with union(variables(‘array_users’),body(‘Get_users_2’)?[‘value’]) to join the latest page of results to the existing contents of the variable, then in the next action, do Set variable and set array_users to the output of that compose. The result is the same but it’s much quicker.

      Liked by 1 person

  2. Hello,

    I tried above way, but i got the error notification fromsecond page onwards “The specified page token value has expired and can no longer be included in your request” on GetUsers2 action.
    Not sure what am doing wrong.

    Any help would be much approciated

    Saan.

    Like

  3. I’ve checked and the automatic pagination under HTTP Action Settings works great, but thanks for the workaround!

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s