[FIXED] How to page through results using Shopify Python API wrapper

Issue

I want to page through the results from the Shopify API using the Python wrapper. The API recently (2019-07) switched to "cursor-based pagination", so I cannot just pass a "page" query parameter to get the next set of results.

The Shopify API docs have a page dedicated to cursor-based pagination.

The API response supposedly includes a link in the response headers that includes info for making another request, but I cannot figure out how to access it. As far as I can tell, the response from the wrapper is a standard Python list that has no headers.

I think I could make this work without using the python API wrapper, but there must be an easy way to get the next set of results.

import shopify

shopify.ShopifyResource.set_site("https://example-store.myshopify.com/admin/api/2019-07")
shopify.ShopifyResource.set_user(API_KEY)
shopify.ShopifyResource.set_password(PASSWORD)

products = shopify.Product.find(limit=5)

# This works fine
for product in products:
    print(product.title)

# None of these work for accessing the headers referenced in the docs
print(products.headers)
print(products.link)
print(products['headers'])
print(products['link'])

# This throws an error saying that "page" is not an acceptable parameter
products = shopify.Product.find(limit=5, page=2)

Can anyone provide an example of how to get the next page of results using the wrapper?

Solution

As mentioned by @babis21, this was a bug in the shopify python api wrapper. The library was updated in January of 2020 to fix it.

For anyone stumbling upon this, here is an easy way to page through all results. This same format also works for other API objects like Products as well.

orders = shopify.Order.find(since_id=0, status='any', limit=250)
for order in orders:
    # Do something with the order
while orders.has_next_page():
    orders = orders.next_page()
    for order in orders:
        # Do something with the remaining orders

Using since_id=0 will fetch ALL orders because order IDs are guaranteed to be greater than 0.

If you don't want to repeat the code that processes the order objects, you can wrap it all in an iterator like this:

def iter_all_orders(status='any', limit=250):
    orders = shopify.Order.find(since_id=0, status=status, limit=limit)
    for order in orders:
        yield order
    
    while orders.has_next_page():
        orders = orders.next_page()
        for order in orders:
            yield order

for order in iter_all_orders():
    # Do something with each order

If you are fetching a large number of orders or other objects (for offline analysis like I was), you will find that this is slow compared to your other options. The GraphQL API is faster than the REST API, but performing bulk operations with the GraphQL API was by far the most efficient.

Answered By - rseabrook

Answer Checked By - Willingham (PHPFixing Volunteer)