Skip to content

Fix sync _get_pages_iterator silently dropping the last page#326

Open
jgarber-cisco wants to merge 1 commit intomeraki:mainfrom
jgarber-cisco:fix-iterator-pagination-last-page
Open

Fix sync _get_pages_iterator silently dropping the last page#326
jgarber-cisco wants to merge 1 commit intomeraki:mainfrom
jgarber-cisco:fix-iterator-pagination-last-page

Conversation

@jgarber-cisco
Copy link
Copy Markdown

@jgarber-cisco jgarber-cisco commented Apr 25, 2026

Bug

The synchronous _get_pages_iterator in RestSession silently drops the last page of paginated results. When iterating through all pages with total_pages="all" (or -1), every page except the final one is yielded correctly. The final page is fetched and parsed, but its items are never yielded to the caller.

Root cause

The while loop in _get_pages_iterator checks for a next (or prev) link in the response headers. When the last page has no such link, the else branch executed break, which exited the loop before reaching the yield block below it:

# BEFORE (bug)
if direction == "next" and "next" in links:
    ...
elif direction == "prev" and "prev" in links:
    ...
else:
    break          # exits loop here

response.close()
# ... extract items ...
for item in return_items:
    yield item     # never reached for the last page

Impact

Any caller using the iterator pagination mode (use_iterator_for_get_pages=True) and paginating through all pages would silently receive fewer results than expected — missing the entire last page. For example, 100 items at 10 per page would yield only 90 items.

The legacy pagination mode (_get_pages_legacy) is not affected — it uses a different loop structure that correctly accumulates all pages.

Fix

Replace break with total_pages = 1. This allows the loop body to continue — closing the response, extracting items, and yielding them — then total_pages decrements to 0, and the while total_pages != 0 condition exits the loop naturally. The subsequent if total_pages != 0 guard prevents any further request from being made, so the stale nextlink variable is never used.

# AFTER (fix)
else:
    total_pages = 1    # yield this page, then exit

This is a one-line change that makes the sync iterator match the async iterator (AsyncRestSession._get_pages_iterator), which already handles this correctly with the same total_pages = 1 pattern at aio/rest_session.py line 375.

Test plan

  • Verified via flake8 (matching CI config) — no new lint errors introduced.
  • Confirmed the async iterator already uses total_pages = 1 and is unaffected.
  • Previously validated against a live paginated endpoint (login-attempts outpost) where the legacy mode returned all items correctly while the iterator consistently dropped the last page — the root cause traced to this exact break.
  • Did not run tests/test_dashboard_api_python_library.py because it requires production API keys.

Author: Jason Garber (@jgarber-cisco)

Made with Cursor

The iterator broke out of the while loop when no next/prev link was
present, skipping the yield of the final page's items. Replace the
break with total_pages = 1 so the last page is yielded before the
loop exits, matching the async iterator's existing behavior.

Made-with: Cursor
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant