Skip to content

OpenConceptLab/ocl_online#470 | Cap concept, mapping and reference lists at LIST_MAX_LIMIT rows per page - #951

Open
snyaggarwal wants to merge 3 commits into
masterfrom
ocl_online#470
Open

snyaggarwal wants to merge 3 commits into
masterfrom
ocl_online#470

Conversation

@snyaggarwal

@snyaggarwal snyaggarwal commented Oct 22, 2026 •

Copy link
Copy Markdown
Contributor

Part of OpenConceptLab/ocl_online#470. Related: OpenConceptLab/ocl_online#532 — the cap is one fixed setting, with no per-account logic in this repo.

Summary

Concept, mapping and reference lists now serve at most LIST_MAX_LIMIT rows per page (new env setting, default 100), so one page of a heavy list stays a short request. A limit over the max is capped to it instead of falling back to 25; the paging headers describe the page actually served.

What changes

  • Where: views flagged is_page_size_capped — concept and mapping lists (sources, source versions, collections, collection versions, expansions, the global lists, search), version histories, a concept's mappings / children / parents, and references. limit is capped once, in BaseAPIView.initialize, so search (ES) and database lists agree.
  • Released-version paging fix: concept and mapping lists of a released source version worked out later pages' offsets from the raw limit, so with limit=2000 page 2 came back empty. They now use the capped limit, and following next reaches every row.
  • Repo detail with includeConcepts / includeMappings / includeReferences caps limit too.
  • Compress: true on a capped list is refused above LIST_MAX_LIMIT rows (was 1,000); users.list_unpaginated stays exempt.
  • Unchanged: orgs, users, sources, collections, repo versions and other cheap lists; lookups; exports; csv=true; brief=true&checksums=true; FHIR $translate / _count. A missing limit still returns 25.

Env

  • LIST_MAX_LIMIT (optional, default 100).

Verification

  • docker exec oclapi2-api-1 python manage.py test core.integration_tests.tests_concepts core.integration_tests.tests_mappings core.integration_tests.tests_collections core.integration_tests.tests_orgs core.common … --keepdb: new tests pass. ConceptPropertyFilterViewTest fails on master as well.
  • docker exec oclapi2-api-1 pylint -j2 core/common …: 10.00/10.

Tests, added to existing files: capped pages and following next (including on a released version), search, global list, version history, limit 1–100 unchanged, the checksums exemption, Compress, references, repo detail, an uncapped orgs list, and cap_page_size.

Before go-live

Per the ticket: the Mapper's Locales fetch moves off limit=500 first, and integrators get notice (OpenConceptLab/ocl_online#473).

🤖 Generated with Claude Code

@snyaggarwal snyaggarwal changed the title OpenConceptLab/ocl_online#470 | Cap concept, mapping and reference lists at the plan max page size OpenConceptLab/ocl_online#470 | Cap concept, mapping and reference lists at LIST_MAX_LIMIT rows per page Oct 22, 2026

@snyaggarwal snyaggarwal left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found two pagination regressions that need changes before this is merged. CI currently reports passing tests and pylint; these cases are not covered by the new tests.

Comment thread core/common/views.py Outdated
Comment thread core/common/mixins.py

@snyaggarwal snyaggarwal left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rereviewed at 8689051. Both findings from my prior review are addressed: requests without limit retain the configured cap, and permitted compressed search/released-version requests avoid pre-slicing. The new tests cover both paths, and the current Tests and Pylint checks pass. I found no further blocking issue in this revision.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant