feat: add filesystem-path schema discovery and multi-document YAML loading

[carlos-marquez-wgu]

Description

Implements ADR 0029: Tutor integration for authorization schema discovery.

• Add OPENEDX_AUTHZ_SCHEMA_FILES Django setting for discovering schema files by absolute filesystem path (discovery.py)
• Introduce DiscoveredFileResource and SourceFileRecord with sentinel values for non-package sources
• Switch YAML parsing from safe_load to safe_load_all so each schema resource is treated as a multi-document stream (loading.py)
• Add tests for filesystem discovery, multi-document streams, and filesystem source identity
• Add tutor plugin to load Authorization schemas contributed through the openedx-authz-schema patch. Most of it is the output of cookiecutter, leaving it as is for future tutor related changes regarding openedx-authz

Testing instructions

1. Install tutor-contrib-authz plugin

pip install -e tutor/tutor-contrib-authz

2. Enable it

tutor plugins enable authz

3. Create a tutor patch to contribute to the authz schema

touch $(tutor config printroot)-plugins/my_authz_schema.py

4. Add the schema contents to my_authz_schema.py

from tutor import hooks

hooks.Filters.ENV_PATCHES.add_item(
    (
        "openedx-authz-schema",
        """
schema_version: "1.0"
priority: 100

role_extensions:
- role_id: course_editor
  add_permissions:
    - courses.export_course
  remove_permissions:
    - courses.manage_tags
  display_name: Course author
  description: Creates and exports course content for this site.
        """,
    )
)

5. Enable your patch

tutor plugins enable my_authz_schema

6. Either run do init tasks or run the load_authz_schema management command directly

tutor dev do init

7. With the management command

tutor dev run cms ./manage.py cms load_authz_schema

8. Console should output the newly applied schema:

Authz schema apply: added 1 p row(s), removed 1 p row(s).

Changes to apply:

Casbin policies (2)
  + p, role^course_editor, act^courses.export_course, course-v1^*, allow
  - p, role^course_editor, act^courses.manage_tags, course-v1^*, allow

Definitions (3)
  ~ role: course_editor
  + role-permission: course_editor -> courses.export_course @ course-v1
  - role-permission: course_editor -> courses.manage_tags @ course-v1

Authz schema applied: 1 Casbin policy row(s) added, 1 removed; definition changes: 1 role, 2 role-permission.

9. To roll back the schema changes: disable the patch and run the management command again

tutor plugins disable my_authz_schema
tutor dev run cms ./manage.py cms load_authz_schema

Merge checklist:
Check off if complete or not applicable:

• Documentation updated (not only docstrings)
• Fixup commits are squashed away
• Unit tests added/updated
• Manual testing instructions provided
• Noted any: Concerns, dependencies, migration issues, deadlines, tickets

Concerns

As per the conversation in this PR, we are taking the approach of setting module as null for file system contributions, what I noted while implementing the changes is that given the table openedx_authz_authzschemasource has this unique constraint:

UNIQUE KEY `authz_source_dist_module_uniq` (`distribution`,`module`)

And for file system contributions, the identifying source is always (<tutor-patch>, NULL), which means all file system contributions collapse into a single source row. MySQL treats NULL != NULL in unique constraints, so the constraint does not prevent duplicate (<tutor-patch>, NULL) rows. The renderer cache handles the single-row case correctly and doesn't break, but if for any reason we get two or more rows with (<tutor-patch>, NULL) (concurrent deployments or something else), renderer.py#L563 will throw a MultipleObjectsReturned error.

Additionally, the single-row constraint means per-file provenance is lost — if OPENEDX_AUTHZ_SCHEMA_FILES lists multiple files, you can't distinguish which file contributed which definition at the database level.

Traceback (most recent call last):
  File "/openedx/edx-platform/./manage.py", line 93, in <module>
    execute_from_command_line([sys.argv[0]] + django_args)
  File "/openedx/venv/lib/python3.12/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
    utility.execute()
  File "/openedx/venv/lib/python3.12/site-packages/django/core/management/__init__.py", line 436, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/openedx/venv/lib/python3.12/site-packages/django/core/management/base.py", line 420, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/openedx/venv/lib/python3.12/site-packages/django/core/management/base.py", line 464, in execute
    output = self.handle(*args, **options)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/openedx-authz/src/openedx_authz/management/commands/load_authz_schema.py", line 80, in handle
    result = pipeline.apply(force=options.get("force", False))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/openedx-authz/src/openedx_authz/engine/schema/pipeline.py", line 110, in apply
    return self._applier.apply(rendered, schema, force=force)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/openedx-authz/src/openedx_authz/engine/renderer.py", line 283, in apply
    self._store_sources(schema)
  File "/mnt/openedx-authz/src/openedx_authz/engine/renderer.py", line 632, in _store_sources
    role=obj, source=source_obj(record), defaults={"origin_kind": schema_models.OriginKind.BASE}
                     ^^^^^^^^^^^^^^^^^^
  File "/mnt/openedx-authz/src/openedx_authz/engine/renderer.py", line 563, in source_obj
    obj, _ = schema_models.AuthzSchemaSource.objects.update_or_create(
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/openedx/venv/lib/python3.12/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/openedx/venv/lib/python3.12/site-packages/django/db/models/query.py", line 990, in update_or_create
    obj, created = self.select_for_update().get_or_create(
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/openedx/venv/lib/python3.12/site-packages/django/db/models/query.py", line 948, in get_or_create
    return self.get(**kwargs), False
           ^^^^^^^^^^^^^^^^^^
  File "/openedx/venv/lib/python3.12/site-packages/django/db/models/query.py", line 638, in get
    raise self.model.MultipleObjectsReturned(
openedx_authz.models.schema.AuthzSchemaSource.MultipleObjectsReturned: get() returned more than one AuthzSchemaSource -- it returned 3!

It is probably unlikely to hit this path, but I wanted to note it as an existing risk, I can add additional code to handle that situation gracefully but wanted to bring this on before I keep going through the rabbit hole (what are we going to do if we hit that situation?)

Alternatively, if it's acceptable, I wanted to bring the other option we discussed in the linked PR comment, and that one being using the file system absolute path as the module, I understand this breaks the pattern of module = python module, but we are already breaking a convention by allowing distribution being just a sentinel value. With the sentinel value approach, this is clearer (just drop the last commit in this PR).

I went ahead and implemented it with module = null in mind, but wanted to bring this up so we can choose to take one trade off or another (more solutions are welcome, thanks!)

AI Usage

Kiro + Claude were used to assist with the creation of the tests and code implementations while thoroughly and carefully guided, everything was reviewed and corrected manually by an actual person and ensured the changes were up to standard and met the closing issue requirements.

Closes:

• Implement absolute filesystem path finding for individual files contributing to authz schema #533
• tutor: Implement the patch support to add custom schemas via tutor. #461

[openedx-webhooks]

Thanks for the pull request, @carlos-marquez-wgu!

This repository is currently maintained by @openedx/committers-openedx-authz.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

• If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
  • This process (including the steps you'll need to take) is documented here.
• If it doesn't, simply proceed with the next step.

🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

• Dependencies

  This PR must be merged before / after / at the same time as ...

• Blockers

  This PR is waiting for OEP-1234 to be accepted.

• Timeline information

  This PR must be merged by XX date because ...

• Partner information

  This is for a course on edx.org.

• Supporting documentation
• Relevant Open edX discussion forum threads

🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

Details

---

Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

• Overview of Review Process for Community Contributions
• Pull Request Status Guide
• Making changes to your pull request

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

• The size and impact of the changes that it introduces
• The need for product review
• Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

[carlos-marquez-wgu]

Hi @openedx/committers-openedx-authz, this PR is ready for reviews