feat: Support plan *args and **kwargs by oliwenmandiamond · Pull Request #1662 · DiamondLightSource/blueapi

oliwenmandiamond · 2026-09-09T15:20:55Z

API changes
Plan parameters: Added support for passing both positional (*args) and keyword (**kwargs) arguments to plans via the API. This is using TaskParametersV2 and TaskRequestV2

Backwards compatibility: Existing requests that pass plan parameters as a simple JSON object of keyword arguments remain supported, TaskParametersV1 and TaskRequestV1. They are automatically converted to the new V2 before plans are ran.

…eapi into support_star_args

codecov · 2026-10-14T11:03:38Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.38%. Comparing base (f9af725) to head (5ff57de).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1662      +/-   ##
==========================================
+ Coverage   96.36%   96.38%   +0.02%     
==========================================
  Files          49       49              
  Lines        3686     3739      +53     
==========================================
+ Hits         3552     3604      +52     
- Misses        134      135       +1

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

oliwenmandiamond · 2026-10-14T16:16:44Z

@DiamondLightSource/developers-daq-core this is ready for review. It should be fully backwards compatible so shouldn't break anything downstream. However, we should depreciate the V1 Task and Parameters and then remove after X amount of time.

fajinyuan

Please see my comments or questions - I only partly went through the changes now. but would like to submit them for today, and continue the review next time - hopefully tomorrow!

abbiemery

Given the large size of the change I don't have time to look at this today, but I shall next week. I would prefer it isn't merged until I have had that chance.

…iven malformed arguments for plan

oliwenmandiamond · 2026-10-16T13:57:10Z

Given the large size of the change I don't have time to look at this today, but I shall next week. I would prefer it isn't merged until I have had that chance.

Thanks, fine by me. The change itself is not that large about 300 lines, but I've gone out of my way to make the unit tests and system tests to be as comprehensive as possible due to the API change as we now have different combinations to test as supporting old parameters and new and unfortunately this has made the PR grow larger than I wanted.

dan-fernandes · 2026-10-16T12:46:56Z

    request: Request,
    response: Response,
-    task_request: Annotated[TaskRequest, Body(..., examples=[example_task_request])],
+    task_request: Annotated[TaskRequestV2, Body(..., examples=[example_task_request])],


Am I right in thinking that this will only accept request bodies that desearlize to a TaskRequestV2? Eg. the request below yields an "Extra inputs are not permitted" error:

curl -X 'POST' \ 'http://localhost:4180/api/v1/tasks' \ -H 'accept: */*' \ -H 'Content-Type: application/json' \ -d '{ "name": "sleep", "params": { "time": 1.0 }, "instrument_session": "cm12345-1" }'

If this is the case (and not my user error) I think it breaks backwards compatibility, so will need to either be reserved for Blueapi 2.0, or re-worked.

yes good spot. This is easily fixed by using TaskRequest = TaskRequestV2 | TaskRequestV1 for the annotation. Fixed now.

Again, I don't think there are existing tests covering this otherwise the CI would of been unhappy?

dan-fernandes · 2026-10-16T13:33:43Z

+        func = ctx.plan_functions[self.name]
+
+        sig = signature(func)
+        bound = sig.bind(*self.params.args, **self.params.kwargs)


Running a plan without the correct parameters (blueapi -c tests/system_tests/config.yaml controller run sleep -i cm12345-1) yields a stack trace (for both server and client) rather than a neat error like below:

$ blueapi -c tests/system_tests/config.yaml controller run sleep -i cm12345-1 Authorization required, but no authorization protocol specified Error: Incorrect parameters supplied Missing value for 'time'

I think this is because sig.bind raises a TypeError rather than the ValidationError which submit_task is expecting.

hmmm, I tried adding this

try: bound = sig.bind(*self.params.args, **self.params.kwargs) except TypeError as e: raise ValidationError() from e

But that didn't fix it. Will have to investigate this more next week We should probably add a new test for this too because the existing would of caught this otherwise

Okay managed to fix it. I switched to use sig.bind_partial so that it doesn't raise error and error is instead raised by pydantic instead, thus restoring previous behaviour. Added test_task-worker.py:test_do_task_with_invalid_parameters_raises_validation_error that tests for this.

dan-fernandes · 2026-10-16T13:37:30Z



-SUBMIT_REQUEST = {
+SUBMIT_REQUEST_V1 = {


If we're continuing to support requests that only contains Keyword arguments, I think the API needs to have TaskParameterV1 tests to promote backwards compatability.

I think this is outdated now, added TaskParametersV1 to test here as "params": {"one": "two"},

dan-fernandes · 2026-10-16T13:52:34Z

@@ -253,6 +255,8 @@ def get_active_task(self) -> WorkerTask:
        return self._request_and_deserialize("/worker/task", WorkerTask)

    def create_task(self, task: TaskRequest) -> TaskResponse:


Not sure if this is used by externally, but Within Blueapi .create_task is only called with a TaskRequestV2. Could this signature be changed to expect TaskRequest2 and not bother with conversion?

Changing this to support V2 only breaks tests

test_rest.py:test_create_task_serialization

fails

AssertionError: expected call not found. Expected: _request_and_deserialize('/tasks', <class 'blueapi.service.model.TaskResponse'>, method='POST', get_exception=<function _create_task_exceptions at 0x7fe3cd8726c0>, data={'name': 'demo', 'instrument_session': 'cm12345-1', 'params': {'args': [], 'kwargs': {'devices': ['foo']}}}) Actual: _request_and_deserialize('/tasks', <class 'blueapi.service.model.TaskResponse'>, method='POST', get_exception=<function _create_task_exceptions at 0x7fe3cd8726c0>, data={'name': 'demo', 'params': {'devices': ['foo']}, 'instrument_session': 'cm12345-1'})

However, this might be okay because it's because I make it test for V1 and V2. So if you think this should be V2 only, we can make that change

dan-fernandes · 2026-10-16T14:07:19Z

    )


+class TaskRequestV2(BlueapiBaseModel):


With your changes, the REST API only accepts TaskRequestV2 (I think?) and the python client automatically converts to TaskRequestV2. As such is there a need for two different internal representations of a task?

This may need to be re-visited if the REST API changes are re-worked. But even if the API accepts both, if possible it would be handy to convert at the top level, rather than have an ambiguous TaskRequest with multiple possible fields throughout the service.

Python client only ever needs to submit things using V2 as you trying to use python so need access *args and **kwargs for plans. I've updated REST API so it should accept V1 and V2 now.

So there's two ways I could of done this. There was TaskRequest which accepts nested structure of kwargs and args, or I could of added an additional parameter at top level. I think the later was much more messy option so updating the model and being nested was cleaner and lesser of two evils in my opinion. I think that has answered your question? (let me if it hasn't...)

dan-fernandes · 2026-10-16T14:10:43Z


    #: API version to publish in OpenAPI schema
-    REST_API_VERSION: ClassVar[str] = "1.5.0"
+    REST_API_VERSION: ClassVar[str] = "1.6.0"


Need to double check that the API isn't broken before confirming this API version

yeah, I updated assuming everything was okay and to get all tests to pass. If we agree it should 2.0.0 that's fine, just let me know

oliwenmandiamond added 6 commits September 9, 2026 15:01

Support *args in plan args

ff91bf5

Merge branch 'main' into support_star_args

b130973

Add tests to cover *args case

fc09b96

Simplify logic

d638884

Fix test

d3d45e1

Use default_factory for args

44ce2fa

oliwenmandiamond changed the title ~~Support *args~~ Support plan *args Sep 9, 2026

oliwenmandiamond added 4 commits September 9, 2026 16:24

Propergate change

32cc60e

Fix lint

42bad03

fix SUBMIT_REQUEST

29b05e4

Correct doc string

adfa659

oliwenmandiamond changed the title ~~Support plan *args~~ feat: Support plan *args Sep 9, 2026

oliwenmandiamond mentioned this pull request Sep 11, 2026

Refactor wrapped scan plans so they more user friendly DiamondLightSource/dodal#2050

Open

4 tasks

oliwenmandiamond and others added 9 commits September 21, 2026 13:43

Merge branch 'main' into support_star_args

5983d91

Add backwards compatability

e26f0b0

Update task parameter test

23e09b6

Merge branch 'main' into support_star_args

9b11a11

Clean up

461272d

Merge branch 'support_star_args' of github.com:DiamondLightSource/blu…

9d13e6e

…eapi into support_star_args

Merge branch 'main' into support_star_args

8d638c4

Fix typing

f882f08

Update openapi

a3e46ef

oliwenmandiamond added 3 commits October 14, 2026 13:23

Fix lint

88a1b48

Add rest tak request v1 and v2

bcbd630

Fix test

2e9d6f1

shree-iyengar-dls mentioned this pull request Oct 14, 2026

refactor: Include Bluesky plan stubs by default #1694

Draft

oliwenmandiamond added 2 commits October 14, 2026 14:26

update all rest tests to use v1 and v2

6484e38

Add additional tests

b430aa9

oliwenmandiamond marked this pull request as ready for review October 14, 2026 16:04

oliwenmandiamond requested a review from a team as a code owner October 14, 2026 16:05

Minor corrections

4fffc33

oliwenmandiamond added 2 commits October 15, 2026 08:06

Update _build_args doc string

9f214f1

Improve _type_spec_for_function logic

8f5fa12

oliwenmandiamond changed the title ~~feat: Support plan *args~~ feat: Support plan *args and **kwargs Oct 15, 2026

fajinyuan reviewed Oct 15, 2026

View reviewed changes

Comment thread docs/reference/openapi.yaml

Comment thread src/blueapi/cli/cli.py Outdated

Comment thread src/blueapi/cli/cli.py

Comment thread src/blueapi/cli/cli.py

oliwenmandiamond added 3 commits October 16, 2026 10:51

Improve test to use args and kwargs

970f95c

Update tests for protocol and rest_api to be stronger, cover v1 and v2

05b4cfd

Add more combinations for system tests

38fb396

abbiemery requested changes Oct 16, 2026

View reviewed changes

Fix system tests, discovered blueapi bug where it freezes when plan g…

38c4835

…iven malformed arguments for plan

dan-fernandes reviewed Oct 16, 2026

View reviewed changes

oliwenmandiamond added 5 commits October 16, 2026 15:28

Fix annotation for main submit_task

48f0703

Update openapi.yaml

ab1a9a4

Implement import feedback.

7e368aa

Add feedback, correctly raise ValidationError in do_task and add test

e517e73

Add comment why test is needed

5ff57de

		@@ -253,6 +255,8 @@ def get_active_task(self) -> WorkerTask:
		return self._request_and_deserialize("/worker/task", WorkerTask)

		def create_task(self, task: TaskRequest) -> TaskResponse:



		SUBMIT_REQUEST = {
		SUBMIT_REQUEST_V1 = {

		)


		class TaskRequestV2(BlueapiBaseModel):

Conversation

oliwenmandiamond commented Sep 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov Bot commented Oct 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

oliwenmandiamond commented Oct 14, 2026

Uh oh!

fajinyuan left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

abbiemery left a comment

Choose a reason for hiding this comment

Uh oh!

oliwenmandiamond commented Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dan-fernandes Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dan-fernandes Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dan-fernandes Oct 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

oliwenmandiamond commented Sep 9, 2026 •

edited

Loading

codecov Bot commented Oct 14, 2026 •

edited

Loading

oliwenmandiamond commented Oct 16, 2026 •

edited

Loading

dan-fernandes Oct 16, 2026 •

edited

Loading

dan-fernandes Oct 16, 2026 •

edited

Loading

dan-fernandes Oct 16, 2026 •

edited

Loading