Message bus replacement service

[tpoliaw]

The server and client currently communicate via both HTTP/rest and an external message bus. This adds a fair amount of complexity, especially around the submission of tasks where two requests are required separated by subscription to the message bus and the task worker has to keep track of multiple submitted but unstarted tasks.

A simpler approach could be to use a long lived HTTP connection that streams the events directly to the client. This could potentially be gRPC or websockets (or something else?).

We would then be left with a single run_plan(name, args) endpoint and could remove submit_task, delete_submitted_task, get_tasks, set_active_task and get_task.

This could also enable us to support aborting plans if the client exits (eg Ctrl-C from the terminal aborts the plan) or pausing and resuming a plan within the same process.

[DominicOram]

A simpler approach could be to use a long lived HTTP connection that streams the events directly to the client.

I assume the events will also be streamed elsewhere so that external services can monitor them and e.g. trigger processing?

[tpoliaw]

There are no plans/designs yet but I can't see why not.

I imagine events could still be broadcast on a message bus (maybe skipping the progress events?). Alternatively there could be a listen endpoint could be included that allows a direct connection to monitor events. Or some kind of webhooks could be configured so that eg start/stop events could be sent to specific services.

[DominicOram]

Ok, I would just say that if you're going to change this then it would be good to give very widespread notice. We're currently starting to use that message bus in quite a few places.

[tpoliaw]

Sure, we'll make sure people are aware of changes and have a chance to give feedback on ideas. I imagine this would become a blueapi v2 if it goes ahead.

[abbiemery]

See also #1415 and initial WIP in #1413

[fajinyuan]

Does your proposal (websocket) here continue to support multiple clients running or monitor experiment data collection process required by collaborating scientists from different locations?

[tpoliaw]

There should be no change in behaviour for who can run a plan or when as part of this change. The main motivation is to remove the need for running a plan to be a three-step process, reducing the complexity of both the client and the server. It should also allow web clients to monitor the plans they submit more easily as they can natively connect to websockets but not the message bus.