OPAL scope and data update event

[federicoragazzoni]

Describe the bug
I have configured my opal server and opal client to work with scopes.
When I publish a data update event through the server's rest APIs I get an event payload that is not correct and does not cause the client to update.
if I manually modify the payload and publish the event on the broadcast channel then the client updates.

To Reproduce
my client environment

my server environment

the data update request body posted on /scopes/myscope/data/update

the data update event payload (from broadcast channel)

the error on client
opal_client.data.rpc | INFO | Received notification of event: myscope:data:policy_data
opal_client.data.updater | INFO | Updating policy data, reason: it's my business
opal_client.data.updater | INFO | Triggering data update with id: 258732c3adad403d80f4b6bdc1ed9329
opal_client.data.updater |DEBUG | Data entry url='' config={} topics=['data:policy_data'] dst_path='/posts' save_method='PUT' data=[..] has no topics matching the data topics, skipping
websockets.legacy.protocol |DEBUG | > TEXT '{"request": null, "response": {"result": "None"...4f43824e40272f85fb69"}}' [118 bytes]

because client listens "myscope:data:policy_data". The two topics section in event payload are not equal.
If I publish a fixed payload with both topics "myscope:data:policy_data" on broadcast channel the client can update the data correctly

Do you have any suggestions?
Thanks

OPAL version

• Version: [e.g. 0.1.0]

[dadrus]

I was able to overcome the issue by applying the following modifications to the implementation of the endpoint:

@router.post("/{scope_id}/data/update")
async def publish_data_update_event(
    update: DataUpdate,
    claims: JWTClaims = Depends(authenticator),
    scope_id: str = Path(..., description="Scope ID"),
):
    try:
        require_peer_type(authenticator, claims, PeerType.datasource)

        restrict_optional_topics_to_publish(authenticator, claims, update)

        for entry in update.entries:
            # {scope_id}: is added
            entry.topics = [f"{scope_id}:data:{topic}" for topic in entry.topics]

        await DataUpdatePublisher(
            # ServerSideTopicPublisher is used instead of the ScopedServerSideTopicPublisher
            ServerSideTopicPublisher(pubsub_endpoint)
        ).publish_data_updates(update)
    except Unauthorized as ex:
        logger.error(f"Unauthorized to publish update: {repr(ex)}")
        raise

Here is also an examples DataUpdate payload:

{
  "id": "58bb77af-22a4-4327-be83-d44197f2c724",
  "entries": [
    {
      "url": "",
      "config": {},
      "topics": [
        "data"
      ],
      "dst_path": "documents",
      "save_method": "PATCH",
      "data": [
        {
          "op": "add",
          "path": "0cde79f8-7903-4ec0-931e-b7adf9fe3e4c",
          "value": {
            "parent_id": "ip:1"
          }
        }
      ]
    }
  ],
  "reason": "New document created",
  "callback": {
    "callbacks": []
  }
}

[vorbidan]

You can also specify topics when you configure scope:

curl --request PUT 'http://opal-server:7002/scopes' \
--header 'Content-Type: application/json' \
--data-raw '{
  "scope_id": "my-app",
  "policy": {...},
   "data": {
    "entries": [
      {
        "url": ...,
        "topics": ["my-app:data:policy_data"]
      }]}
     ...
    }

[vorbidan]

You can also specify topics when you configure scope:

curl --request PUT 'http://opal-server:7002/scopes' \
--header 'Content-Type: application/json' \
--data-raw '{
  "scope_id": "my-app",
  "policy": {...},
   "data": {
    "entries": [
      {
        "url": ...,
        "topics": ["my-app:data:policy_data"]
      }]}
     ...
    }

This only solves the initial data fetch by the scoped client, since it simply retrieves the data from the scope on the server. Further updates via /scopes/{scope_id}/data/update will result in topics validation failure by the client.