Describe the bug
A data update sent to a scope via POST /scopes/{scope_id}/data/update never reaches that scope's OPAL clients; every entry is dropped, whatever topic you send.
The endpoint rewrites each entry's topics to data:{topic} (scopes/api.py#L350) and publishes through ScopedServerSideTopicPublisher, which scope-prefixes only the channel (publisher.py#L206), not the payload. So the message is delivered on channel {scope_id}:data:{topic} (the client is subscribed there), but each entry's own topics stay data:{topic}. The scoped client filters entries with set(entry.topics).isdisjoint(self._data_topics) against its namespaced {scope_id}:data:{topic} (updater.py#L482), so the entry is skipped before any fetch. No topic value works: the server forces data:{topic} and the client only matches {scope_id}:data:{topic}. The global POST /data/config is not an alternative for scoped clients, since it publishes un-prefixed channels they are not subscribed to.
To Reproduce
docker/docker-compose-scopes-example.yml, with the client pointed at scope documents:
opal_client:
environment:
- OPAL_SCOPE_ID=documents
- OPAL_DATA_TOPICS=data # subscribes to documents:data:data
- OPAL_LOG_LEVEL=DEBUG
$ curl -s -X POST http://localhost:7002/scopes/documents/data/update \
-H 'Content-Type: application/json' \
--data-binary '{"entries":[{"url":"http://opal_server:7002/healthcheck","topics":["data"],"dst_path":"/scope_probe","save_method":"PUT"}],"reason":"repro"}'
$ curl -s http://localhost:8181/v1/data/scope_probe
{}
The channel matches (so it is delivered), the entry topics do not (so it is dropped):
opal_server | data_update_publisher | Publishing data update to topics: {'data:data'}, entries: [{'path': '/scope_probe', 'topics': ['data:data']}]
opal_server | opal_common.topics.publisher | Publishing to topics: ['documents:data:data']
opal_client | data.updater | Data entry ... topics=['data:data'] dst_path='/scope_probe' ... has no topics matching the data topics, skipping
full server + client log
opal_server.data.data_update_publisher | INFO | Publishing data update to topics: {'data:data'}, reason: repro, entries: [{'url': 'http://opal_server:7002/healthcheck', 'method': 'PUT', 'path': '/scope_probe', 'inline_data': False, 'topics': ['data:data']}]
opal_common.topics.publisher | INFO | Publishing to topics: ['documents:data:data']
opal_client.data.updater | INFO | Subscribing to topics: ['documents:data:data']
opal_client.data.updater | DEBUG | Data entry url='http://opal_server:7002/healthcheck' config=None topics=['data:data'] dst_path='/scope_probe' save_method='PUT' data=None has no topics matching the data topics, skipping
Expected behavior
The update reaches the scope's clients. The entry-content topics should be scope-prefixed to {scope_id}:data:{topic}, same as the channel. Happy to open a PR.
OPAL version
master (76f898e); permitio/opal-server:latest tracks the same code.
Related: #779.
Describe the bug
A data update sent to a scope via
POST /scopes/{scope_id}/data/updatenever reaches that scope's OPAL clients; every entry is dropped, whatever topic you send.The endpoint rewrites each entry's topics to
data:{topic}(scopes/api.py#L350) and publishes throughScopedServerSideTopicPublisher, which scope-prefixes only the channel (publisher.py#L206), not the payload. So the message is delivered on channel{scope_id}:data:{topic}(the client is subscribed there), but each entry's owntopicsstaydata:{topic}. The scoped client filters entries withset(entry.topics).isdisjoint(self._data_topics)against its namespaced{scope_id}:data:{topic}(updater.py#L482), so the entry is skipped before any fetch. No topic value works: the server forcesdata:{topic}and the client only matches{scope_id}:data:{topic}. The globalPOST /data/configis not an alternative for scoped clients, since it publishes un-prefixed channels they are not subscribed to.To Reproduce
docker/docker-compose-scopes-example.yml, with the client pointed at scopedocuments:The channel matches (so it is delivered), the entry topics do not (so it is dropped):
full server + client log
Expected behavior
The update reaches the scope's clients. The entry-content topics should be scope-prefixed to
{scope_id}:data:{topic}, same as the channel. Happy to open a PR.OPAL version
master(76f898e);permitio/opal-server:latesttracks the same code.Related: #779.