Download binary or non-UTF-8 encoded data

[sebastiangaiser]

Hey,

depending on the format of Kafka messages, the console can display it e.g. text, json, or not, e.g. Avro Protobuf.
For decoding Avro or Protobuf, a registry would be required. On the other hand, operating a registry is additional work.
For simple debugging, it would be helpful to provide the possibility to simply download the Kafka message(s) and decode it later.

WDYT? For us this would be a helpful addition.

Thanks in advance 👋🏻

[MikeEdgar]

This is a reasonable request. The key and value could likely be made available for any record, it simply would not be displayed in UI when it is binary without a usable schema.

[dyson-025]

Hi! I'd like to work on this if it's available.

My understanding is that the goal is to allow downloading the raw key/value bytes even when the UI can't decode or display the message (e.g. binary, Avro, or Protobuf without a schema registry). My plan is to expose the raw data for download while keeping the existing rendering behaviour unchanged.

Please let me know if that approach aligns with what you have in mind. Thanks!

[MikeEdgar]

My thinking on this one is that we can base-64 encode the key/value when non-text characters are detected in the data from the Kafka topic (RecordData class). The meta object for the record can also be updated to include the content type - either text/plain or application/octet-stream and optionally the encoding base64.

With that information, the UI could be updated to either allow download of the data (creating a data URL) or display the existing message about binary data (e.g. in the table of records).

{
  "data": [{
    "type": "records",
    "meta": {
       "content": {
         "key": { /* Assumed text/plain if missing/undefined? */
           "type": "text/plain"
         },
         "value": {
           "type": "application/octet-stream",
           "encoding": "base64"
         }
       }
    },
    "attributes": {
      "key": "plain text value of the key",
      "value": "<Base64-encoded value of the value>"
    }
  }]
}

[dyson-025]

Thanks for the detailed guidance! That approach makes sense to me.

I'll start by looking at the RecordData class and update the response model to include the content type and optional Base64 encoding metadata, then make the corresponding UI changes to support downloading binary content while preserving the current behaviour for text data.

I'll put together a draft implementation and open a PR for review. Thanks!