[PoC] Setup docker machine with Kafka and use Microck to publish from Spec

[bluemonk3y]

1. Docker compose a Kafka env with schema registry
2. Run SpecMesh against the env via CLI
3. Run Microck against the 'flattened' spec to publish AVRO data into Kafka
4. Use kafka-console-consumer to verify data is being generated

[bluemonk3y]

Things to explore:

• https://www.baeldung.com/ops/kafka-docker-setup
• microcks docker compose (runs kafka and everything) https://microcks.io/documentation/installing/docker-compose/

[bluemonk3y]

Hello @HorseBandit - welcome!

Pls have a look at this branch and follow the steps on here?

• branch: https://github.com/specmesh/datamesh-slimstack/tree/1-dockercompose-kakfa
• blog: https://microcks.io/blog/async-features-with-docker-compose/

[jvestell]

@bluemonk3y Thank you! Proceeding.

[bluemonk3y]

Due to Microcks terrible Kafka data generator (using ts-millis for key) - I'm parking this for now. I'm going to look at datagen from specs leveraging:

• https://github.com/awslabs/amazon-msk-data-generator
• https://github.com/confluentinc/avro-random-generator

[navery-max]

this: https://github.com/specmesh/kafka-random-generator