I was wondering if anyone has experience with the new kafka messaging in Progress ABL. I don't see it discussed much in this community.

I can see how it might be easy for a developer to send string/json data; but is there anything faster or smaller that could be used as a payload? Has anyone tried using protobuf or avro, for example?

I suspect that working with binary formats is a lot more challenging from a programmer standpoint. And it wouldn't necessary increase performance, when producing data from a single AVM session.

I started investigating, in order to create an integration with Databricks. Databricks now has a kafka-compliant API that can be used to transmit data to a lakehouse. It is part of their "zerobus-ingest" feature. I'm guessing that performance will max out before we reach 100k rows/minute on a simple table being sent from ABL, especially given the dependency on string/json data. Perhaps we can scale that up by using lots of parallel AVM sessions to produce records simultaneously?