Archives
-
Using Azure Event Hubs As A Message Hub For IoT Scenarios
When we build apps for larger IoT scenarios or working on Big Data solutions, one major challenge would be to handle millions of messages received from hundreds of systems where these messages will be subscribed by hundreds of other systems. To handle this kind of problem using our traditional enterprise messaging systems and publish-subscribe message brokers would be really a challenge as we need to handle massive amount of stream of data in a second, in which we may need to process the data and events in near real-time. Our most of the existing publish-subscribe message brokers would be failed when we deal with millions of messages in a second. Technologies like Apache Kafka solves this problem by providing a high-throughput distributed messaging system with a distributed commit log. Kafka is designed for cluster-centric design where you may need to leverage distributed coordination tools such as Apache ZooKeeper for coordinating and synchronizing the cluster nodes. This may leads to lot of operational complexity when working with larger distributed systems.