Apache Kafka is reliable messaging infrastructure that is used in production at different companies. It communicates with other software through it's wire protocol.
mps (massive publish subscribe) is an eventing framework over the top of kafka that enables user-land communicate with kafka with the ambition of enabling event driven applications utilize kafka without having to worry about the wire-protocol. mps is open source and available at github.com/milindparikh/mps. It's still a baby; so be gentle. it is written in Erlang and rests on shoulders of giants... of particular mention is OTP, Riak_Core and Cowboy.
This post, which is about setting up the mps and seeing some toy demos, is the percursor to the next post; which will explain the technical underpinnings of mps. In the meantime and for the impatient, the source is the guide (at least those who grok Erlang).
.png)
Now open up two browsers sessions at localhost:8080 and localhost:8080/publish. You must subscribe to an event first within your subscription client at Topic, Key range. So in the screenshot below, you are subscribing to Topic1, Key1 in the subscription client. Then at the publishing client, you can publish a value that can be seen at the subscription client.
.png)
Trying something more adventurous, com.yahoo.sales.* at the subscription level and com.yahoo.sales.books with a value of 100 at the publishing level, provides a clearer concept of key filtering at the subscription and publishing level.
.png)
The important thing to note is that there are no kafka topics so far at all (and will not be in this section). This independence, as you shall see later, is the part of the key (pun not intended) to the eventing framework. You can actually broadcast a message as well like com.(yahoo|amazon).sales.books.
.png)
NEXT UP -- Introducing mps, an eventing framework for Apache Kafka 0.8 - PART II
mps (massive publish subscribe) is an eventing framework over the top of kafka that enables user-land communicate with kafka with the ambition of enabling event driven applications utilize kafka without having to worry about the wire-protocol. mps is open source and available at github.com/milindparikh/mps. It's still a baby; so be gentle. it is written in Erlang and rests on shoulders of giants... of particular mention is OTP, Riak_Core and Cowboy.
This post, which is about setting up the mps and seeing some toy demos, is the percursor to the next post; which will explain the technical underpinnings of mps. In the meantime and for the impatient, the source is the guide (at least those who grok Erlang).
Kicking the tires
Building of mps is mentioned in README.md at github.com/milindparikh/mps.
It does require some hand holding; but essentially you need to end "make rel" in a clean fashion. You must have started kafka on your localhost at 9092; unless you grok erlang (in which case you can change the code).
./rel/mps/bin/mps console will start you off with a set of 64 vnodes on which ~65000 htopics (which are very different than topics in kafka) will be deployed. It also comes with web-server at localhost:8080
At the command prompt of the console, you must type (with the period):
mps:create_stream().
This deploys the 65000 htopics on the 64 vnodes and it takes time ( 30 or so seconds; at least on my ubuntu powered virtual machine ); even it is assures you by saying [ok|ok.....].
.png)
Now open up two browsers sessions at localhost:8080 and localhost:8080/publish. You must subscribe to an event first within your subscription client at Topic, Key range. So in the screenshot below, you are subscribing to Topic1, Key1 in the subscription client. Then at the publishing client, you can publish a value that can be seen at the subscription client.
.png)
Trying something more adventurous, com.yahoo.sales.* at the subscription level and com.yahoo.sales.books with a value of 100 at the publishing level, provides a clearer concept of key filtering at the subscription and publishing level.
.png)
The important thing to note is that there are no kafka topics so far at all (and will not be in this section). This independence, as you shall see later, is the part of the key (pun not intended) to the eventing framework. You can actually broadcast a message as well like com.(yahoo|amazon).sales.books.
.png)
NEXT UP -- Introducing mps, an eventing framework for Apache Kafka 0.8 - PART II
No comments:
Post a Comment